Pricing

from $1.00 / 1,000 results

Hacker News Search & Monitor

Search and monitor Hacker News stories, comments, Ask HN, Show HN, jobs and users. No auth, clean structured JSON. Filter by points, date, author, and tags.

Pricing

from $1.00 / 1,000 results

Rating

0.0

(0)

Developer

Nicolas van Arkens

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

.dockerignore

.git
.venv
__pycache__
*.pyc
storage
test_*.py

requirements.txt

1apify < 3.0
2pydantic >= 2.8, < 2.12
3browserforge == 1.2.3
4httpx

src/main.py

1"""Hacker News Search & Monitor — Apify Actor.
2
3Uses the free, no-auth Algolia Hacker News Search API:
4  https://hn.algolia.com/api/v1/search          (sorted by relevance: points & comments)
5  https://hn.algolia.com/api/v1/search_by_date  (sorted by date: most recent first)
6
7API contract verified against a live response. Relevant fields per hit:
8  objectID, title, url, author, points, num_comments, created_at,
9  created_at_i, story_id, story_title, story_url, comment_text, _tags
10Top-level: hits, nbHits, nbPages, page, hitsPerPage
11"""
12
13from __future__ import annotations
14
15import asyncio
16from datetime import datetime, timezone
17from urllib.parse import urlencode
18
19import httpx
20from apify import Actor
21
22BASE_RELEVANCE = "https://hn.algolia.com/api/v1/search"
23BASE_DATE = "https://hn.algolia.com/api/v1/search_by_date"
24HN_ITEM_URL = "https://news.ycombinator.com/item?id={}"
25HN_USER_URL = "https://news.ycombinator.com/user?id={}"
26PAGE_SIZE = 100  # Algolia max hitsPerPage is 1000, but 100 keeps requests light and reliable.
27
28# Maps the user-facing contentType to Algolia `tags`.
29CONTENT_TYPE_TAGS = {
30    "story": "story",
31    "comment": "comment",
32    "ask_hn": "ask_hn",
33    "show_hn": "show_hn",
34    "poll": "poll",
35    "job": "job",
36    "all": "(story,comment)",
37}
38
39
40def _date_to_timestamp(date_str: str) -> int | None:
41    """Parse a YYYY-MM-DD string into a unix timestamp (UTC). Returns None if unparseable."""
42    if not date_str:
43        return None
44    for fmt in ("%Y-%m-%d", "%Y/%m/%d", "%d-%m-%Y"):
45        try:
46            dt = datetime.strptime(date_str.strip(), fmt).replace(tzinfo=timezone.utc)
47            return int(dt.timestamp())
48        except ValueError:
49            continue
50    return None
51
52
53def build_tags(content_type: str, author: str | None) -> str:
54    """Combine content-type tag with an optional author filter (ANDed)."""
55    base = CONTENT_TYPE_TAGS.get(content_type, "story")
56    if author:
57        author_tag = f"author_{author.strip()}"
58        # Algolia ANDs comma-separated tags. Parenthesised groups stay intact.
59        return f"{base},{author_tag}"
60    return base
61
62
63def build_numeric_filters(
64    min_points: int | None,
65    min_comments: int | None,
66    created_after_ts: int | None,
67    created_before_ts: int | None,
68) -> str:
69    """Build the Algolia numericFilters string from the active constraints."""
70    filters: list[str] = []
71    if min_points is not None:
72        filters.append(f"points>={min_points}")
73    if min_comments is not None:
74        filters.append(f"num_comments>={min_comments}")
75    if created_after_ts is not None:
76        filters.append(f"created_at_i>={created_after_ts}")
77    if created_before_ts is not None:
78        filters.append(f"created_at_i<{created_before_ts}")
79    return ",".join(filters)
80
81
82def transform_hit(hit: dict, include_text: bool) -> dict:
83    """Convert a raw Algolia hit into a clean, stable output record.
84
85    Pure function — no I/O — so it can be unit-tested against captured fixtures.
86    """
87    tags = hit.get("_tags", []) or []
88    # Determine item type from tags (more reliable than guessing from fields).
89    if "comment" in tags:
90        item_type = "comment"
91    elif "poll" in tags:
92        item_type = "poll"
93    elif "job" in tags:
94        item_type = "job"
95    elif "show_hn" in tags:
96        item_type = "show_hn"
97    elif "ask_hn" in tags:
98        item_type = "ask_hn"
99    else:
100        item_type = "story"
101
102    object_id = hit.get("objectID")
103
104    record = {
105        "id": object_id,
106        "type": item_type,
107        "title": hit.get("title") or hit.get("story_title"),
108        "url": hit.get("url") or hit.get("story_url"),
109        "author": hit.get("author"),
110        "points": hit.get("points"),
111        "numComments": hit.get("num_comments"),
112        "createdAt": hit.get("created_at"),
113        "createdAtTimestamp": hit.get("created_at_i"),
114        "storyId": hit.get("story_id"),
115        "hnUrl": HN_ITEM_URL.format(object_id) if object_id else None,
116        "authorUrl": HN_USER_URL.format(hit["author"]) if hit.get("author") else None,
117    }
118
119    if include_text:
120        # Self-post body lives in story_text; comment body in comment_text.
121        record["text"] = hit.get("comment_text") or hit.get("story_text")
122
123    return record
124
125
126async def main() -> None:
127    async with Actor:
128        actor_input = await Actor.get_input() or {}
129
130        query = (actor_input.get("query") or "").strip()
131        content_type = actor_input.get("contentType", "story")
132        sort_by = actor_input.get("sortBy", "relevance")
133        max_items = int(actor_input.get("maxItems", 100))
134        min_points = actor_input.get("minPoints")
135        min_comments = actor_input.get("minComments")
136        author = actor_input.get("author")
137        include_text = actor_input.get("includeCommentText", True)
138
139        created_after_ts = _date_to_timestamp(actor_input.get("createdAfter", ""))
140        created_before_ts = _date_to_timestamp(actor_input.get("createdBefore", ""))
141
142        base_url = BASE_DATE if sort_by == "date" else BASE_RELEVANCE
143        tags = build_tags(content_type, author)
144        numeric_filters = build_numeric_filters(
145            min_points if isinstance(min_points, int) else None,
146            min_comments if isinstance(min_comments, int) else None,
147            created_after_ts,
148            created_before_ts,
149        )
150
151        Actor.log.info(
152            f"Searching HN | query={query!r} type={content_type} sort={sort_by} "
153            f"tags={tags} numericFilters={numeric_filters or '(none)'} maxItems={max_items}"
154        )
155
156        pushed = 0
157        page = 0
158        total_pages: int | None = None
159
160        async with httpx.AsyncClient(timeout=30.0, headers={"User-Agent": "apify-hn-search/0.1"}) as client:
161            while pushed < max_items:
162                params = {
163                    "tags": tags,
164                    "hitsPerPage": min(PAGE_SIZE, max_items - pushed),
165                    "page": page,
166                }
167                if query:
168                    params["query"] = query
169                if numeric_filters:
170                    params["numericFilters"] = numeric_filters
171
172                url = f"{base_url}?{urlencode(params)}"
173
174                # Retry a few times with backoff for transient errors / rate limits.
175                data = None
176                for attempt in range(1, 4):
177                    try:
178                        resp = await client.get(url)
179                        if resp.status_code == 429:
180                            wait = attempt * 3
181                            Actor.log.warning(f"Rate limited (429). Backing off {wait}s...")
182                            await asyncio.sleep(wait)
183                            continue
184                        resp.raise_for_status()
185                        data = resp.json()
186                        break
187                    except (httpx.HTTPError, ValueError) as exc:
188                        Actor.log.warning(f"Request attempt {attempt} failed: {exc}")
189                        if attempt == 3:
190                            Actor.log.error(f"Giving up on page {page} after 3 attempts.")
191                        else:
192                            await asyncio.sleep(attempt * 2)
193
194                if data is None:
195                    break
196
197                hits = data.get("hits", [])
198                if total_pages is None:
199                    total_pages = data.get("nbPages", 0)
200                    Actor.log.info(
201                        f"Matched {data.get('nbHits', 0)} items across {total_pages} pages."
202                    )
203
204                if not hits:
205                    Actor.log.info("No more results.")
206                    break
207
208                batch = []
209                for hit in hits:
210                    if pushed >= max_items:
211                        break
212                    batch.append(transform_hit(hit, include_text))
213                    pushed += 1
214
215                if batch:
216                    # Billing is handled automatically by the synthetic
217                    # `apify-default-dataset-item` event configured in the Console:
218                    # every item pushed to the default dataset is charged. No explicit
219                    # charge call is needed here.
220                    await Actor.push_data(batch)
221                    Actor.log.info(f"Pushed {pushed}/{max_items} items (page {page}).")
222
223                page += 1
224                if total_pages is not None and page >= total_pages:
225                    Actor.log.info("Reached last available page.")
226                    break
227
228        Actor.log.info(f"Done. Returned {pushed} items.")

src/main.py

1import asyncio
2from .main import main
3
4asyncio.run(main())

.actor/actor.json

{
    "actorSpecification": 1,
    "name": "hacker-news-search",
    "title": "Hacker News Search & Monitor",
    "description": "Search and monitor Hacker News stories, comments, Ask HN, Show HN, jobs and users. No auth, clean structured JSON. Filter by points, date, author, and tags.",
    "version": "0.1",
    "buildTag": "latest",
    "meta": {
        "templateId": "python-start"
    },
    "input": "./input_schema.json",
    "dockerfile": "./Dockerfile",
    "storages": {
        "dataset": "./dataset_schema.json"
    }
}

.actor/dataset_schema.json

{
    "actorSpecification": 1,
    "views": {
        "overview": {
            "title": "Overview",
            "transformation": {
                "fields": [
                    "title",
                    "type",
                    "author",
                    "points",
                    "numComments",
                    "createdAt",
                    "url",
                    "hnUrl"
                ]
            },
            "display": {
                "component": "table",
                "properties": {
                    "title": { "label": "Title", "format": "text" },
                    "type": { "label": "Type", "format": "text" },
                    "author": { "label": "Author", "format": "text" },
                    "points": { "label": "Points", "format": "number" },
                    "numComments": { "label": "Comments", "format": "number" },
                    "createdAt": { "label": "Created", "format": "text" },
                    "url": { "label": "Link", "format": "link" },
                    "hnUrl": { "label": "HN Discussion", "format": "link" }
                }
            }
        }
    }
}

.actor/Dockerfile

FROM apify/actor-python:3.12

COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

COPY . ./

CMD ["python", "-m", "src"]

.actor/input_schema.json

{
    "title": "Hacker News Search & Monitor",
    "type": "object",
    "schemaVersion": 1,
    "properties": {
        "query": {
            "title": "Search query",
            "type": "string",
            "description": "Keyword(s) to search for across Hacker News. Leave empty to fetch the latest/front-page items by date without a keyword filter (useful for monitoring a feed).",
            "editor": "textfield",
            "prefill": "AI"
        },
        "contentType": {
            "title": "Content type",
            "type": "string",
            "description": "Which kind of items to return.",
            "enum": ["story", "comment", "ask_hn", "show_hn", "poll", "job", "all"],
            "enumTitles": ["Stories", "Comments", "Ask HN", "Show HN", "Polls", "Jobs", "All (stories + comments)"],
            "default": "story"
        },
        "sortBy": {
            "title": "Sort by",
            "type": "string",
            "description": "Relevance ranks by points & comments (best for discovery). Date returns most recent first (best for monitoring/alerts).",
            "enum": ["relevance", "date"],
            "enumTitles": ["Relevance (points & comments)", "Date (most recent first)"],
            "default": "relevance"
        },
        "maxItems": {
            "title": "Maximum items",
            "type": "integer",
            "description": "How many items to return in total. The actor paginates automatically.",
            "default": 100,
            "minimum": 1,
            "maximum": 10000
        },
        "minPoints": {
            "title": "Minimum points",
            "type": "integer",
            "description": "Only return stories with at least this many points. Leave empty for no minimum. (Applies to stories; comments do not have points.)",
            "minimum": 0,
            "unit": "points"
        },
        "minComments": {
            "title": "Minimum comments",
            "type": "integer",
            "description": "Only return stories with at least this many comments. Leave empty for no minimum.",
            "minimum": 0,
            "unit": "comments"
        },
        "author": {
            "title": "Author username",
            "type": "string",
            "description": "Only return items written by this Hacker News username (e.g. 'pg'). Leave empty for all authors.",
            "editor": "textfield"
        },
        "createdAfter": {
            "title": "Created after",
            "type": "string",
            "description": "Only return items created on or after this date. Format: YYYY-MM-DD.",
            "editor": "datepicker"
        },
        "createdBefore": {
            "title": "Created before",
            "type": "string",
            "description": "Only return items created before this date. Format: YYYY-MM-DD.",
            "editor": "datepicker"
        },
        "includeCommentText": {
            "title": "Include full comment/story text",
            "type": "boolean",
            "description": "Include the full body text of comments and self-posts. Turn off for a lighter, metadata-only dataset.",
            "default": true
        }
    },
    "required": []
}

.actor/README.md

1# Hacker News Search & Monitor 🔎
2
3Search and monitor **Hacker News** — stories, comments, Ask HN, Show HN, jobs, polls, and users — and get clean, structured JSON. No login, no API key, no proxies required. Built on the free official Algolia HN Search index, so it's fast and reliable.
4
5Use it to **track mentions of your company, product, or competitors**, find **Show HN / Ask HN** discussions, monitor a topic over time, or build a Hacker News dataset for research and LLM pipelines.
6
7## What you can do
8
9- 🔍 **Keyword search** across all of Hacker News (titles, self-text, and comments)
10- 🆕 **Monitor by date** — get the most recent matching items first, perfect for alerts and brand monitoring
11- 🎯 **Filter** by content type (story, comment, Ask HN, Show HN, job, poll), minimum points, minimum comments, author, and date range
12- 🧹 **Clean output** — one tidy record per item with direct links to both the source URL and the HN discussion
13- ⚡ **Fast & lightweight** — pure API, no headless browser
14
15## Example use cases
16
17- **Brand / competitor monitoring:** search your company name sorted by date, schedule it daily, and pipe new mentions to Slack, email, or a sheet.
18- **Lead & trend research:** find the highest-scoring stories about a technology or market in a date window.
19- **Show HN / Ask HN discovery:** surface new launches or questions in your niche.
20- **Datasets for AI:** pull thousands of HN comments on a topic for sentiment analysis or RAG.
21
22## Input
23
24| Field | Description |
25|-------|-------------|
26| **Search query** | Keyword(s) to search for. Leave empty to fetch the latest items by date (feed monitoring). |
27| **Content type** | Stories, Comments, Ask HN, Show HN, Polls, Jobs, or All. |
28| **Sort by** | Relevance (points & comments) or Date (most recent first). |
29| **Maximum items** | Total items to return; paginates automatically. |
30| **Minimum points / comments** | Only return stories above a threshold. |
31| **Author** | Restrict to a single HN username. |
32| **Created after / before** | Date range filter (YYYY-MM-DD). |
33| **Include full text** | Include comment/self-post body text, or keep it metadata-only. |
34
35## Output
36
37Each item is returned as a record like this:
38
39```json
40{
41  "id": "48126840",
42  "type": "story",
43  "title": "A Great Essay",
44  "url": "https://example.com/essay",
45  "author": "pg",
46  "points": 412,
47  "numComments": 233,
48  "createdAt": "2026-05-13T20:11:17Z",
49  "createdAtTimestamp": 1778703077,
50  "storyId": 48126840,
51  "hnUrl": "https://news.ycombinator.com/item?id=48126840",
52  "authorUrl": "https://news.ycombinator.com/user?id=pg",
53  "text": null
54}
55```
56
57Export to JSON, CSV, or Excel, or pull results via the Apify API. Connect to Slack, Google Sheets, Zapier, or Make for automated monitoring.
58
59## How it works
60
61This actor queries the public Algolia Hacker News Search API (`hn.algolia.com/api/v1`), the same index that powers HN's own search. It paginates through results, applies your filters server-side where possible, normalizes every item into a consistent schema, and stores it in your dataset. It does not require — and never asks for — any credentials.
62
63## Notes
64
65- This actor accesses only publicly available Hacker News data via a free public API. It is an independent tool and is not affiliated with Y Combinator or Hacker News.
66- Algolia caps results at the most recent/relevant matches per query; for very large historical pulls, narrow by date range and run in windows.

HackerNews Monitor + Auto-Tagger | Free Preview

w4rd0g/hackernews-monitor

Search and monitor HackerNews stories, Show HN, Ask HN, polls, jobs by keyword and date range. Auto-tag every post with your custom keyword list. For trend detection, market intel, AI launch monitoring. FREE during launch preview.

Polara Data

Hacker News Search — Stories, Comments & Developer Sentiment

ryanclinton/hackernews-search

Search and extract stories, comments, polls, Show HN, and Ask HN posts from Hacker News. This actor uses the Algolia HN Search API to find content by keyword, filter by author, date range, minimum points, and comment count -- then returns clean, structured JSON ready for analysis, monitoring, or ...

Ryan Clinton

MCP Server: HackerNews — Search, Comments, Show HN

dltik/mcp-server-hackernews

Apify MCP server for AI agents: 6 tools to query HackerNews via Algolia API. Search stories/comments, top front page, Show HN trending, user profiles. Free official API, $0.005/call.

dltik

Research Tools Mcp

halilc4/research-tools-mcp

All-in-one MCP server for research. See how AI describes your brand. Track citations across ChatGPT, Claude, Gemini & Perplexity in real-time. Plus 30+ tools for SEO, keyword research, SERP analysis, trends, social listening & ad intelligence. Token-efficient MCP server. Pay only for what you use.

Igor Halilovic

1.0

(1)

Desearch AI Search

desearch/ai-search

allows you to perform AI-powered web searches, gathering relevant information from multiple sources, including web pages, research papers, and social media discussions.

Desearch

100

🔍 Google SERP Scraper — Rankings & Keywords

nexgendata/google-search-scraper

Scrape Google Search results — organic listings, featured snippets, People Also Ask, knowledge panels. SERP analysis at scale.

Stephan Corbeil

Hackernews Intelligence

viralanalyzer/hackernews-intelligence

Scrape Hacker News stories, comments, and discussions. Track tech trends, startup news, and developer community sentiment.

viralanalyzer

5.0

(3)

Signalfinder Pro

vamsi-krishna/signalfinder-pro

Find high-intent B2B leads from public sources (Hacker News, GitHub, Stack Exchange, Reddit, RSS). Classifies buying intent, scores leads 0–100, and optionally generates outreach. No paid APIs required.