Hacker News Scraper & Keyword Monitor (Stories, Comments)

By Harlo. Unofficial tool. Not affiliated with or endorsed by Y Combinator, Hacker News or Algolia.

Search and monitor Hacker News for your keywords, brand, product or competitors — and for any story that links to your domain. Data comes from the official HN Search API (hn.algolia.com, the search behind news.ycombinator.com). You get one clean row per story or comment with points, comment count, date, HN link, the story a comment belongs to, and plain-text content. Schedule it to receive only new items.

What you get

• Keyword search across titles, URLs, story text and comments — up to 25 keywords/domains per run, each row tagged with the matchedQueries that found it (duplicates merged).
• Domain mentions: domains: ["yourcompany.com"] returns stories that link to that site or its subdomains (checked against the real link host).
• Exact keyword matching by default: HN Search is typo-tolerant, so a plain search for rust also returns comments saying "just" or "must" (in one 12-hour test: 898 comment hits fuzzy vs 25 exact). This Actor turns typo tolerance off unless you set typoTolerance: true.
• Content types: stories (incl. Show HN / Ask HN), comments, Show HN only, Ask HN only, polls.
• Modes: newest first (monitoring), most relevant (research, all-time with lookbackHours: 0), or what's on the front page right now.
• Filters: lookback window in hours, minimum points, minimum comments (stories).
• Clean text: HTML removed, links kept, HTML entities decoded, email-like strings replaced with [email removed], optional length limit.
• Monitor mode (onlyNewSinceLastRun): first run is a baseline; later runs return only new items (and changed ones if a title, URL or text was edited, with previous). Points and comment counts change all the time and do not trigger a re-delivery.
• Compact output for spreadsheets, Slack/Zapier alerts and LLM pipelines.
• Run summary in OUTPUT: per-search hits, pages read, why each search stopped, truncated flag with a plain-English note.

Empty input works: it returns the newest stories from the last 24 hours.

Input example

{
  "queries": ["apify", "web scraping"],
  "domains": ["github.com"],
  "contentTypes": ["story", "comment"],
  "lookbackHours": 48,
  "minPoints": 0,
  "maxItems": 50,
  "onlyNewSinceLastRun": true
}

Output example (full row)

{
  "id": "50313998",
  "kind": "story",
  "title": "Live benchmark of web scraping APIs and residential proxies (2026)",
  "url": "https://github.com/tianjun588/llm-web-scraping-benchmark",
  "domain": "github.com",
  "hnUrl": "https://news.ycombinator.com/item?id=50313998",
  "points": 1, "numComments": 0,
  "createdAt": "2026-10-31T07:39:18.000Z",
  "storyId": "50313998",
  "storyTitle": "Live benchmark of web scraping APIs and residential proxies (2026)",
  "storyUrl": "https://github.com/tianjun588/llm-web-scraping-benchmark",
  "parentId": null, "onFrontPage": false,
  "author": "tianjun588", "text": null,
  "matchedQueries": ["web scraping", "domain:github.com"],
  "updateType": "new", "previous": null,
  "changeHash": "…", "fetchedAt": "2026-10-31T07:45:01.000Z"
}

Comment rows have kind: "comment", text, storyTitle, storyUrl, parentId; title, url, points and numComments are null (HN Search does not expose comment points).

Pricing

Pay per event: $0.00005 per Actor start + a small fee per dataset row (event item). Only delivered rows are charged, and the run's maximum total charge is respected. See the Pricing tab for the current price.

FAQ

How does monitor mode work? Turn on onlyNewSinceLastRun and schedule the Actor (e.g. hourly or daily). State lives in a named key-value store, hacker-news-keyword-monitor-state, in your own account — one record per input set (keywords, domains, types, mode, lookback, filters). Rows cut by maxItems or your cost limit are delivered on the next run while they are still inside lookbackHours. Keep lookbackHours at least as long as your schedule interval.

How far back can I search? Set lookbackHours: 0 for all-time. HN Search returns at most 1,000 hits per keyword and content type; narrow the keyword or use minPoints for big topics (truncated tells you when this happens).

Can I use it from an AI agent / MCP? Yes — call it through the Apify MCP server (mcp.apify.com) or the API; use outputFormat: "compact" and a small maxItems to keep payloads small.

What about personal data? Only public HN usernames are included (includeAuthor: false removes them). No user profiles, karma or "about" text are fetched, and email-like strings in post text are redacted. Use the data in line with Hacker News' and Algolia's terms and applicable privacy law.

Is it official? No. It uses the public, keyless HN Search API by Algolia (documented at hn.algolia.com/api) at a polite pace (≥ 250 ms between requests, retries with backoff on 429/5xx, per-run request cap). It does not scrape news.ycombinator.com pages, so HN's robots.txt crawl rules for the website are not involved. No login, no API key. Client identifier: hacker-news-keyword-monitor/0.1 (Apify Actor by Harlo; unofficial; digitalpromohub.support@gmail.com).

Limits: HN Search indexes new items within minutes but not instantly; deleted/dead items may be missing; matching is HN Search's word matching (multi-word keywords must all appear, not necessarily as an exact phrase).

Support: digitalpromohub.support@gmail.com