Hacker News Scraper & Keyword Monitor (Stories + Comments)

Search Hacker News stories and comments by keyword and get back only the items that really mention your keyword. Turn on monitor mode and every scheduled run returns just the new mentions since the last run, ready for Slack, email, Google Sheets or a webhook. It uses the official public HN Search API (hn.algolia.com). No login, no API key, no proxy.

What does Hacker News Scraper & Keyword Monitor do?

• Searches HN stories, comments, Ask HN, Show HN, job posts and polls for one or more keywords, newest first.
• Exact mentions only (on by default). The raw HN search is typo-tolerant and also matches usernames. For apify over the last 365 days (measured 2026-09-24), the raw search returned 105 items and this Actor's exact filter kept 73. The other 32 were words like ipify, appify and ApiFB, or authors whose name looks like the keyword.
• Monitor mode: remembers what it already returned (in a key-value store in your account) and returns only new items next time. You can keep several monitors apart with a Monitor ID.
• One row per HN item, even when it matches several keywords (matchedKeywords lists all of them). Each row also records where the match was found (title, url or text).
• Comments come with their story title, story URL and story link, so an alert makes sense without opening HN.
• Adds a short snippet around the match. HTML is converted to plain text.

Who is it for? Use cases

• Brand and product monitoring. Founders, DevRel and marketing teams see when their product, domain or company is discussed, and can reply while the thread is still active.
• Competitor tracking. Collect every mention of competing tools and read what users praise or complain about.
• Launch tracking. Follow the Show HN and the comments after you launch.
• Market and technology research. Count how often a library, framework or topic comes up over months or years. The index goes back to 2006.
• Lead and hiring signals. Find "Ask HN: Who wants to be hired?" comments or job posts that mention a skill (for example "data engineer" remote).
• LLM and RAG pipelines. Clean JSON text with links, ready for summarisation or sentiment analysis.

Why use this Actor?

	This Actor	Typical HN scrapers
Stories and comments	Yes	Often stories only
Exact-mention filter (no typo/author matches)	Yes, on by default	No, raw search results
Only new items since the last run	Built-in monitor mode	Rerun and de-duplicate yourself
Comment rows include story title/URL	Yes	Varies
Several keywords in one run, merged	Yes (matchedKeywords)	One query per run is common
Proxy / login / API key	None	None

What data do you get?

Field	Description
type	story, comment, ask_hn, show_hn, job or poll
id, hnUrl	HN item ID and link
title, url	Post title and external link (stories). null for comments
author	HN username
points, numComments	Score and comment count (stories). HN does not publish comment scores
createdAt	ISO 8601, UTC
storyId, storyTitle, storyUrl, storyHnUrl	The story the item belongs to (for comments: the parent story)
parentId	Direct parent (comments)
matchedKeywords	Every keyword of your input that this item matched
matchedIn	Where the match was found: title, url, text
snippet	About 220 characters around the match
text	Full plain text (switch off with includeText: false)
changeType	new in monitor mode, otherwise null
scrapedAt	When the run fetched the item

A RUN_SUMMARY record in the key-value store shows, per keyword, how many API hits were scanned, how many matched, how many were new, and any error.

How much does it cost?

Pay per event, and the platform usage is included:

Event	Price
Item returned (apify-default-dataset-item)	$0.001 ($1 per 1,000 items)
Keyword searched (keyword-checked, once per keyword per run)	$0.001
Actor start	$0.00005

Examples, calculated from the prices above:

Scenario	Calculation	Cost
1 keyword, latest 100 mentions	0.00005 + 0.001 + 100 × 0.001	≈ $0.10
5 keywords, 1 year of history, 2,000 items	0.00005 + 5 × 0.001 + 2,000 × 0.001	≈ $2.01
Daily monitor, 5 keywords, about 10 new mentions a day, 30 days	30 × (0.00005 + 0.005) + 300 × 0.001	≈ $0.45 / month
Hourly monitor, 5 keywords, about 10 new mentions a day, 30 days	720 × (0.00005 + 0.005) + 300 × 0.001	≈ $3.94 / month

You can set Max cost per run in the run options. The Actor reads it and stops returning items before it would go over.

How to use it

1. Click Try for free (an Apify account is free).
2. Enter your keywords, one per line. Put phrases in double quotes, for example "vector database".
3. Choose the content types and the look-back window.
4. Click Start. Download the results as JSON, CSV, Excel or HTML, or use the API.
5. For alerts: turn on Monitor mode, save the run as a Task, add a Schedule (hourly or daily), and connect an integration (Slack, email, Google Sheets, Zapier, Make, webhook).

Input example

{
  "keywords": ["apify", "\"web scraping\""],
  "contentTypes": ["story", "comment"],
  "daysBack": 30,
  "maxItemsPerKeyword": 10,
  "exactMatch": true,
  "minPoints": 0,
  "includeText": true,
  "monitorMode": false,
  "monitorId": "default"
}

Keyword syntax: all words are required (AND). Matching ignores case and requires whole words (go does not match google). A quoted phrase must appear as written, and hyphens count as spaces ("open source" matches open-source).

Output example

The following output was taken from a real run on 2026-09-24 with the input above (text shortened). A comment:

{
  "type": "comment",
  "id": "49825437",
  "hnUrl": "https://news.ycombinator.com/item?id=49825437",
  "title": null,
  "url": null,
  "author": "tiew9Vii",
  "points": null,
  "numComments": null,
  "createdAt": "2026-09-24T02:22:13Z",
  "storyId": "49822654",
  "storyTitle": "OpenAI agents hacked Australian Medicare system",
  "storyUrl": "https://www.reuters.com/world/asia-pacific/australia-pm-albanese-says-openai-breached-medicare-sydney-morning-herald-2026-09-23/",
  "storyHnUrl": "https://news.ycombinator.com/item?id=49822654",
  "parentId": "49825138",
  "matchedKeywords": ["\"web scraping\""],
  "matchedIn": ["text"],
  "snippet": "…organisation, the prime minister had “frank” words while an individual faces jail time for using open AI for web scraping https://www.abc.net.au/news/2026-09-10/christopher-duff-to-s...",
  "text": "In a large portion of the civilised world, there’s strict gun controls, …",
  "changeType": null,
  "scrapedAt": "2026-09-24T14:24:12Z"
}

A Show HN post (input {"keywords": ["rust"], "contentTypes": ["show_hn"], "daysBack": 30, "minPoints": 10, "includeText": false}):

{
  "type": "show_hn",
  "id": "49804217",
  "hnUrl": "https://news.ycombinator.com/item?id=49804217",
  "title": "Show HN: graf (1000x faster graphify in Rust)",
  "url": "https://github.com/ctxrs/graf",
  "author": "ripped_britches",
  "points": 12,
  "numComments": 3,
  "createdAt": "2026-09-22T16:46:31Z",
  "storyId": "49804217",
  "storyTitle": "Show HN: graf (1000x faster graphify in Rust)",
  "storyUrl": "https://github.com/ctxrs/graf",
  "storyHnUrl": "https://news.ycombinator.com/item?id=49804217",
  "parentId": null,
  "matchedKeywords": ["rust"],
  "matchedIn": ["title", "text"],
  "snippet": "…I dug into it and saw an opportunity to make it better. It is rewritten in rust and uses sqlite instead of NetworkX. …",
  "changeType": null,
  "scrapedAt": "2026-09-24T14:24:14Z"
}

Points and comment counts are the values at the time of the run.

Monitor mode

• The first run returns the newest maxItemsPerKeyword items from the last daysBack days. Each later run with the same Monitor ID returns only items it has not returned before, with changeType: "new".
• When nothing new has appeared, the run succeeds with 0 items. On a pay-per-event plan you then pay only the start and keyword events.
• Each later run searches again from one hour before the newest item it saw, so items that reach the search index late are still caught. Duplicates are removed.
• If a run stops at your max cost per run, the items it did not return are not marked as seen, so the next run returns them.
• If you add a keyword to an existing monitor, its first run behaves like a first run.
• The saved state is in the key-value store hacker-news-monitor-state in your Apify account. Delete the record monitor-<ID> to start over.

Tested on 2026-09-24. The first monitor run returned 203 items. A second run a moment later returned 0. After 3 remembered IDs were removed from the state on purpose, the next run returned exactly those 3 as new.

Tips

• Use distinctive keywords. "acme cloud" or acmecloud.com give cleaner results than a common word.
• Search your domain name too. Links posted to HN match through the url field.
• minPoints (for example 50) turns the Actor into a filter for popular stories on a topic.
• Set includeText: false when you only need links and snippets.
• For long periods, raise maxItemsPerKeyword. The Actor reads the index in pages of 1,000, newest first. In a test it went back 2 years for rust and returned 2,500 items in about 6 seconds.

Integrations and API

Run it from your own code with the Apify API clients.

Python:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("panda_studio/hacker-news-monitor").call(run_input={
    "keywords": ["your-product", "\"your company\""],
    "monitorMode": True,
    "monitorId": "brand",
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["createdAt"], item["type"], item["hnUrl"], item["snippet"])

Node.js:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });
const run = await client.actor('panda_studio/hacker-news-monitor').call({
    keywords: ['your-product'],
    contentTypes: ['story', 'comment'],
    daysBack: 7,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

cURL (runs the Actor and returns the items in one call):

curl -X POST "https://api.apify.com/v2/acts/panda_studio~hacker-news-monitor/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keywords": ["apify"], "daysBack": 7}'

AI agents (MCP): add https://mcp.apify.com?tools=panda_studio/hacker-news-monitor to Claude, Cursor or VS Code. For Claude Code:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=panda_studio/hacker-news-monitor"

Is it legal to scrape Hacker News?

This Actor does not scrape HN pages. It uses the HN Search API (https://hn.algolia.com/api/v1/), which Algolia runs for Y Combinator and documents publicly at hn.algolia.com/api. The API needs no key and returns public posts and comments. The Actor waits between requests and retries politely when the API asks it to slow down (HTTP 429). The data includes public HN usernames. If you store or republish the data, follow the rules that apply to you, such as GDPR and HN's own guidelines. This is not legal advice.

FAQ

How is this different from the HN search box? It runs several keywords at once, drops typo and username matches, adds the story context to comments, returns only new items on a schedule, and exports to CSV or JSON, or sends the results to your tools.

Can I get results in real time? You can schedule the Actor as often as every few minutes. It can only see items once they are in the HN Search index, and each run re-checks the last hour so that late-indexed items are not missed.

Why does a comment's title show null? HN comments have no title. Use storyTitle and storyUrl for the story the comment belongs to.

Can I turn off exact matching? Yes. Set exactMatch: false to get the raw, typo-tolerant search results (it can also match usernames).

How far back can I go? HN Search covers items back to 2006. Set daysBack up to 7,300 and raise maxItemsPerKeyword.

Does it return deleted or flagged items? Items removed from the HN Search index are not returned. Points and comment counts are the values at the time of the run.

Do I need a proxy or an HN account? No. Runs use 256 MB of memory by default and usually finish in a few seconds.

Something is missing or broken? Open an issue on the Issues tab. Include the input and the RUN_SUMMARY record.

Changelog

• 0.1 (2026-09-24): First release. Keyword search over stories, comments, Ask HN, Show HN, jobs and polls, an exact-mention filter, monitor mode, multi-keyword merging, and a stop at your max cost per run.