Hacker News Scraper & Keyword Monitor

This Hacker News scraper exports and monitors HN stories and comments by keyword, domain or user. Watch Hacker News for your keywords, your domains and the users you follow, through the official HN Firebase API and the public Algolia HN Search API. Each scheduled run returns only the stories and comments that are new since the last run and that pass your points and comments thresholds. Front page, Ask HN, Show HN and job feeds are supported too.

Why this actor: HN alert e-mails are limited and keyword search on the site has no memory. This monitor gives you clean rows for a spreadsheet, Slack or an AI summary, and never sends the same item twice.

🇩🇪 Deutsche Version

Contents

Key features · Use cases · Quick start · Input · Output · Pricing · Integrations · Limits · FAQ · Legal and data source · Changelog · Related actors · Support

Key features

• Official sources only: HN Firebase API for feeds and items, Algolia HN Search API for keyword, domain and user search. No HTML scraping, no login, no key.
• Three kinds of watch in one run: keywords (stories or comments), domains (who links to your site) and users (what someone posts).
• Feeds: top, new, best, Ask HN, Show HN and job stories.
• Thresholds: minimum points and comments. An item that has not reached them yet is not remembered, so it is reported later once it crosses the line.
• Monitor mode: only new items per Monitor name; duplicates across watches are removed and matchedBy says which watch found the item.
• Comments come with their story title and link; text is converted from HTML to plain text (optional).
• Polite: Algolia about 2 requests per second, Firebase in small parallel batches, retries with backoff.

Use cases

• Founders and marketers: know when your product, company or competitor is mentioned or linked.
• Developer relations: follow discussions about your library, API or language.
• Investors and scouts: new Show HN launches above 50 points.
• Recruiters: new HN job posts and "Who is hiring" comments for a stack.
• AI agents and RAG: a fresh tech-news feed with points and comment counts.

Quick start

1. Click Try for free / Start. The form is prefilled (stories about AI from the last 72 hours), so the first run works without changes.
2. Replace the example with your own keywords, domains or HN users and adjust the filters.
3. Open the Output tab. Use the views (Overview, Comments) or export JSON, CSV, Excel or HTML.
4. To monitor: switch on Only new since last run, give the watch a Monitor name, save it as a task and add a schedule (for example daily at 07:00). From the second run you receive only changes.

Input

Fill in any of keywords, domains, users or feeds. Each watch is run separately and the results are merged without duplicates.

{
  "keywords": [
    "postgres",
    "\"vector database\""
  ],
  "domains": [
    "github.com/pgvector"
  ],
  "users": [
    "pg"
  ],
  "feeds": [
    "showstories"
  ],
  "itemTypes": [
    "story",
    "comment"
  ],
  "minPoints": 10,
  "lastHours": 24,
  "maxItemsPerSource": 100,
  "onlyNew": true,
  "stateKey": "postgres-watch"
}

Tips: put a phrase in quotes for an exact match. Minimum points only applies to stories (comments have no public score). Schedule the actor every hour or every day; Look back (hours) should be at least the time between runs.

Output

Each row is one item. changeType tells you why it is in the dataset: BASELINE (first run), NEW (item not seen before) and SEEN (only when monitor mode is off).

Real rows from a run on 1 November 2026:

Change	Matched by	Type	Title	Domain	Points	Comments
BASELINE	keyword: rust	story	Show HN: Quarter – A TUI launcher to browse, search and launch your g…	github.com	1	0
BASELINE	keyword: rust	story	Qt Bridges Technology Preview: Qt Quick UIs for Rust and C# Developers	qt.io	4	0
BASELINE	keyword: rust	story	FerroUI a cross-platform UI framework for Rust	ferroui.github.io	3	1

One full record (long values shortened):

{
  "changeType": "BASELINE",
  "matchedBy": "keyword: rust",
  "id": 50329627,
  "type": "story",
  "title": "Show HN: Quarter – A TUI launcher to browse, search and launch your games",
  "url": "https://github.com/stronxyo/quarter",
  "domain": "github.com",
  "author": "stronxyo",
  "points": 1,
  "numComments": 0,
  "createdAt": "2026-11-01T20:40:43Z",
  "storyId": 50329627,
  "storyTitle": "Show HN: Quarter – A TUI launcher to browse, search and launch your games",
  "parentId": null,
  "hnUrl": "https://news.ycombinator.com/item?id=50329627",
  "source": "Hacker News (Algolia HN Search API)",
  "scrapedAt": "2026-11-01T21:06:41.590Z"
}

A run summary is saved as RUN_SUMMARY in the run's key-value store (counts per change type, items returned per watch, skipped below threshold, failed watches).

Pricing

Pay per result: $1.00 per 1,000 items plus $0.002 per run. Apify platform usage is included, so there is nothing else to pay. Check the Pricing tab for the current price.

Scenario	Volume	Cost
Try it: prefilled AI stories	100 items, 1 run	$0.10
Hourly brand watch, about 5 new items a day	150 items, 720 runs	$1.59
One-off export of a month of mentions	2,000 items, 1 run	$2.00

Monitor mode keeps a schedule cheap: after the first run you pay only for new stories and comments. Apify's free plan includes $5 of monthly credit, which covers small monitors.

Integrations

• Schedules: Console → Schedules → add this actor's saved task (daily or hourly). Keep the same Monitor name so each run compares with the previous one.
• Webhooks: on the Integrations tab add a webhook for Run succeeded to POST the run to your own endpoint, then fetch rows from https://api.apify.com/v2/datasets/{datasetId}/items?format=json.
• Slack and email: add the Slack or Email integration on the Integrations tab to get a message after each run.
• Zapier, Make and n8n: use the official Apify app ("Watch actor runs" trigger, then "Get dataset items"). Filter on changeType to forward only NEW items.
• Google Sheets: connect through Make or Zapier, or export the dataset as CSV and import it.
• MCP and AI agents: add https://mcp.apify.com?tools=heaped_qualm/hackernews-monitor to Claude, Cursor or any MCP client. The flat rows are easy for agents to read.

Python (pip install apify-client):

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("heaped_qualm/hackernews-monitor").call(run_input={"keywords": ["postgres", "\"vector database\""], "domains": ["github.com/pgvector"], "users": ["pg"], "feeds": ["showstories"], "itemTypes": ["story", "comment"], "minPoints": 10, "lastHours": 24, "maxItemsPerSource": 100, "onlyNew": true, "stateKey": "postgres-watch"})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["changeType"], item["title"], item["hnUrl"])

JavaScript (npm install apify-client):

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('heaped_qualm/hackernews-monitor').call({"keywords": ["postgres", "\"vector database\""], "domains": ["github.com/pgvector"], "users": ["pg"], "feeds": ["showstories"], "itemTypes": ["story", "comment"], "minPoints": 10, "lastHours": 24, "maxItemsPerSource": 100, "onlyNew": true, "stateKey": "postgres-watch"});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.table(items);

HTTP (waits for the run and returns the rows):

curl -X POST "https://api.apify.com/v2/acts/heaped_qualm~hackernews-monitor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keywords": ["postgres", "\"vector database\""], "domains": ["github.com/pgvector"], "users": ["pg"], "feeds": ["showstories"], "itemTypes": ["story", "comment"], "minPoints": 10, "lastHours": 24, "maxItemsPerSource": 100, "onlyNew": true, "stateKey": "postgres-watch"}'

Limits

• Up to 1,000 items per watch per run; look-back window up to one year.
• Algolia indexes new items within about a minute; points and comment counts are as of the run.
• The monitor remembers up to 50,000 items per Monitor name; the oldest are forgotten first.
• Comments have no public score, so Minimum points does not filter them.

FAQ

Is it allowed to collect Hacker News data? Yes, through the official route. Y Combinator publishes the HN API to make public HN data available, and Algolia runs the public HN Search API. This actor uses only those two and links every row to the item on news.ycombinator.com.

Can I watch who links to my website? Yes. Add your domain under Domains. Every story linking to it becomes a row.

Why is an item reported later than it was posted? Because it reached your points or comments threshold only then. Items below the threshold are not remembered.

Does it include comment text? Yes, as plain text, unless you switch Include text off.

How do I get a full export instead of only changes? Switch off Only new since last run, or use a new Monitor name.

Why are all rows BASELINE on the first run? The first run has nothing to compare with, so everything it finds is the baseline. Later runs with the same Monitor name report only changes.

The run failed or returned nothing. What now? Read the status message and the RUN_SUMMARY record. Zero items usually means the threshold is too high or the look-back window too short. If it still looks wrong, open an issue on the Issues tab with the run link.

Legal and data source

Data source: Hacker News (Y Combinator) via the official HN API and the Algolia HN Search API. Posts and comments belong to their authors; each row links to the original item (hnUrl). Use the data in line with the HN guidelines and applicable law.

This actor is independent and not affiliated with or endorsed by Y Combinator, Hacker News or Algolia. Data is provided as is, without warranty.

Changelog

• 0.1 (Nov 2026): first release with keyword, domain, user and feed watches, thresholds, monitor mode and an automated test suite (100% line coverage).

Related actors

More free monitors by the same author, built on official APIs:

• ClinicalTrials.gov Scraper & Trial Monitor: ClinicalTrials.gov scraper and trial monitor
• SEC EDGAR Scraper: 8-K, 10-K, Form 4 Filings: SEC EDGAR scraper for 8-K, 10-K, 10-Q and Form 4 filings
• PubMed Scraper & New Article Monitor: PubMed scraper and new article monitor
• Greenhouse, Lever & Ashby Jobs Scraper: Greenhouse, Lever & Ashby jobs scraper and monitor

Support

Open an issue on the Issues tab of this page. Please include the run link and what you expected to see. Feature requests are welcome.

More monitors from the same author: ClinicalTrials.gov Scraper & Trial Monitor · SEC EDGAR Filings Monitor · PubMed Scraper & New Article Monitor · Greenhouse, Lever, Ashby & Recruitee Jobs Monitor

---

🇩🇪 Deutsch: Hacker-News-Monitor: Alerts für Stichwörter, Domains und Nutzer

Überwacht Hacker News nach Ihren Stichwörtern, Domains und Nutzern über die offizielle HN-Firebase-API und die öffentliche Algolia HN Search API und liefert pro Lauf nur neue Stories und Kommentare über Ihren Punkte- und Kommentar-Schwellen.

Funktionen

• Nur offizielle APIs, kein HTML-Scraping, kein Login
• Stichwort-, Domain- und Nutzer-Watches plus Feeds (Top, New, Ask, Show, Jobs)
• Schwellen für Punkte und Kommentare
• Monitor-Modus: nur Neues seit dem letzten Lauf
• Kommentare mit Titel und Link der Story

Anwendungsfälle

• Gründer und Marketing: Erwähnungen von Produkt und Wettbewerb
• Developer Relations: Diskussionen über Ihre Bibliothek
• Investoren: neue Show-HN-Launches
• Recruiter: neue Jobposts

Schnellstart

1. Start klicken. Das Formular ist vorausgefüllt, der erste Lauf funktioniert ohne Änderungen.
2. Eigene Stichwörter, Domains oder HN-Nutzer eintragen und Filter anpassen.
3. Ergebnisse im Tab Output ansehen oder als JSON, CSV oder Excel exportieren.
4. Für die Überwachung Only new since last run einschalten, einen Monitor-Namen vergeben, als Task speichern und einen Zeitplan anlegen. Ab dem zweiten Lauf kommen nur noch Änderungen.

Eingabebeispiel: siehe JSON oben (Postgres-Stichwörter, eine Domain, ein Nutzer und Show HN ab 10 Punkten).

Ausgabe: eine Zeile pro Eintrag. changeType zeigt den Grund: BASELINE (erster Lauf), NEW, SEEN (nur ohne Monitor-Modus).

Preis: $1.00 pro 1.000 Einträge plus $0.002 pro Lauf, Plattformkosten inklusive. Beispiel: stündliche Marken-Watch mit etwa 5 neuen Einträgen pro Tag ≈ $1.59 im Monat. Den aktuellen Preis zeigt der Tab Pricing.

Integrationen: Zeitpläne, Webhooks, Slack, E-Mail, Zapier, Make, n8n, Google Sheets, API (Python, JavaScript, HTTP) und MCP für KI-Agenten (https://mcp.apify.com?tools=heaped_qualm/hackernews-monitor).

Grenzen: bis 1.000 Einträge pro Watch und Lauf; Rückblick bis ein Jahr.

Rechtliches und Datenquelle: Quelle: Hacker News (Y Combinator) über die offizielle HN-API und die Algolia HN Search API; Beiträge gehören ihren Autoren; Link zum Original je Zeile. Kein Bezug zu und keine Unterstützung durch Y Combinator, Hacker News or Algolia. Daten ohne Gewähr.

Support: Tab Issues auf dieser Seite, bitte mit Link zum Lauf.