# Hacker News Scraper & Keyword Monitor (`heaped_qualm/hackernews-monitor`) Actor

Hacker News scraper and keyword monitor using the official HN API and Algolia search. Track keywords, domains and users, and get only new stories and comments since the last run, with points/comments thresholds and front-page, Ask and Show HN feeds.

- **URL**: https://apify.com/heaped_qualm/hackernews-monitor.md
- **Developed by:** [Artem Kulyk](https://apify.com/heaped_qualm) (community)
- **Categories:** News, Developer tools, AI
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 item results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hacker News Scraper & Keyword Monitor

This **Hacker News scraper** exports and monitors HN stories and comments by keyword, domain or user. Watch **Hacker News** for your keywords, your domains and the users you follow, through the **official HN Firebase API** and the public **Algolia HN Search API**. Each scheduled run returns only the **stories and comments that are new since the last run** and that pass your points and comments thresholds. Front page, Ask HN, Show HN and job feeds are supported too.

**Why this actor:** HN alert e-mails are limited and keyword search on the site has no memory. This monitor gives you clean rows for a spreadsheet, Slack or an AI summary, and never sends the same item twice.

🇩🇪 [Deutsche Version](#deutsch)

### Contents

[Key features](#key-features) · [Use cases](#use-cases) · [Quick start](#quick-start) · [Input](#input) · [Output](#output) · [Pricing](#pricing) · [Integrations](#integrations) · [Limits](#limits) · [FAQ](#faq) · [Legal and data source](#legal-and-data-source) · [Changelog](#changelog) · [Related actors](#related-actors) · [Support](#support)

### Key features

- Official sources only: HN Firebase API for feeds and items, Algolia HN Search API for keyword, domain and user search. No HTML scraping, no login, no key.
- **Three kinds of watch in one run:** keywords (stories or comments), domains (who links to your site) and users (what someone posts).
- **Feeds:** top, new, best, Ask HN, Show HN and job stories.
- **Thresholds:** minimum points and comments. An item that has not reached them yet is not remembered, so it is reported later once it crosses the line.
- **Monitor mode:** only new items per Monitor name; duplicates across watches are removed and `matchedBy` says which watch found the item.
- Comments come with their story title and link; text is converted from HTML to plain text (optional).
- Polite: Algolia about 2 requests per second, Firebase in small parallel batches, retries with backoff.

### Use cases

- **Founders and marketers:** know when your product, company or competitor is mentioned or linked.
- **Developer relations:** follow discussions about your library, API or language.
- **Investors and scouts:** new Show HN launches above 50 points.
- **Recruiters:** new HN job posts and "Who is hiring" comments for a stack.
- **AI agents and RAG:** a fresh tech-news feed with points and comment counts.

### Quick start

1. Click **Try for free** / **Start**. The form is prefilled (stories about AI from the last 72 hours), so the first run works without changes.
2. Replace the example with your own keywords, domains or HN users and adjust the filters.
3. Open the **Output** tab. Use the views (Overview, Comments) or export JSON, CSV, Excel or HTML.
4. To monitor: switch on **Only new since last run**, give the watch a **Monitor name**, save it as a task and add a **schedule** (for example daily at 07:00). From the second run you receive only changes.

### Input

Fill in any of keywords, domains, users or feeds. Each watch is run separately and the results are merged without duplicates.

```json
{
  "keywords": [
    "postgres",
    "\"vector database\""
  ],
  "domains": [
    "github.com/pgvector"
  ],
  "users": [
    "pg"
  ],
  "feeds": [
    "showstories"
  ],
  "itemTypes": [
    "story",
    "comment"
  ],
  "minPoints": 10,
  "lastHours": 24,
  "maxItemsPerSource": 100,
  "onlyNew": true,
  "stateKey": "postgres-watch"
}
```

Tips: put a phrase in quotes for an exact match. **Minimum points** only applies to stories (comments have no public score). Schedule the actor every hour or every day; **Look back (hours)** should be at least the time between runs.

### Output

Each row is one item. `changeType` tells you why it is in the dataset: `BASELINE` (first run), `NEW` (item not seen before) and `SEEN` (only when monitor mode is off).

Real rows from a run on 1 November 2026:

| Change | Matched by | Type | Title | Domain | Points | Comments |
|---|---|---|---|---|---|---|
| BASELINE | keyword: rust | story | Show HN: Quarter – A TUI launcher to browse, search and launch your g… | github.com | 1 | 0 |
| BASELINE | keyword: rust | story | Qt Bridges Technology Preview: Qt Quick UIs for Rust and C# Developers | qt.io | 4 | 0 |
| BASELINE | keyword: rust | story | FerroUI a cross-platform UI framework for Rust | ferroui.github.io | 3 | 1 |

One full record (long values shortened):

```json
{
  "changeType": "BASELINE",
  "matchedBy": "keyword: rust",
  "id": 50329627,
  "type": "story",
  "title": "Show HN: Quarter – A TUI launcher to browse, search and launch your games",
  "url": "https://github.com/stronxyo/quarter",
  "domain": "github.com",
  "author": "stronxyo",
  "points": 1,
  "numComments": 0,
  "createdAt": "2026-11-01T20:40:43Z",
  "storyId": 50329627,
  "storyTitle": "Show HN: Quarter – A TUI launcher to browse, search and launch your games",
  "parentId": null,
  "hnUrl": "https://news.ycombinator.com/item?id=50329627",
  "source": "Hacker News (Algolia HN Search API)",
  "scrapedAt": "2026-11-01T21:06:41.590Z"
}
```

<details><summary>All output fields</summary>

| Field | Description |
|---|---|
| `changeType` | BASELINE on the first run of a monitor, NEW afterwards; SEEN only when monitor mode is off. |
| `matchedBy` | Which source found the item, e.g. keyword: postgres, domain: github.com, user: pg, feed: topstories. |
| `id` | Hacker News item ID. |
| `type` | story, comment, job or poll. |
| `title` | Story title (null for comments). |
| `url` | Link the story points to. |
| `domain` | Host of the link, without www. |
| `author` | HN username of the author. |
| `points` | Score (stories, jobs, polls). |
| `numComments` | Number of comments on the story. |
| `createdAt` | Creation time (ISO 8601, UTC). |
| `storyId` | ID of the story (for comments: the story they belong to). |
| `storyTitle` | Title of the story (for comments: the story they belong to). |
| `parentId` | Parent item ID for comments. |
| `text` | Plain text of the comment or post (if enabled). |
| `hnUrl` | Link to the discussion on news.ycombinator.com. |
| `source` | API the row came from. |
| `scrapedAt` | When this run fetched the item (ISO 8601). |

</details>

A run summary is saved as `RUN_SUMMARY` in the run's key-value store (counts per change type, items returned per watch, skipped below threshold, failed watches).

### Pricing

Pay per result: **$1.00 per 1,000 items** plus **$0.002 per run**. Apify platform usage is included, so there is nothing else to pay. Check the **Pricing** tab for the current price.

| Scenario | Volume | Cost |
|---|---|---|
| Try it: prefilled AI stories | 100 items, 1 run | **$0.10** |
| Hourly brand watch, about 5 new items a day | 150 items, 720 runs | **$1.59** |
| One-off export of a month of mentions | 2,000 items, 1 run | **$2.00** |

Monitor mode keeps a schedule cheap: after the first run you pay only for new stories and comments. Apify's free plan includes $5 of monthly credit, which covers small monitors.

### Integrations

- **Schedules:** Console → Schedules → add this actor's saved task (daily or hourly). Keep the same **Monitor name** so each run compares with the previous one.
- **Webhooks:** on the Integrations tab add a webhook for *Run succeeded* to POST the run to your own endpoint, then fetch rows from `https://api.apify.com/v2/datasets/{datasetId}/items?format=json`.
- **Slack and email:** add the Slack or Email integration on the Integrations tab to get a message after each run.
- **Zapier, Make and n8n:** use the official Apify app ("Watch actor runs" trigger, then "Get dataset items"). Filter on `changeType` to forward only `NEW` items.
- **Google Sheets:** connect through Make or Zapier, or export the dataset as CSV and import it.
- **MCP and AI agents:** add `https://mcp.apify.com?tools=heaped_qualm/hackernews-monitor` to Claude, Cursor or any MCP client. The flat rows are easy for agents to read.

**Python** (`pip install apify-client`):

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("heaped_qualm/hackernews-monitor").call(run_input={"keywords": ["postgres", "\"vector database\""], "domains": ["github.com/pgvector"], "users": ["pg"], "feeds": ["showstories"], "itemTypes": ["story", "comment"], "minPoints": 10, "lastHours": 24, "maxItemsPerSource": 100, "onlyNew": true, "stateKey": "postgres-watch"})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["changeType"], item["title"], item["hnUrl"])
```

**JavaScript** (`npm install apify-client`):

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('heaped_qualm/hackernews-monitor').call({"keywords": ["postgres", "\"vector database\""], "domains": ["github.com/pgvector"], "users": ["pg"], "feeds": ["showstories"], "itemTypes": ["story", "comment"], "minPoints": 10, "lastHours": 24, "maxItemsPerSource": 100, "onlyNew": true, "stateKey": "postgres-watch"});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.table(items);
```

**HTTP** (waits for the run and returns the rows):

```bash
curl -X POST "https://api.apify.com/v2/acts/heaped_qualm~hackernews-monitor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keywords": ["postgres", "\"vector database\""], "domains": ["github.com/pgvector"], "users": ["pg"], "feeds": ["showstories"], "itemTypes": ["story", "comment"], "minPoints": 10, "lastHours": 24, "maxItemsPerSource": 100, "onlyNew": true, "stateKey": "postgres-watch"}'
```

### Limits

- Up to 1,000 items per watch per run; look-back window up to one year.
- Algolia indexes new items within about a minute; points and comment counts are as of the run.
- The monitor remembers up to 50,000 items per Monitor name; the oldest are forgotten first.
- Comments have no public score, so **Minimum points** does not filter them.

### FAQ

**Is it allowed to collect Hacker News data?**
Yes, through the official route. Y Combinator publishes the [HN API](https://github.com/HackerNews/API) to make public HN data available, and Algolia runs the public [HN Search API](https://hn.algolia.com/api). This actor uses only those two and links every row to the item on news.ycombinator.com.

**Can I watch who links to my website?**
Yes. Add your domain under **Domains**. Every story linking to it becomes a row.

**Why is an item reported later than it was posted?**
Because it reached your points or comments threshold only then. Items below the threshold are not remembered.

**Does it include comment text?**
Yes, as plain text, unless you switch **Include text** off.

**How do I get a full export instead of only changes?**
Switch off **Only new since last run**, or use a new **Monitor name**.

**Why are all rows BASELINE on the first run?**
The first run has nothing to compare with, so everything it finds is the baseline. Later runs with the same Monitor name report only changes.

**The run failed or returned nothing. What now?**
Read the status message and the `RUN_SUMMARY` record. Zero items usually means the threshold is too high or the look-back window too short. If it still looks wrong, open an issue on the **Issues** tab with the run link.

### Legal and data source

Data source: [Hacker News](https://news.ycombinator.com) (Y Combinator) via the official [HN API](https://github.com/HackerNews/API) and the [Algolia HN Search API](https://hn.algolia.com/api). Posts and comments belong to their authors; each row links to the original item (`hnUrl`). Use the data in line with the [HN guidelines](https://news.ycombinator.com/newsguidelines.html) and applicable law.

This actor is independent and not affiliated with or endorsed by Y Combinator, Hacker News or Algolia. Data is provided as is, without warranty.

### Changelog

- 0.1 (Nov 2026): first release with keyword, domain, user and feed watches, thresholds, monitor mode and an automated test suite (100% line coverage).

### Related actors

More free monitors by the same author, built on official APIs:

- [ClinicalTrials.gov Scraper & Trial Monitor](https://apify.com/heaped_qualm/clinicaltrials-monitor): ClinicalTrials.gov scraper and trial monitor
- [SEC EDGAR Scraper: 8-K, 10-K, Form 4 Filings](https://apify.com/heaped_qualm/sec-edgar-monitor): SEC EDGAR scraper for 8-K, 10-K, 10-Q and Form 4 filings
- [PubMed Scraper & New Article Monitor](https://apify.com/heaped_qualm/pubmed-monitor): PubMed scraper and new article monitor
- [Greenhouse, Lever & Ashby Jobs Scraper](https://apify.com/heaped_qualm/job-board-monitor): Greenhouse, Lever & Ashby jobs scraper and monitor

### Support

Open an issue on the **Issues** tab of this page. Please include the run link and what you expected to see. Feature requests are welcome.

**More monitors from the same author:** [ClinicalTrials.gov Scraper & Trial Monitor](https://apify.com/heaped_qualm/clinicaltrials-monitor) · [SEC EDGAR Filings Monitor](https://apify.com/heaped_qualm/sec-edgar-monitor) · [PubMed Scraper & New Article Monitor](https://apify.com/heaped_qualm/pubmed-monitor) · [Greenhouse, Lever, Ashby & Recruitee Jobs Monitor](https://apify.com/heaped_qualm/job-board-monitor)

***

<a name="deutsch"></a>

### 🇩🇪 Deutsch: Hacker-News-Monitor: Alerts für Stichwörter, Domains und Nutzer

Überwacht **Hacker News** nach Ihren Stichwörtern, Domains und Nutzern über die **offizielle HN-Firebase-API** und die öffentliche **Algolia HN Search API** und liefert pro Lauf nur **neue Stories und Kommentare** über Ihren Punkte- und Kommentar-Schwellen.

**Funktionen**

- Nur offizielle APIs, kein HTML-Scraping, kein Login
- Stichwort-, Domain- und Nutzer-Watches plus Feeds (Top, New, Ask, Show, Jobs)
- Schwellen für Punkte und Kommentare
- Monitor-Modus: nur Neues seit dem letzten Lauf
- Kommentare mit Titel und Link der Story

**Anwendungsfälle**

- Gründer und Marketing: Erwähnungen von Produkt und Wettbewerb
- Developer Relations: Diskussionen über Ihre Bibliothek
- Investoren: neue Show-HN-Launches
- Recruiter: neue Jobposts

**Schnellstart**

1. **Start** klicken. Das Formular ist vorausgefüllt, der erste Lauf funktioniert ohne Änderungen.
2. Eigene Stichwörter, Domains oder HN-Nutzer eintragen und Filter anpassen.
3. Ergebnisse im Tab **Output** ansehen oder als JSON, CSV oder Excel exportieren.
4. Für die Überwachung **Only new since last run** einschalten, einen **Monitor-Namen** vergeben, als Task speichern und einen **Zeitplan** anlegen. Ab dem zweiten Lauf kommen nur noch Änderungen.

**Eingabebeispiel:** siehe JSON oben (Postgres-Stichwörter, eine Domain, ein Nutzer und Show HN ab 10 Punkten).

**Ausgabe:** eine Zeile pro Eintrag. `changeType` zeigt den Grund: BASELINE (erster Lauf), NEW, SEEN (nur ohne Monitor-Modus).

**Preis:** $1.00 pro 1.000 Einträge plus $0.002 pro Lauf, Plattformkosten inklusive. Beispiel: stündliche Marken-Watch mit etwa 5 neuen Einträgen pro Tag ≈ $1.59 im Monat. Den aktuellen Preis zeigt der Tab **Pricing**.

**Integrationen:** Zeitpläne, Webhooks, Slack, E-Mail, Zapier, Make, n8n, Google Sheets, API (Python, JavaScript, HTTP) und MCP für KI-Agenten (`https://mcp.apify.com?tools=heaped_qualm/hackernews-monitor`).

**Grenzen:** bis 1.000 Einträge pro Watch und Lauf; Rückblick bis ein Jahr.

**Rechtliches und Datenquelle:** Quelle: Hacker News (Y Combinator) über die offizielle HN-API und die Algolia HN Search API; Beiträge gehören ihren Autoren; Link zum Original je Zeile. Kein Bezug zu und keine Unterstützung durch Y Combinator, Hacker News or Algolia. Daten ohne Gewähr.

**Support:** Tab **Issues** auf dieser Seite, bitte mit Link zum Lauf.

# Actor input Schema

## `keywords` (type: `array`):

Full-text searches across Hacker News titles, URLs and text (Algolia HN Search), one per line, e.g. postgres, "large language model", your brand or competitor. Leave every source empty to watch the keyword AI.

## `domains` (type: `array`):

Get items whose link points to these sites, e.g. github.com, yourcompany.com, nytimes.com. Matched against the story URL.

## `users` (type: `array`):

Hacker News usernames to follow (their stories or comments, depending on item types), e.g. patio11, tptacek.

## `feeds` (type: `array`):

Lists from the official Hacker News Firebase API. Items are checked against the filters below (type, age, points, comments).

## `itemTypes` (type: `array`):

Which kinds of items to return. Add Comments to catch mentions in discussions (score thresholds do not apply to comments, which have no public score).

## `minPoints` (type: `integer`):

Only stories, jobs and polls with at least this score. A story below the threshold is not remembered, so it is reported later once it crosses the threshold.

## `minComments` (type: `integer`):

Only stories with at least this many comments. Same delayed reporting as Minimum points.

## `lastHours` (type: `integer`):

Only items created in the last N hours. 0 = no limit (search results are newest first).

## `maxItemsPerSource` (type: `integer`):

Maximum items checked per source and item type per run (newest first). For feeds: how many list entries are fetched.

## `onlyNew` (type: `boolean`):

Monitor mode: return only items not seen in earlier runs under the same monitor name. The first run returns everything as BASELINE. It starts switched off in the form, so a manual test run always shows results (already seen items are marked SEEN). Switch it on in the task you schedule. API calls that omit this field get monitor mode on.

## `stateKey` (type: `string`):

Name of this monitor's memory (letters, digits and dashes; other characters become dashes). Use a different name for each saved task or schedule.

## `includeText` (type: `boolean`):

Adds the plain text of comments and Ask/Show HN posts. Switch off for smaller rows. Texts are written by HN users; check reuse rights before republishing them.

## Actor input object example

```json
{
  "keywords": [
    "postgres",
    "\"open source\" llm"
  ],
  "domains": [
    "github.com",
    "stripe.com"
  ],
  "users": [
    "patio11",
    "dang"
  ],
  "feeds": [
    "topstories",
    "showstories"
  ],
  "itemTypes": [
    "story"
  ],
  "minPoints": 50,
  "minComments": 10,
  "lastHours": 72,
  "maxItemsPerSource": 100,
  "onlyNew": false,
  "stateKey": "default",
  "includeText": true
}
```

# Actor output Schema

## `results` (type: `string`):

All new/changed stories and comments found in this run (default dataset, all fields).

## `overview` (type: `string`):

Default dataset, 'overview' view.

## `comments` (type: `string`):

Default dataset, 'comments' view.

## `runSummary` (type: `string`):

JSON summary of this run (counts, first-run flag, finishedAt) in the default key-value store record RUN_SUMMARY. Monitoring state persists separately in the named key-value store hackernews-monitor-state under STATE-<stateKey>.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "keywords": [
        "AI"
    ],
    "lastHours": 72,
    "onlyNew": false
};

// Run the Actor and wait for it to finish
const run = await client.actor("heaped_qualm/hackernews-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "keywords": ["AI"],
    "lastHours": 72,
    "onlyNew": False,
}

# Run the Actor and wait for it to finish
run = client.actor("heaped_qualm/hackernews-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "keywords": [
    "AI"
  ],
  "lastHours": 72,
  "onlyNew": false
}' |
apify call heaped_qualm/hackernews-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,heaped_qualm/hackernews-monitor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/vhcOdtz7lcX1RCAIB/builds/kfgR5Icw07HfMK7BC/openapi.json