# Hacker News Scraper & Keyword Monitor (Stories + Comments) (`panda_studio/hacker-news-monitor`) Actor

Search Hacker News stories and comments by keyword and get only exact mentions (no typo-matched noise). Monitor mode returns only NEW mentions since the last run for brand, competitor and topic alerts. Official HN Search API, no login, no proxy.

- **URL**: https://apify.com/panda\_studio/hacker-news-monitor.md
- **Developed by:** [panda studio](https://apify.com/panda_studio) (community)
- **Categories:** Developer tools, Marketing, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hacker News Scraper & Keyword Monitor (Stories + Comments)

Search **Hacker News stories and comments** by keyword and get back only the items that **really mention your keyword**. Turn on **monitor mode** and every scheduled run returns just the **new mentions since the last run**, ready for Slack, email, Google Sheets or a webhook. It uses the official public HN Search API (hn.algolia.com). No login, no API key, no proxy.

### What does Hacker News Scraper & Keyword Monitor do?

- Searches HN **stories, comments, Ask HN, Show HN, job posts and polls** for one or more keywords, newest first.
- **Exact mentions only (on by default).** The raw HN search is typo-tolerant and also matches usernames. For `apify` over the last 365 days (measured 2026-09-24), the raw search returned **105** items and this Actor's exact filter kept **73**. The other 32 were words like *ipify*, *appify* and *ApiFB*, or authors whose name looks like the keyword.
- **Monitor mode:** remembers what it already returned (in a key-value store in *your* account) and returns only new items next time. You can keep several monitors apart with a Monitor ID.
- One row per HN item, even when it matches several keywords (`matchedKeywords` lists all of them). Each row also records **where** the match was found (`title`, `url` or `text`).
- Comments come with their **story title, story URL and story link**, so an alert makes sense without opening HN.
- Adds a short **snippet** around the match. HTML is converted to plain text.

### Who is it for? Use cases

- **Brand and product monitoring.** Founders, DevRel and marketing teams see when their product, domain or company is discussed, and can reply while the thread is still active.
- **Competitor tracking.** Collect every mention of competing tools and read what users praise or complain about.
- **Launch tracking.** Follow the Show HN and the comments after you launch.
- **Market and technology research.** Count how often a library, framework or topic comes up over months or years. The index goes back to 2006.
- **Lead and hiring signals.** Find "Ask HN: Who wants to be hired?" comments or job posts that mention a skill (for example `"data engineer" remote`).
- **LLM and RAG pipelines.** Clean JSON text with links, ready for summarisation or sentiment analysis.

### Why use this Actor?

| | This Actor | Typical HN scrapers |
|---|---|---|
| Stories **and** comments | Yes | Often stories only |
| Exact-mention filter (no typo/author matches) | Yes, on by default | No, raw search results |
| Only new items since the last run | Built-in monitor mode | Rerun and de-duplicate yourself |
| Comment rows include story title/URL | Yes | Varies |
| Several keywords in one run, merged | Yes (`matchedKeywords`) | One query per run is common |
| Proxy / login / API key | None | None |

### What data do you get?

| Field | Description |
|---|---|
| `type` | `story`, `comment`, `ask_hn`, `show_hn`, `job` or `poll` |
| `id`, `hnUrl` | HN item ID and link |
| `title`, `url` | Post title and external link (stories). `null` for comments |
| `author` | HN username |
| `points`, `numComments` | Score and comment count (stories). HN does not publish comment scores |
| `createdAt` | ISO 8601, UTC |
| `storyId`, `storyTitle`, `storyUrl`, `storyHnUrl` | The story the item belongs to (for comments: the parent story) |
| `parentId` | Direct parent (comments) |
| `matchedKeywords` | Every keyword of your input that this item matched |
| `matchedIn` | Where the match was found: `title`, `url`, `text` |
| `snippet` | About 220 characters around the match |
| `text` | Full plain text (switch off with `includeText: false`) |
| `changeType` | `new` in monitor mode, otherwise `null` |
| `scrapedAt` | When the run fetched the item |

A `RUN_SUMMARY` record in the key-value store shows, per keyword, how many API hits were scanned, how many matched, how many were new, and any error.

### How much does it cost?

Pay per event, and the platform usage is included:

| Event | Price |
|---|---|
| Item returned (`apify-default-dataset-item`) | **$0.001** ($1 per 1,000 items) |
| Keyword searched (`keyword-checked`, once per keyword per run) | **$0.001** |
| Actor start | $0.00005 |

Examples, calculated from the prices above:

| Scenario | Calculation | Cost |
|---|---|---|
| 1 keyword, latest 100 mentions | 0.00005 + 0.001 + 100 × 0.001 | **≈ $0.10** |
| 5 keywords, 1 year of history, 2,000 items | 0.00005 + 5 × 0.001 + 2,000 × 0.001 | **≈ $2.01** |
| Daily monitor, 5 keywords, about 10 new mentions a day, 30 days | 30 × (0.00005 + 0.005) + 300 × 0.001 | **≈ $0.45 / month** |
| Hourly monitor, 5 keywords, about 10 new mentions a day, 30 days | 720 × (0.00005 + 0.005) + 300 × 0.001 | **≈ $3.94 / month** |

You can set **Max cost per run** in the run options. The Actor reads it and stops returning items before it would go over.

### How to use it

1. Click **Try for free** (an Apify account is free).
2. Enter your **keywords**, one per line. Put phrases in double quotes, for example `"vector database"`.
3. Choose the **content types** and the **look-back window**.
4. Click **Start**. Download the results as JSON, CSV, Excel or HTML, or use the API.
5. For alerts: turn on **Monitor mode**, save the run as a **Task**, add a **Schedule** (hourly or daily), and connect an integration (Slack, email, Google Sheets, Zapier, Make, webhook).

### Input example

```json
{
  "keywords": ["apify", "\"web scraping\""],
  "contentTypes": ["story", "comment"],
  "daysBack": 30,
  "maxItemsPerKeyword": 10,
  "exactMatch": true,
  "minPoints": 0,
  "includeText": true,
  "monitorMode": false,
  "monitorId": "default"
}
```

Keyword syntax: all words are required (AND). Matching ignores case and requires whole words (`go` does not match `google`). A quoted phrase must appear as written, and hyphens count as spaces (`"open source"` matches *open-source*).

### Output example

The following output was taken from a real run on 2026-09-24 with the input above (text shortened). A comment:

```json
{
  "type": "comment",
  "id": "49825437",
  "hnUrl": "https://news.ycombinator.com/item?id=49825437",
  "title": null,
  "url": null,
  "author": "tiew9Vii",
  "points": null,
  "numComments": null,
  "createdAt": "2026-09-24T02:22:13Z",
  "storyId": "49822654",
  "storyTitle": "OpenAI agents hacked Australian Medicare system",
  "storyUrl": "https://www.reuters.com/world/asia-pacific/australia-pm-albanese-says-openai-breached-medicare-sydney-morning-herald-2026-09-23/",
  "storyHnUrl": "https://news.ycombinator.com/item?id=49822654",
  "parentId": "49825138",
  "matchedKeywords": ["\"web scraping\""],
  "matchedIn": ["text"],
  "snippet": "…organisation, the prime minister had “frank” words while an individual faces jail time for using open AI for web scraping https://www.abc.net.au/news/2026-09-10/christopher-duff-to-s...",
  "text": "In a large portion of the civilised world, there’s strict gun controls, …",
  "changeType": null,
  "scrapedAt": "2026-09-24T14:24:12Z"
}
```

A Show HN post (input `{"keywords": ["rust"], "contentTypes": ["show_hn"], "daysBack": 30, "minPoints": 10, "includeText": false}`):

```json
{
  "type": "show_hn",
  "id": "49804217",
  "hnUrl": "https://news.ycombinator.com/item?id=49804217",
  "title": "Show HN: graf (1000x faster graphify in Rust)",
  "url": "https://github.com/ctxrs/graf",
  "author": "ripped_britches",
  "points": 12,
  "numComments": 3,
  "createdAt": "2026-09-22T16:46:31Z",
  "storyId": "49804217",
  "storyTitle": "Show HN: graf (1000x faster graphify in Rust)",
  "storyUrl": "https://github.com/ctxrs/graf",
  "storyHnUrl": "https://news.ycombinator.com/item?id=49804217",
  "parentId": null,
  "matchedKeywords": ["rust"],
  "matchedIn": ["title", "text"],
  "snippet": "…I dug into it and saw an opportunity to make it better. It is rewritten in rust and uses sqlite instead of NetworkX. …",
  "changeType": null,
  "scrapedAt": "2026-09-24T14:24:14Z"
}
```

Points and comment counts are the values at the time of the run.

### Monitor mode

- The first run returns the newest `maxItemsPerKeyword` items from the last `daysBack` days. Each later run with the same **Monitor ID** returns only items it has not returned before, with `changeType: "new"`.
- When nothing new has appeared, the run succeeds with 0 items. On a pay-per-event plan you then pay only the start and keyword events.
- Each later run searches again from one hour before the newest item it saw, so items that reach the search index late are still caught. Duplicates are removed.
- If a run stops at your max cost per run, the items it did not return are **not** marked as seen, so the next run returns them.
- If you add a keyword to an existing monitor, its first run behaves like a first run.
- The saved state is in the key-value store `hacker-news-monitor-state` in your Apify account. Delete the record `monitor-<ID>` to start over.

Tested on 2026-09-24. The first monitor run returned 203 items. A second run a moment later returned 0. After 3 remembered IDs were removed from the state on purpose, the next run returned exactly those 3 as `new`.

### Tips

- Use distinctive keywords. `"acme cloud"` or `acmecloud.com` give cleaner results than a common word.
- Search your **domain name** too. Links posted to HN match through the `url` field.
- `minPoints` (for example 50) turns the Actor into a filter for popular stories on a topic.
- Set `includeText: false` when you only need links and snippets.
- For long periods, raise `maxItemsPerKeyword`. The Actor reads the index in pages of 1,000, newest first. In a test it went back 2 years for `rust` and returned 2,500 items in about 6 seconds.

### Integrations and API

Run it from your own code with the Apify API clients.

Python:

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("panda_studio/hacker-news-monitor").call(run_input={
    "keywords": ["your-product", "\"your company\""],
    "monitorMode": True,
    "monitorId": "brand",
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["createdAt"], item["type"], item["hnUrl"], item["snippet"])
```

Node.js:

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });
const run = await client.actor('panda_studio/hacker-news-monitor').call({
    keywords: ['your-product'],
    contentTypes: ['story', 'comment'],
    daysBack: 7,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

cURL (runs the Actor and returns the items in one call):

```bash
curl -X POST "https://api.apify.com/v2/acts/panda_studio~hacker-news-monitor/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"keywords": ["apify"], "daysBack": 7}'
```

AI agents (MCP): add `https://mcp.apify.com?tools=panda_studio/hacker-news-monitor` to Claude, Cursor or VS Code. For Claude Code:

```bash
claude mcp add --transport http apify "https://mcp.apify.com?tools=panda_studio/hacker-news-monitor"
```

### Is it legal to scrape Hacker News?

This Actor does not scrape HN pages. It uses the **HN Search API** (`https://hn.algolia.com/api/v1/`), which Algolia runs for Y Combinator and documents publicly at hn.algolia.com/api. The API needs no key and returns public posts and comments. The Actor waits between requests and retries politely when the API asks it to slow down (HTTP 429). The data includes public HN usernames. If you store or republish the data, follow the rules that apply to you, such as GDPR and HN's own guidelines. This is not legal advice.

### FAQ

**How is this different from the HN search box?**
It runs several keywords at once, drops typo and username matches, adds the story context to comments, returns only new items on a schedule, and exports to CSV or JSON, or sends the results to your tools.

**Can I get results in real time?**
You can schedule the Actor as often as every few minutes. It can only see items once they are in the HN Search index, and each run re-checks the last hour so that late-indexed items are not missed.

**Why does a comment's `title` show `null`?**
HN comments have no title. Use `storyTitle` and `storyUrl` for the story the comment belongs to.

**Can I turn off exact matching?**
Yes. Set `exactMatch: false` to get the raw, typo-tolerant search results (it can also match usernames).

**How far back can I go?**
HN Search covers items back to 2006. Set `daysBack` up to 7,300 and raise `maxItemsPerKeyword`.

**Does it return deleted or flagged items?**
Items removed from the HN Search index are not returned. Points and comment counts are the values at the time of the run.

**Do I need a proxy or an HN account?**
No. Runs use 256 MB of memory by default and usually finish in a few seconds.

**Something is missing or broken?**
Open an issue on the Issues tab. Include the input and the `RUN_SUMMARY` record.

### Changelog

- **0.1 (2026-09-24)**: First release. Keyword search over stories, comments, Ask HN, Show HN, jobs and polls, an exact-mention filter, monitor mode, multi-keyword merging, and a stop at your max cost per run.

# Actor input Schema

## `keywords` (type: `array`):

One search per line. Words are all required (AND). Put a phrase in double quotes to require it exactly, e.g. <code>"vector database"</code>. Brand names, competitor names, product names, libraries, your domain, etc.

## `contentTypes` (type: `array`):

What to search. <code>story</code> includes Ask HN and Show HN posts. Pick <code>ask\_hn</code> or <code>show\_hn</code> alone to get only those.

## `daysBack` (type: `integer`):

Only items created in the last N days. The HN index goes back to 2006, so large values work too. In monitor mode this is the window for the first run only.

## `maxItemsPerKeyword` (type: `integer`):

Newest first. Keeps cost predictable: you pay per item returned.

## `exactMatch` (type: `boolean`):

On (recommended): keep only items whose own title, URL or text contains every word/phrase (case-insensitive, whole word). The raw HN search is typo-tolerant and also matches author names, so without this a search for <code>apify</code> also returns comments about <code>ipify</code> or <code>appify</code>. Off: return whatever the HN search returns.

## `minPoints` (type: `integer`):

Skip stories with fewer points. Comments have no public score and are not affected.

## `includeText` (type: `boolean`):

Add the full comment / post text (HTML converted to plain text). A short snippet around the match is always included.

## `monitorMode` (type: `boolean`):

Return only items not returned by a previous run with the same Monitor ID. Schedule the Actor (e.g. hourly or daily) and connect Slack, email or a webhook to get alerts. State is kept in a named key-value store in your own account.

## `monitorId` (type: `string`):

Name of the saved state for monitor mode. Use a different ID for each independent monitor (e.g. <code>brand</code>, <code>competitors</code>).

## Actor input object example

```json
{
  "keywords": [
    "apify",
    "\"web scraping\""
  ],
  "contentTypes": [
    "story",
    "comment"
  ],
  "daysBack": 7,
  "maxItemsPerKeyword": 50,
  "exactMatch": true,
  "minPoints": 0,
  "includeText": true,
  "monitorMode": false,
  "monitorId": "default"
}
```

# Actor output Schema

## `items` (type: `string`):

No description

## `runSummary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "keywords": [
        "apify",
        "\"web scraping\""
    ],
    "contentTypes": [
        "story",
        "comment"
    ],
    "daysBack": 7,
    "maxItemsPerKeyword": 50
};

// Run the Actor and wait for it to finish
const run = await client.actor("panda_studio/hacker-news-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "keywords": [
        "apify",
        "\"web scraping\"",
    ],
    "contentTypes": [
        "story",
        "comment",
    ],
    "daysBack": 7,
    "maxItemsPerKeyword": 50,
}

# Run the Actor and wait for it to finish
run = client.actor("panda_studio/hacker-news-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "keywords": [
    "apify",
    "\\"web scraping\\""
  ],
  "contentTypes": [
    "story",
    "comment"
  ],
  "daysBack": 7,
  "maxItemsPerKeyword": 50
}' |
apify call panda_studio/hacker-news-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,panda_studio/hacker-news-monitor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/vhlqWD9pn4xrDw1Zf/builds/dmXaJi1RU5Nykebbx/openapi.json