# Hacker News Scraper & Keyword Monitor (Stories, Comments) (`brightish_ascent/hacker-news-keyword-monitor`) Actor

Unofficial; not affiliated with Y Combinator or Algolia. Search and monitor Hacker News stories and comments for keywords, brands and domains via the official HN Search API. Points/comments filters, clean text, only-new items for scheduled runs. By Harlo.

- **URL**: https://apify.com/brightish_ascent/hacker-news-keyword-monitor.md
- **Developed by:** [Michael Rice](https://apify.com/brightish_ascent) (community)
- **Categories:** Business, Developer tools, Agents
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.20 / 1,000 hn items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hacker News Scraper & Keyword Monitor (Stories, Comments)

**By Harlo.** Unofficial tool. Not affiliated with or endorsed by Y Combinator, Hacker News or Algolia.

Search and monitor Hacker News for your keywords, brand, product or competitors — and for any story that links to your domain. Data comes from the **official HN Search API** (hn.algolia.com, the search behind news.ycombinator.com). You get one clean row per story or comment with points, comment count, date, HN link, the story a comment belongs to, and plain-text content. Schedule it to receive only **new** items.

### What you get

- **Keyword search** across titles, URLs, story text and comments — up to 25 keywords/domains per run, each row tagged with the `matchedQueries` that found it (duplicates merged).
- **Domain mentions:** `domains: ["yourcompany.com"]` returns stories that link to that site or its subdomains (checked against the real link host).
- **Exact keyword matching by default:** HN Search is typo-tolerant, so a plain search for `rust` also returns comments saying "just" or "must" (in one 12-hour test: 898 comment hits fuzzy vs 25 exact). This Actor turns typo tolerance off unless you set `typoTolerance: true`.
- **Content types:** stories (incl. Show HN / Ask HN), comments, Show HN only, Ask HN only, polls.
- **Modes:** newest first (monitoring), most relevant (research, all-time with `lookbackHours: 0`), or what's on the **front page** right now.
- **Filters:** lookback window in hours, minimum points, minimum comments (stories).
- **Clean text:** HTML removed, links kept, HTML entities decoded, email-like strings replaced with `[email removed]`, optional length limit.
- **Monitor mode** (`onlyNewSinceLastRun`): first run is a baseline; later runs return only `new` items (and `changed` ones if a title, URL or text was edited, with `previous`). Points and comment counts change all the time and do **not** trigger a re-delivery.
- **Compact output** for spreadsheets, Slack/Zapier alerts and LLM pipelines.
- **Run summary** in `OUTPUT`: per-search hits, pages read, why each search stopped, `truncated` flag with a plain-English note.

Empty input works: it returns the newest stories from the last 24 hours.

### Input example

```json
{
  "queries": ["apify", "web scraping"],
  "domains": ["github.com"],
  "contentTypes": ["story", "comment"],
  "lookbackHours": 48,
  "minPoints": 0,
  "maxItems": 50,
  "onlyNewSinceLastRun": true
}
```

### Output example (full row)

```json
{
  "id": "50313998",
  "kind": "story",
  "title": "Live benchmark of web scraping APIs and residential proxies (2026)",
  "url": "https://github.com/tianjun588/llm-web-scraping-benchmark",
  "domain": "github.com",
  "hnUrl": "https://news.ycombinator.com/item?id=50313998",
  "points": 1, "numComments": 0,
  "createdAt": "2026-10-31T07:39:18.000Z",
  "storyId": "50313998",
  "storyTitle": "Live benchmark of web scraping APIs and residential proxies (2026)",
  "storyUrl": "https://github.com/tianjun588/llm-web-scraping-benchmark",
  "parentId": null, "onFrontPage": false,
  "author": "tianjun588", "text": null,
  "matchedQueries": ["web scraping", "domain:github.com"],
  "updateType": "new", "previous": null,
  "changeHash": "…", "fetchedAt": "2026-10-31T07:45:01.000Z"
}
```

Comment rows have `kind: "comment"`, `text`, `storyTitle`, `storyUrl`, `parentId`; `title`, `url`, `points` and `numComments` are `null` (HN Search does not expose comment points).

### Pricing

Pay per event: **$0.00005 per Actor start** + a small fee per dataset row (event `item`). Only delivered rows are charged, and the run's maximum total charge is respected. See the Pricing tab for the current price.

### FAQ

**How does monitor mode work?** Turn on `onlyNewSinceLastRun` and schedule the Actor (e.g. hourly or daily). State lives in a named key-value store, `hacker-news-keyword-monitor-state`, in your own account — one record per input set (keywords, domains, types, mode, lookback, filters). Rows cut by `maxItems` or your cost limit are delivered on the next run while they are still inside `lookbackHours`. Keep `lookbackHours` at least as long as your schedule interval.

**How far back can I search?** Set `lookbackHours: 0` for all-time. HN Search returns at most 1,000 hits per keyword and content type; narrow the keyword or use `minPoints` for big topics (`truncated` tells you when this happens).

**Can I use it from an AI agent / MCP?** Yes — call it through the Apify MCP server (mcp.apify.com) or the API; use `outputFormat: "compact"` and a small `maxItems` to keep payloads small.

**What about personal data?** Only public HN usernames are included (`includeAuthor: false` removes them). No user profiles, karma or "about" text are fetched, and email-like strings in post text are redacted. Use the data in line with Hacker News' and Algolia's terms and applicable privacy law.

**Is it official?** No. It uses the public, keyless HN Search API by Algolia (documented at hn.algolia.com/api) at a polite pace (≥ 250 ms between requests, retries with backoff on 429/5xx, per-run request cap). It does not scrape news.ycombinator.com pages, so HN's robots.txt crawl rules for the website are not involved. No login, no API key. Client identifier: `hacker-news-keyword-monitor/0.1 (Apify Actor by Harlo; unofficial; digitalpromohub.support@gmail.com)`.

**Limits:** HN Search indexes new items within minutes but not instantly; deleted/dead items may be missing; matching is HN Search's word matching (multi-word keywords must all appear, not necessarily as an exact phrase).

Support: digitalpromohub.support@gmail.com

# Actor input Schema

## `queries` (type: `array`):

Words or phrases to search for in titles, URLs, story text and comments (HN Search full-text). One search per keyword. Leave empty (and domains empty) to get the latest items.

## `domains` (type: `array`):

Find stories that link to these sites, e.g. "github.com" or "yourcompany.com" (subdomains included). Matches story URLs only. queries + domains max 25 per run.

## `contentTypes` (type: `array`):

What to return. "story" already includes Show HN and Ask HN posts.

## `mode` (type: `string`):

newest = most recent first; relevance = HN Search ranking (relevance, then points); front_page = stories on the HN front page right now (ignores lookbackHours).

## `lookbackHours` (type: `integer`):

Only items created in the last N hours. 0 = no time limit (all-time search, up to 1,000 hits per keyword).

## `minPoints` (type: `integer`):

Keep only stories with at least this many points. Not applied to comments.

## `minComments` (type: `integer`):

Keep only stories with at least this many comments. Not applied to comments.

## `typoTolerance` (type: `boolean`):

Off (default) = exact keyword matching, so "rust" does not also return "just" or "must". On = HN Search's default typo tolerance (more results, more noise).

## `includeAuthor` (type: `boolean`):

Add the public HN username of each post. No user profiles are fetched.

## `includeText` (type: `boolean`):

Add story/comment text as plain text (HTML removed, links kept, email-like strings replaced with "\[email removed]").

## `maxTextChars` (type: `integer`):

Trim text to this many characters (0 = no limit).

## `onlyNewSinceLastRun` (type: `boolean`):

Remember delivered items per input set in a named key-value store in your account (hacker-news-keyword-monitor-state). Later runs return only new items, plus items whose title, URL or text was edited. First run = baseline (returns current matches).

## `maxItems` (type: `integer`):

Maximum dataset rows per run (each row is one charged event). In monitor mode, rows over the limit are delivered on the next run.

## `outputFormat` (type: `string`):

full = all fields; compact = id, kind, title, url, hnUrl, points, numComments, createdAt, storyTitle, text, matchedQueries, updateType.

## Actor input object example

```json
{
  "queries": [
    "apify"
  ],
  "contentTypes": [
    "story"
  ],
  "mode": "newest",
  "lookbackHours": 24,
  "minPoints": 0,
  "minComments": 0,
  "typoTolerance": false,
  "includeAuthor": true,
  "includeText": true,
  "maxTextChars": 2000,
  "onlyNewSinceLastRun": false,
  "maxItems": 100,
  "outputFormat": "full"
}
```

# Actor output Schema

## `results` (type: `string`):

Default dataset. full rows: id, kind (story | show_hn | ask_hn | poll | comment), title, url, domain, hnUrl, points, numComments, createdAt, storyId, storyTitle, storyUrl, parentId, onFrontPage, author (public username; omitted with includeAuthor=false), text (plain text; omitted with includeText=false), matchedQueries (keywords / domain:x that found it), updateType (monitor mode: new | changed; otherwise null), previous (changed rows: title, url, points, createdAt), changeHash, fetchedAt. compact rows: id, kind, title, url, hnUrl, points, numComments, createdAt, storyTitle, text, matchedQueries, updateType.

## `runSummary` (type: `string`):

Fields: ok, count, chargeEvent, chargedCount, chargeLimitReached (only when hit), outputFormat, mode, contentTypes, typoTolerance, sinceDate, matchingItems, searches (query, group, pagesRead, hitsReturned, nbHits, stopReason), searchesFailed, truncated, truncationNote, message, warnings, onlyNewSinceLastRun, watermarkStore, watermarkKey, baselineRun, newCount, changedCount, skippedUnchanged, searchesStoppedEarly, apiRequests (total, retries, throttled, capped, minIntervalMs, perRunCap, userAgent), disclaimer. On failure: ok false, error.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "apify"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("brightish_ascent/hacker-news-keyword-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": ["apify"] }

# Run the Actor and wait for it to finish
run = client.actor("brightish_ascent/hacker-news-keyword-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "apify"
  ]
}' |
apify call brightish_ascent/hacker-news-keyword-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,brightish_ascent/hacker-news-keyword-monitor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/5gzdK46EA0zoB3AHo/builds/UDARwEP1KrBB43JFx/openapi.json