# Hacker News Scraper – Stories, Comments, Search & Monitor (`pinkish_armada/hacker-news-scraper`) Actor

Hacker News front page, new, best, Ask HN, Show HN and jobs, or search all of HN by keyword, date, points and author, with full comment threads. Official HN API + Algolia; monitor mode returns only new items.

- **URL**: https://apify.com/pinkish_armada/hacker-news-scraper.md
- **Developed by:** [Phuc Tiet](https://apify.com/pinkish_armada) (community)
- **Categories:** News, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 returned items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hacker News Scraper – Stories, Comments, Search & Monitor

**Get Hacker News data as clean JSON: the live front page, new, best, Ask HN, Show HN and job stories — or search 15+ years of HN stories and comments by keyword, date, points and author, with full comment threads.** Built on the **official Hacker News API** and the **Algolia HN Search API**, so it is fast, complete and never blocked.

- 🔥 **Live lists:** top (front page) with rank, new, best, Ask HN, Show HN, jobs
- 🔎 **Full-history search:** stories, comments, Ask/Show HN, jobs, polls or front-page stories — by keyword, date range, minimum points/comments and author
- 💬 **Comment threads:** attach the nested discussion to each story (depth and size limits) — comments are free
- 🔔 **Monitor mode:** `onlyNewItems` returns only stories/comments you haven't seen before — perfect for brand mentions, competitor launches and topic alerts
- 📈 **No 1,000-result cap for date-sorted searches:** time-windowed paging gets every match
- 🧹 **Clean fields:** decoded text (no HTML entities), domain, story type (ask_hn, show_hn, launch_hn, job ...), ISO dates, HN permalink
- 💸 **$0.50 per 1,000 items**

### Use cases

- **Brand & competitor monitoring:** schedule hourly searches for your product, company or competitors in stories and comments, get only new mentions
- **Launch tracking:** follow Show HN / Launch HN posts in your niche
- **Trend research:** what does HN think about Rust, Postgres, AI agents? Export years of stories with points and comment counts
- **Content curation & newsletters:** daily top stories with minimum points
- **AI / NLP datasets:** discussion threads for sentiment analysis, summarisation and RAG
- **Hiring market research:** aggregate monthly "Who is hiring?" job threads (company postings)

### Pricing

| Event | Price |
|---|---|
| Item returned (story or comment row) | **$0.0005** ($0.50 per 1,000) |

Comment threads attached to stories (`includeComments`) are **included free**. Example: the top 30 front-page stories with their discussions every day ≈ **$0.45 per month**. Monitor mode only charges for new items. Comparable HN Actors in the Apify Store charge about $0.0002–$0.005 per item (October 2026); few offer comment threads, monitoring and unlimited date-sorted search together.

### Input examples

Front page with discussions:

```json
{ "mode": "list", "lists": ["top"], "maxItems": 30, "includeComments": true, "maxCommentsPerStory": 50 }
```

Search stories about a topic with at least 100 points in the last year:

```json
{ "mode": "search", "queries": ["postgres"], "searchIn": "stories", "minPoints": 100, "dateFrom": "1 year", "sort": "date", "maxItems": 500 }
```

Monitor new mentions of your brand in comments and stories (schedule hourly):

```json
{ "mode": "search", "queries": ["yourbrand"], "searchIn": "all", "onlyNewItems": true, "stateStoreName": "yourbrand-hn" }
```

### Output example

```json
{
  "status": "ok",
  "source": "list:top",
  "id": 49977979,
  "type": "story",
  "rank": 1,
  "title": "Mistral Large 4",
  "url": "https://mistral.ai/news/mistral-large-4/",
  "domain": "mistral.ai",
  "text": "https://docs.mistral.ai/models/mistral-large-4-0",
  "author": "Philpax",
  "points": 1406,
  "numComments": 881,
  "createdAt": "2026-10-06T13:15:49.000Z",
  "hnUrl": "https://news.ycombinator.com/item?id=49977979",
  "comments": [
    { "id": 49978056, "author": "crimsoneer", "createdAt": "2026-10-06T13:22:07.000Z", "text": "Woah, this seems like a big deal…", "depth": 0 }
  ],
  "commentsIncluded": 2
}
```

Comment rows (search in comments) include `storyId` and `storyTitle`. Views: **Stories** and **Comments**. Export JSON, CSV, Excel, or use the API.

### Notes

- **Relevance** sorting returns at most the top 1,000 matches per query (an Algolia limit); **newest first** has no such limit.
- Points and comment counts are live at the time of the run.
- Deleted and dead items are skipped.
- Data comes from the official public APIs (`hacker-news.firebaseio.com` and `hn.algolia.com`); no login, no HTML scraping. Usernames are public HN handles; please use the data responsibly and respect the HN community.

### FAQ

**How far back does search go?** The whole history of Hacker News (2006 onwards).

**Can I get every comment on a story?** Yes — set `includeComments` with a high `maxCommentsPerStory` (up to 2,000) and `maxCommentDepth`.

**How do I get alerts?** Use monitor mode with a schedule and add a Slack/email/webhook integration on "Run succeeded".

### Related Actors

- **RSS Feed Reader & Monitor** — monitor blogs and news sites the same way
- **AI Web Scraper** — extract structured data from the articles HN links to
- **Website Screenshot & PDF** — capture the linked pages

# Actor input Schema

## `mode` (type: `string`):

Lists = live HN pages (front page, new, best, Ask, Show, jobs). Search = full-text search across all of Hacker News history.

## `lists` (type: `array`):

Which live lists to fetch, in HN ranking order.

## `queries` (type: `array`):

Keywords to search, e.g. "rust", "postgres", your product or company name. Leave empty with filters to get everything matching the filters.

## `searchIn` (type: `string`):

What to search (search mode).

## `sort` (type: `string`):

Newest first, or by relevance/popularity (relevance is limited to the top 1,000 matches per query).

## `maxItems` (type: `integer`):

Maximum stories or comments per list or query.

## `minPoints` (type: `integer`):

Only stories with at least this many points.

## `minComments` (type: `integer`):

Only stories with at least this many comments.

## `dateFrom` (type: `string`):

Only items created after this date (2026-01-31) or within a period ("7 days", "1 month").

## `dateTo` (type: `string`):

Only items created before this date.

## `author` (type: `string`):

Only items posted by this HN username.

## `includeComments` (type: `boolean`):

Attach the discussion (nested comment tree) to each story. Comments are included free of charge.

## `maxCommentsPerStory` (type: `integer`):

Limit the number of comments attached per story.

## `maxCommentDepth` (type: `integer`):

0 = top-level comments only.

## `onlyNewItems` (type: `boolean`):

Remember returned items in a named key-value store and output only items not seen in previous runs. Schedule it to get alerts for new mentions or new front-page stories.

## `stateStoreName` (type: `string`):

Key-value store used by monitor mode. Use different names for independent monitors.

## Actor input object example

```json
{
  "mode": "list",
  "lists": [
    "top"
  ],
  "queries": [
    "llm agents"
  ],
  "searchIn": "stories",
  "sort": "date",
  "maxItems": 100,
  "includeComments": false,
  "maxCommentsPerStory": 100,
  "maxCommentDepth": 10,
  "onlyNewItems": false,
  "stateStoreName": "hacker-news-state"
}
```

# Actor output Schema

## `results` (type: `string`):

Rank, title, link, points, comments, author, date.

## `allFields` (type: `string`):

Full rows including text and nested comments.

## `summary` (type: `string`):

Items returned and errors.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "llm agents"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("pinkish_armada/hacker-news-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": ["llm agents"] }

# Run the Actor and wait for it to finish
run = client.actor("pinkish_armada/hacker-news-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "llm agents"
  ]
}' |
apify call pinkish_armada/hacker-news-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,pinkish_armada/hacker-news-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Xss8WVaMV5pU0Sv1O/builds/CSI9dH3Ax3xzbo8fY/openapi.json