Hacker News Scraper

Search Hacker News and export stories, comments and user data as clean JSON: points, author, timestamps and direct links. One click, no required fields, no LLM.

What it does

Hacker News is where a large part of the software industry argues, launches and changes its mind in public. This actor searches the full HN archive — not just the front page — and returns stories or comments as clean JSON: title, external link, HN discussion link, author, points, comment count and timestamp. Search by relevance or by date, filter by score, or pull the current front page with one setting. Comments come back with their parent story's title attached, so a comment is never delivered without context. No API key, no login, no proxy.

Why this one:

• Searches the entire archive, not only the front page
• Stories and comments from one actor, with parent-story context preserved
• Relevance or date ordering, plus a minimum-score filter
• Reads the open HN search index — no API key, no login, no proxy
• No LLM anywhere — deterministic output, predictable costs, no hallucinated fields
• Clean by default — canonical URLs (tracking parameters stripped), parsed numbers, merged duplicates

Quick start (no code)

1. Click Try for free / Start — every field has a working default, nothing is required.
2. (Optional) change query to what you need.
3. Open the Dataset tab when the run finishes → export as JSON, CSV or Excel.

Input

Field	Required	Default	Description
query	no	artificial intelligence	What to search for. Leave empty and set Search type to front_page to pull the current front page instead.
searchType	no	story	story, comment, front_page, ask_hn, show_hn or poll.
sortByDate	no	false	Off: best matches first. On: newest first — what you want for scheduled monitoring.
minPoints	no	0	Only return items at or above this score. 0 means no filter.
maxResults	no	20	Maximum clean results (capped at 500)
enrich	no	false	Deterministic enrichment per record — see below
monitor	no	false	Compare with the previous run, flag NEW records only

Example input:

{
  "query": "rust",
  "searchType": "story",
  "minPoints": 100,
  "maxResults": 50
}

Output

Real example record (from a live run):

{
  "id": "22238335",
  "title": "Why Discord is switching from Go to Rust",
  "url": "https://blog.discordapp.com/why-discord-is-switching-from-go-to-rust-a190bbca2b1f",
  "discussion_url": "https://news.ycombinator.com/item?id=22238335",
  "seller": "Sikul",
  "points": 1582,
  "num_comments": 642,
  "created_at": "2020-02-04T17:30:40Z",
  "tags": ["story", "author_Sikul", "story_22238335"],
  "completeness": 0.5
}

The final _summary row carries run totals (total_clean, deduped, enriched); in monitor mode a _changes row lists keys new since the last run.

Field reference

Field	Meaning
id	Hacker News item id (stable, deduplication key)
title	Story title. On comment records this is the parent story's title, so comments arrive with context
url	External link when the story points off-site, otherwise the HN discussion URL — always filled
discussion_url	The HN discussion page, always present alongside the external link
seller	HN username of the author. The field is named seller because the shared schema maps author to that slot — the value is the account name
points / num_comments	Score and comment count at scrape time
created_at / tags	ISO timestamp and Algolia tags (story, comment, front_page, author_<name>)
story_id / description	Parent story id and the comment or post text, when the item has one
price / price_text	Parsed numeric value + original text, when the source publishes one
completeness	0–1 filled-fields score (with enrich)

Use cases

• Launch monitoring — watch every mention of your product or competitor, sorted by date, on a schedule.
• Developer research — measure how a language, framework or company is discussed over years, not just today.
• Trend detection — pull front_page on a schedule and track what reaches it and how fast.
• Sentiment and argument mining — export the comment tree of a story and analyze it in your own stack.
• AI agents — let an agent answer 'what did HN say about X?' with real posts and scores via MCP.

Enrichment (optional, charged only when it produces something)

Set enrich: true and every record additionally gets: e-mail addresses extracted from the description (when present), the canonical domain of the record's URL, and a completeness score (0–1, how many core fields are filled). Deterministic — the same input always yields the same output — and you are only charged for records that actually got enriched. Records where enrichment adds nothing are free.

Monitor mode — change alerts on a schedule

Set monitor: true and the actor compares the current run with the previous one (per-actor named storage) and flags only NEW records. Combine with Apify Schedules for a daily/hourly watch: the _changes summary row lists what appeared since the last run, and the change-alert event is charged per new record only — an unchanged run costs you almost nothing.

Use it from your code

Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_API_TOKEN>")
run = client.actor("EnezLi/hacker-news-scraper").call(run_input={  "query": "rust",  "searchType": "story",  "minPoints": 100,  "maxResults": 50 })
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_API_TOKEN>' });
const { defaultDatasetId } = await client.actor('EnezLi/hacker-news-scraper').call({  "query": "rust",  "searchType": "story",  "minPoints": 100,  "maxResults": 50 });
const { items } = await client.dataset(defaultDatasetId).listItems();
console.log(items);

curl

curl -X POST "https://api.apify.com/v2/acts/EnezLi~hacker-news-scraper/run-sync-get-dataset-items?token=<YOUR_API_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{  "query": "rust",  "searchType": "story",  "minPoints": 100,  "maxResults": 50 }'

Use it with AI agents (MCP)

This actor is agent-ready: it appears in Apify's AGENTS / MCP servers catalog, so any MCP-capable assistant (Claude, custom agents, LangGraph tools) can discover and call it with a one-line tool call — zero required fields means an agent can run it safely with defaults. Connect your agent to the Apify MCP server and ask for live data in natural language.

Pricing — Pay-Per-Event, start is free

Event	When charged
Actor start	Free ($0) — try it with one click
result	Per clean result returned
enrichment	Only per record that actually got enriched
change-alert	Monitor mode: per NEW record since the previous run

No subscription, no minimum. Volume discounts apply automatically through Apify account tiers (up to −44% on GOLD). Typical run cost example: 20 results ≈ a few cents total — you can predict your bill from the numbers above before you run.

Is this legal?

This actor collects publicly available data only — the same information any visitor sees in a browser, via public endpoints. It does not bypass logins, collect private personal data, or store credentials. You are responsible for using the output in compliance with the source site's terms and the laws that apply to you (e.g. GDPR when the output contains personal data).

This Actor is not affiliated with, endorsed by, or connected to Hacker News or Y Combinator. All product names are trademarks of their respective owners.

Support & feedback

Found a bug, need another field, or want a variant for a related platform? Open an issue on the Issues tab — issues are monitored and answered, and frequently-requested fields get added to the standard output. The actor is maintained as part of a scraper family built on one shared, tested core (bugs fixed once are fixed everywhere).

Changelog

• 0.1 (2026-07) — initial public release: search, dedup, optional enrichment, monitor mode, PPE pricing.

Limitations (honest ones)

Public Hacker News content only. Scores and comment counts are snapshots at scrape time, not live values. Deleted or flagged items are not returned, and comment threads come back as flat records rather than a nested tree.

FAQ

Can I get comments, not just stories?

Yes — set Search type to comment. Each comment carries its parent story's title and id, so you never get a wall of contextless text.

How do I pull the front page?

Set Search type to front_page and leave the query empty. The query field is then not sent at all, so nothing narrows the result.

Why is the author in a field called seller?

All actors in this family share one normalization core, and that core maps the author role to seller. The value is the HN username. The dataset schema documents this explicitly so nothing is surprising downstream.

Are points and comment counts live?

They are the values at the moment of scraping. HN scores keep moving, so for tracking over time run on a schedule and keep the timestamps.

Do I need an API key or account on the source platform?

No. The actor uses public endpoints — you only need your Apify account.

Does it use AI / an LLM?

No. The core is fully deterministic: same input, same output, no hallucinations, no per-token costs.

Can I run it on a schedule?

Yes — use Apify Schedules; combine with monitor mode to pay only for what's new.

What's the maximum number of results?

500 per run (memory-safe cap). Run multiple queries or schedule runs for more.

How is my bill calculated?

Only from the events in the Pricing table — start is free, and there is no subscription.