Hacker News Scraper: Stories, Search & Comment Analysis

Get the Hacker News front page, any HN list, or a search across all of HN history, with every story's comment thread already read and summarized: who is talking, which terms keep coming up, and which comments drew the most replies.

Launch Sale: 90% OFF for the first week! Every price below is cut by 90% during launch week, and you still pay only for results that come back.

Quick start

Press Start. The sample pulls the current top 10 front-page stories and reads up to 50 comments on each. It finishes in about 10 seconds. Switch What to scrape to Search all of HN and type a topic to track it.

What you get

• One row per story: rank, title, link, domain, author, points, comment count, posted time, type (story, Ask HN, Show HN, job)
• Thread analysis on every story: comments read, unique commenters, deepest reply level, top 5 commenters, top 10 terms, the 5 most-replied top-level comments with links
• Optional: every comment as its own row (author, text, depth, reply count, parent, link)
• Search mode: Algolia full-text search over all HN stories or comments, by relevance or newest first, with point and date filters
• A RUN_SUMMARY.json record with counts and any skipped items

Good for

• Tracking mentions of your product, competitors or a technology on HN
• Daily front-page digests for a newsletter or Slack
• Launch-day monitoring of your Show HN post and its discussion
• Building datasets of developer opinion for research or LLM evaluation

Pricing

Launch Sale: 90% OFF for the first week!

• $0.002 per story (thread analysis included) — $2 per 1,000
• $0.0003 per comment row, only if you turn comment rows on
• The sample run (10 stories) costs $0.02

You pay only for results that come back. Status rows (not found, errors, skipped) are free. Set a max cost per run in the run options and the run stops cleanly when it hits it. Normal Apify platform usage applies.

Input

Field	What it does
mode	Pick a live Hacker News list, or 'search' to query all of HN history through Algolia.
searchQuery	Used when 'What to scrape' is 'search'. Example: LLM agents, Rust, Series A, your brand name.
maxItems	How many stories to return.
analyzeComments	Read each story's full thread and add top commenters, key terms and the most-replied comments. Included in the story price.
maxCommentsPerStory	Comments are read top-down (most visible first).
outputCommentRows	Adds one dataset row per comment (charged per comment). Leave off if the per-story analysis is enough.
minPoints	Skip stories below this score. 0 keeps everything.
sinceDays	Keep stories posted in the last N days. 0 means no date limit.
searchTags	What kind of item the search matches (search mode only).
sortBy	Relevance (Algolia ranking, favours popular items) or newest first.

Output

Results go to the default dataset (table view in the Console, or JSON/CSV/Excel export). A RUN_SUMMARY.json record in the key-value store has counts, failures and the prices used.

Limits

• Uses the official Hacker News Firebase API and Algolia HN Search. Both are public; no login needed.
• Algolia returns at most 1,000 hits per search. Narrow by date for more.
• Deleted and dead items are skipped. Scores and comment counts are as of the run.
• All collection is done by this actor directly against public sources. It never calls other paid actors.

Run it from code

Same input as the Console sample:

{
  "mode": "top",
  "searchQuery": "AI agents",
  "maxItems": 10,
  "maxCommentsPerStory": 50
}

curl -X POST "https://api.apify.com/v2/acts/physealabs~hackernews-intelligence-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" -d @input.json

from apify_client import ApifyClient
items = ApifyClient("YOUR_APIFY_TOKEN").actor("physealabs/hackernews-intelligence-scraper").call(run_input={"mode": "top", "searchQuery": "AI agents", "maxItems": 10, "maxCommentsPerStory": 50})

It also works from Zapier, Make, n8n and Apify schedules.

About

Built by Physea Labs · Liminal, the team behind Liminal, a router that turns hard questions into grounded, checkable answers. Questions or a source you want added? Open an issue on the actor page.