Hacker News Scraper — Stories, Search & Comments API

Hacker News Scraper exports stories from Hacker News: the front page, newest, best, Ask HN and Show HN lists, keyword search across all of HN since 2006, or specific stories by link. Each story has title, URL, domain, points, comment count, author, date and text, and you can add the full comment thread with authors, replies and depth.

It uses the two official Hacker News APIs, the HN Firebase API and the HN Search API by Algolia. No HTML scraping, no blocking, no proxies: the front page takes about a second.

What it does

• Story lists: top (front page, with rank), new, best, ask (Ask HN) and show (Show HN), up to 500 stories each.
• Search all of Hacker News: keywords in titles, URLs and text; restrict to Ask HN, Show HN, polls or the current front page; sort by relevance or date; up to 1,000 results per query.
• Specific stories: paste HN item links or IDs.
• Comment threads: optional, in HN's ranking order for top-level comments, with replies nested depth-first (depth, parentId). Limit by count or depth.
• Filters: minimum points and "posted since" date in every mode.
• Clean text: Ask HN posts and comments come as readable plain text (paragraphs kept, links expanded to full URLs, HTML entities decoded).

Why use it

• Trend and launch tracking: monitor what's on the front page, or every mention of your product, competitors or tech stack.
• Market and developer research: collect Show HN launches or Ask HN discussions on a topic.
• AI and NLP datasets: export comment threads for summarisation, sentiment analysis or RAG.
• Newsletters and dashboards: schedule daily runs of the best stories above 200 points.

How to use

1. Choose What to scrape: a story list, a search, or specific stories.
2. Set the list, the search query, or paste story links.
3. Optional: set Minimum points, Posted since, Max stories, and turn on Include comments.
4. Click Start and export the dataset as JSON, CSV or Excel, or fetch it with the Apify API.

Input

Field	Default	What it does
mode	list	list, search or stories.
list	top	top, new, best, ask or show (list mode).
searchQuery	empty	Keywords (search mode). Empty = all stories matching the filters.
searchType	story	story, ask_hn, show_hn, front_page or poll.
searchSort	relevance	relevance or date (newest first).
storyIds	none	HN item IDs or links (stories mode).
minPoints	0	Only stories with at least this many points.
sinceDate	none	Only stories posted on or after this date (YYYY-MM-DD).
maxItems	30	Maximum stories (lists up to 500, search up to 1,000).
includeComments	false	Add the comment thread to each story.
maxCommentsPerStory	200	Comment cap per story (up to 5,000).
maxCommentDepth	0	1 = top-level comments only, 2 = plus replies, … 0 = unlimited.
maxConcurrency	5	Parallel API requests.

Examples:

{ "mode": "search", "searchQuery": "rust", "minPoints": 500, "maxItems": 150 }

{ "mode": "search", "searchType": "show_hn", "searchSort": "date", "sinceDate": "2026-10-01", "maxItems": 200 }

{ "mode": "stories", "storyIds": ["https://news.ycombinator.com/item?id=8863"], "includeComments": true, "maxCommentDepth": 3 }

Output

Real example from the default input (front page, Oct 22 2026):

{
  "id": 50195959,
  "title": "Quad9 (DNS Resolver) to Exit France",
  "url": "https://quad9.net/news/blog/au-revoir-france/",
  "domain": "quad9.net",
  "hnUrl": "https://news.ycombinator.com/item?id=50195959",
  "type": "story",
  "author": "farrokhi",
  "points": 960,
  "commentsCount": 346,
  "createdAt": "2026-10-22T14:32:38.000Z",
  "text": null,
  "rank": 1,
  "scrapedAt": "2026-10-23T00:15:36.686Z"
}

type is story, ask, show or poll. rank is the position in the list (list mode only). text holds the body of Ask HN and text posts.

With Include comments on, each story also has comments and commentsIncluded (real example, texts shortened):

{
  "id": 8863,
  "title": "My YC app: Dropbox - Throw away your USB drive",
  "points": 104,
  "commentsCount": 71,
  "comments": [
    { "id": 9224, "parentId": 8863, "depth": 0, "author": "BrandonM", "text": "I have a few qualms with this app:\n\n1. For a Linux user, …", "createdAt": "2007-04-05T15:16:54.000Z" },
    { "id": 9272, "parentId": 9224, "depth": 1, "author": "dhouston", "text": "1. re: the first part, many people want something plug and play. …", "createdAt": "2007-04-05T16:47:01.000Z" }
  ],
  "commentsIncluded": 64
}

commentsCount is HN's total; commentsIncluded can be lower because of your limits and because deleted comments (and their replies) are skipped.

Requested stories that can't be saved are listed with an error and are not charged: NOT_FOUND, NOT_A_STORY (a comment or a job ad), TIMEOUT, NETWORK_ERROR.

Pricing

Pay per result:

Event	Price
Story saved	$0.0002 ($0.20 per 1,000)
Comment included (only with "Include comments")	$0.0001 ($0.10 per 1,000)
Actor start	Apify default ($0.00005)

Examples: the front page (30 stories) costs $0.006; 1,000 search results $0.20; the 10 best stories with all their comments (about 4,400) about $0.44. Set a max cost per run and the Actor stops when the budget is reached; the last story's comments are trimmed to fit (commentsTruncatedByBudget: true).

FAQ

Is this allowed? Yes. It uses the public Hacker News API (published by Y Combinator) and the HN Search API by Algolia, both meant for programmatic access. It does not scrape HTML or log in.

Why are job posts missing? YC job ads (type: job) are skipped in lists and rejected in stories mode. This Actor is for stories and discussions.

Can I get more than 1,000 search results? Algolia returns at most 1,000 hits per query. Split the query by date with Posted since and several runs.

How fresh is the data? Lists and items come live from the HN API. Search results are usually indexed within minutes.

Limitations

• Search covers stories (not comment search) and is capped at 1,000 results per query.
• Replies are ordered chronologically under their parent; only top-level comments follow HN's ranking.
• User profiles and karma are not included.