Hacker News Scraper: Stories, Comments, Search, Users (API)
Pricing
from $0.14 / 1,000 output items
Get Hacker News data from the official APIs: top, new, best, Ask HN, Show HN and job stories, keyword and date search, full comment threads, and user profiles. Points, comments, authors and dates as clean rows, with an only-new mode for monitoring.
Lists: the stories on HN's front-page lists. Search: Algolia's HN search by keyword, tag, date and points. Items: specific stories or comments by ID or URL. Users: user profiles, optionally with their latest submissions.
Lists mode. One or more of the official story lists. A story on several lists is output once (with its rank in each list in listRanks).
[ "top"]Lists mode: how many stories to read from the top of each list (the API gives up to 500 for top, new and best, 200 for Ask, Show and jobs). Search mode: how many search results to read in total (more than 1,000 needs Sort = Date; see the README).
Search mode. Words to look for in titles, URLs, story text and comments (Algolia full-text search). Leave empty to get everything that matches the tags and dates.
Search mode. Date: newest first, and can read more than 1,000 results (it moves to older time windows automatically). Relevance: Algolia's ranking (text match, then points), at most 1,000 results.
Search mode. A date (2026-10-01), a time (2026-10-01T08:00:00Z) or a period back from now ('7 days', '24 hours').
Search mode. Same formats. A date on its own includes that whole day (UTC).
Search mode. Only stories with at least this many points (comments have no points, so this leaves them out).
Search mode. Only stories with at least this many comments.
Items mode. Item IDs (8863) or Hacker News URLs (https://news.ycombinator.com/item?id=8863): stories, comments, jobs or polls.
Users mode. Usernames (case-sensitive) or profile URLs (https://news.ycombinator.com/user?id=pg). Each gives one row: created date, karma, about text and number of submissions.
Users mode. Also output each user's latest N submissions, newest first (0 = profile only, up to 1,000). Each submission is an output row.
Users mode. Which submissions to list.
Also get each story's comment thread (one Algolia request per story with comments). Each included comment is charged like a row; see the pricing section.
Comments are taken in thread order (top-level comments in HN's ranking, each followed by its replies) up to this many.
Story text, comments and user 'about' come from HN as HTML. Plain text (links written out), Markdown, or the original HTML.
Off: deleted and dead (flagged or killed) stories are left out, and deleted comments are dropped unless they have replies (then they stay as a free placeholder row so the thread stays connected). On: all are output with deleted / dead set to true.
Output only stories (and search results) not output by an earlier run with the same settings. The first run outputs everything it finds. Memory: the last 50,000 IDs, in the key-value store 'hacker-news-scraper-state'.
Optional. Runs with the same memory name share their 'only new' memory. By default each saved task has its own memory per set of settings. Letters, digits, '-', '_' and '.'.
How many item requests to send to HN's Firebase API at the same time (1–20). Algolia requests are always limited to 2 at a time and spaced out to stay within its public rate limit.
Comments format
commentsFormat
Optional
Nested: a 'comments' array on the story row, replies inside replies. Flat: one row per comment after its story, with storyId, parentId and depth (best for spreadsheets and filtering).