Hacker News Stories & Comments Scraper
Pricing
from $2.49 / 1,000 results
Hacker News Stories & Comments Scraper
Hacker News Scraper SD - Hacker News Scraper is a news data tool that extracts Hacker News stories and comments from the front page, new, best, Ask HN, Show HN, jobs and Who is hiring threads, plus full-text search and comment trees via the official APIs.
Pricing
from $2.49 / 1,000 results
Rating
0.0
(0)
Developer
Neuro Scraper
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
15 days ago
Last modified
Categories
Share
Hacker News Scraper — stories, comments, front page and job posts
Hacker News Scraper collects Hacker News stories and comments through the two official public APIs: the HN Firebase API run by Y Combinator, and the Algolia HN Search API. It needs no account and no key, and it never loads the news.ycombinator.com pages.
It does four things, in any mix:
- Lists → the live front page (
top),new,best, Ask HN, Show HN and YC job posts, in Hacker News' own order (rank). - Who is hiring? → the top-level job posts of the latest monthly "Ask HN: Who is hiring?" thread.
- Search → full-text search of stories, comments or both, by relevance or newest first, with date and points filters.
- Story links → one story (or comment) with its whole comment tree, each comment with its
depth.
Input
| Field | Type | Default | Meaning |
|---|---|---|---|
lists | multi-select | top (prefill) | top, new, best, ask, show, job, whoishiring |
searchTerms | list | – | Search terms, one per line |
searchType | select | story | Stories, comments, or stories + comments + polls |
searchSort | select | relevance | Relevance (max 1,000 results) or newest first (no cap) |
fuzzyMatching | boolean | false | Algolia typo tolerance; off gives complete, stable results |
dateFrom, dateTo | YYYY-MM-DD | – | Optional date window for searches |
minPoints | integer | – | Optional minimum points (stories only) |
storyUrls | list | – | https://news.ycombinator.com/item?id=... links or ids → item + comment tree |
maxResults | integer | 30 | Rows per list, search term or story link |
maxItems | integer | – | Optional cap on the whole run |
proxyConfiguration | proxy | off | Not needed for the official APIs |
Output (one row per story or comment)
| Field | Meaning |
|---|---|
type | story, comment, job or poll |
id, hnUrl | Hacker News item id and link |
title, url, domain | Story title, linked URL and its site |
text | Comment text, Ask HN / job text, as plain text (links kept) |
author, points, commentCount, createdAt | Author, score, number of comments, post time (UTC) |
storyId, storyTitle, parentId | For comments: the story and the parent item |
depth | Story links: 0 = the item, 1 = top-level comment, ...; Who is hiring: 1 |
rank | Lists: position in the live list (1 = top of the front page) |
source, sourceInput, scrapedAt | list / whoishiring / search / item, your input, UTC time |
Measured limits (2026-10-03, local runs)
- Lists hold what Hacker News publishes: top 500, new 500, best 200, Ask HN 17, Show HN 148, jobs 31 ids at the time of testing.
- List items are read 50 at a time from Algolia (points and comment counts matched Firebase exactly in our check); items Algolia does not have yet, and all job posts, come from Firebase one by one.
- Relevance search stops at 1,000 results (Algolia's limit). Newest-first search walks back in time and has no cap: 1,500 results in 32 seconds in our test.
- With fuzzy matching on, Algolia answers filtered searches non-exhaustively: the same query returned between 52 and 175 stories on repeated calls. Exact-word matching (the default) returned the same 50 every time.
- Deleted and dead comments are not in the comment tree.
- Speed: 781 rows (three lists + a 250-result search) in 51 seconds, about one request per second.
Example
Hacker News content is written by its users: keep the author and hnUrl for attribution.
