Hacker News Scraper — Submissions, Jobs, Users & Comments
Pricing
from $0.50 / 1,000 items
Scrape Hacker News top/new/best/ask/show/job/user submissions. Returns title, author, score, comments, URL, and full text. No login required. Pay-per-result.
Keywords to look for, one per entry (for example "python", "rust engineer"). Filters the titles of whichever Story Feed (storyType) is selected. To search HN job posts ("(YC ...) Is Hiring"), set storyType to "job". Each query is matched case-insensitively against the item title (prefix-token match: all words are required, in any order, and a query word also matches longer words that start with it). HN job titles usually embed the company name, so company names match too. A story matching several queries is returned once, tagged with the first matching query. Leave empty to get everything in the feed.
[][ "python", "rust engineer"]Cap per search query when searchQueries is used. Results are split round-robin across queries so one query cannot starve the others. Ignored without searchQueries. If Maximum Stories is 0, the total cap is this value times the number of queries.
Which HN feed to scrape. 'top' = front page (/news), 'new' = newest, 'best' = all-time best, 'ask' = Ask HN, 'show' = Show HN, 'job' = YC jobs (/jobs). Search Queries filter the titles of whichever feed is selected here, so to search HN job posts ("(YC ...) Is Hiring") with searchQueries, set this to 'job'.
Specific HN story/thread IDs to scrape (e.g. [47601859]). Overrides storyType. Use with includeComments to extract comments from any thread.
HN usernames to scrape all submissions from (e.g. ["whoishiring", "pg"]). Returns all their stories and comments.
Also extract comments for each story. Useful for 'Who is Hiring' threads, Ask HN discussions, etc.
How deep to fetch comment replies. 1 = top-level only, 2 = include direct replies, 3 = replies to replies. Max 5.
Maximum number of stories to return from the feed. With searchQueries this caps the total across all queries (0 = maxResultsPerQuery x number of queries).