Hacker News Scraper: Stories, Comments & Who's Hiring Jobs

Hacker News scraper: search stories and comments, export the front page and comment threads, and turn the monthly "Who is hiring?" thread into structured job rows. Official Hacker News APIs only, no HTML scraping, no API key, no login. Turn on Only new items and schedule the run to monitor brand mentions, Show HN launches and new job posts. From $0.50 per 1,000 stories and comments.

What you get

• Search all of Hacker News since 2006 by keyword, content type (stories, comments, Show HN, Ask HN, Launch HN, polls), author, date range, minimum points and minimum comments. Sort by popularity or by date. Date order has no 1,000 hit limit.
• Front page and feeds: top, new, best, Ask HN, Show HN and job feeds with the live rank of each story.
• Threads with comments: give one or many story links and get the story plus its comments with depth, rank, author and parent. Limit the reply depth and the number of comments.
• Who is hiring parser: every job post of the monthly "Ask HN: Who is hiring?" thread as one row with company, role, location, remote, hybrid or onsite, visa sponsorship, salary range and currency, employment type, company site, apply link, e-mail domain and tech keywords. Filter by keyword, location, remote, visa and salary.
• Only-new monitoring: the Actor remembers what earlier runs saved and returns only items you have not seen. Nothing new ends the run SUCCEEDED with an empty dataset.
• Clean text: HN HTML becomes plain text. Links keep their full address, entities are decoded.

Typical users: founders and marketers who track mentions of a product, competitor or keyword, recruiters and job seekers who read Who is hiring, researchers who study discussions, and AI agents that need Hacker News as structured data.

Quick start

1. Click Try for free. The prefilled input searches stories about AI agents from the last year with at least 100 points and saves 20 of them.
2. Change Mode and the fields for your task. Fields of other modes are ignored.
3. Click Start. Download the dataset as JSON, CSV, Excel or HTML, or read it through the Apify API.

Modes

Mode	What it does	Main fields
search	Stories, comments, Show HN and more by keyword, date, author, points	queries, contentTypes, sortBy, author, createdWithinDays, minPoints
frontPage	A live HN list with the rank of each story	feed (top, new, best, ask, show, job)
thread	One or more stories with their comments	storyUrls, commentDepth, maxCommentsPerStory
whoIsHiring	Job posts of a monthly Who is hiring thread, parsed into fields	hiringMonth, jobKeywords, jobLocations, remoteOnly, visaSponsorshipOnly, salaryOnly

If you leave mode out, it is guessed: story links mean thread, search fields mean search, hiring filters mean whoIsHiring, anything else means frontPage. A run with no input at all returns one free help row.

Input examples

Remote Python jobs from the latest Who is hiring thread:

{
    "mode": "whoIsHiring",
    "hiringMonth": "latest",
    "remoteOnly": true,
    "jobKeywords": ["python"],
    "maxItems": 100
}

Show HN launches of the last 7 days with at least 50 points:

{
    "mode": "search",
    "contentTypes": ["show_hn"],
    "createdWithinDays": 7,
    "minPoints": 50,
    "sortBy": "date",
    "maxItems": 100
}

Mentions of a brand in stories and comments, new ones only:

{
    "mode": "search",
    "queries": ["apify"],
    "contentTypes": ["story", "comment"],
    "onlyNew": true,
    "maxItems": 200
}

A thread with the first two reply levels:

{
    "mode": "thread",
    "storyUrls": ["https://news.ycombinator.com/item?id=8863"],
    "commentDepth": 2,
    "maxCommentsPerStory": 200
}

The 30 top stories on the front page right now:

{
    "mode": "frontPage",
    "feed": "top",
    "maxItems": 30
}

The prefill, stories about AI agents:

{
    "mode": "search",
    "queries": ["AI agents"],
    "contentTypes": ["story"],
    "sortBy": "popularity",
    "createdWithinDays": 365,
    "minPoints": 100,
    "maxItems": 20
}

Monitoring with only-new runs

1. Set the search you want to follow, for example queries: ["apify"] with contentTypes: ["story", "comment"].
2. Turn on Only new items since the last run.
3. Save the input as a Task and add a Schedule, for example every hour or every morning.
4. Add an integration to the task: Slack, e-mail, Google Sheets, Make, Zapier, n8n or a webhook on "Run succeeded".

How it works:

• The first run saves the current matches (newest first, up to Max items) and remembers them.
• Later runs with the same settings save only items that earlier runs did not save. You pay only for new items.
• Search mode goes by date, newest first, and looks back a few hours past the last run, because Algolia can index items late. With a points or comments filter it looks back three days, because older stories can reach the threshold later.
• Front page mode watches the top N places of the feed (N is Max items) and returns stories that were not delivered before.
• Who is hiring mode returns job posts you have not seen. With hiringMonth: "latest" a new month starts fresh.
• Thread mode returns new comments of the threads you follow.
• Nothing new ends the run SUCCEEDED with an empty dataset and the status "Nothing new since the last run". Integrations that react to dataset items stay quiet.
• The memory lives in the named key-value store hackernews-scraper-only-new in your own Apify account. The record name is a hash of your search settings, so a changed query starts a new memory. Set State name (stateKey) to keep one memory across changes, or to run two separate monitors.
• If a run stops at Max items or at your cost limit, the next run picks up what was left, except in front page mode where the window is fixed.

Output

Every row has the same keys. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel. Stories, comments and job posts use the same table. Job posts add the job fields.

A story from search mode:

{
    "id": 47460525,
    "type": "story",
    "title": "OpenCode – Open source AI coding agent",
    "url": "https://opencode.ai/",
    "domain": "opencode.ai",
    "author": "rbanffy",
    "points": 1274,
    "numComments": 618,
    "createdAt": "2026-03-20T21:03:52.000Z",
    "hnUrl": "https://news.ycombinator.com/item?id=47460525",
    "text": null,
    "tags": [],
    "storyId": 47460525,
    "storyTitle": "OpenCode – Open source AI coding agent",
    "parentId": null,
    "depth": null,
    "rank": null,
    "matchedQuery": "AI agents"
}

A comment from thread mode:

{
    "id": 49944047,
    "type": "comment",
    "title": null,
    "url": null,
    "domain": null,
    "author": "driverdan",
    "points": null,
    "numComments": null,
    "createdAt": "2026-10-03T13:23:26.000Z",
    "hnUrl": "https://news.ycombinator.com/item?id=49944047",
    "text": "IMO the announcement is better https://aleph-alpha.com/en/blog/kolibri-has-landed-a-sovereign-open-weight-model/",
    "tags": [],
    "storyId": 49943034,
    "storyTitle": "Aleph Alpha Kolibri: How the sovereign German LLM works",
    "parentId": 49943034,
    "depth": 1,
    "rank": 2,
    "matchedQuery": null
}

A job post from Who is hiring mode (text shortened here):

{
    "id": 49924974,
    "type": "job_post",
    "company": "Curai",
    "role": null,
    "location": "Remote (US)",
    "remote": true,
    "hybrid": false,
    "onsite": false,
    "visaSponsorship": null,
    "salaryText": null,
    "salaryMin": null,
    "salaryMax": null,
    "salaryCurrency": null,
    "salaryPeriod": null,
    "employmentType": "full-time",
    "companyUrl": "https://curaihealth.com",
    "applyUrl": "https://jobs.lever.co/curai/46e0dd2f-c1dd-4ca2-9095-5b45f477b656",
    "applyEmailDomain": null,
    "techKeywords": ["Python", "React", "React Native", "PostgreSQL", "DynamoDB", "AWS", "Docker", "Terraform"],
    "header": "Curai | Remote (US) | Full-time | https://curaihealth.com",
    "author": "hackily",
    "createdAt": "2026-10-01T18:02:35.000Z",
    "hnUrl": "https://news.ycombinator.com/item?id=49924974",
    "threadId": 49922569,
    "threadTitle": "Ask HN: Who is hiring? (October 2026)",
    "threadMonth": "2026-10",
    "rank": 8,
    "numReplies": 0,
    "text": "Curai | Remote (US) | Full-time | https://curaihealth.com\n\nWe're building AI agents to expand access to affordable, and high quality healthcare. ..."
}

The settings the run used, counts, the thread found and only-new statistics are saved in the SUMMARY record of the run's key-value store.

Data fields

Field	Meaning
id, hnUrl	Hacker News item ID and the link to the item. The ID is a stable key for deduplication.
type	story, comment, poll, job (a YC job ad) or job_post (a parsed Who is hiring comment).
title, url, domain	Story title, the link it points to and that link's host. Empty for comments and Ask HN.
author, createdAt	Hacker News username and creation time in UTC.
points, numComments	Story score and live comment count. Empty for comments.
text	Plain text of a comment, Ask HN or job post.
tags	show_hn, ask_hn, launch_hn, tell_hn or front_page.
storyId, storyTitle, parentId	The story a comment belongs to and its direct parent.
depth	Comment level. 1 is a top-level comment.
rank	Position in the feed (front page mode), in the thread (comments) or in the hiring thread (job posts).
matchedQuery	The search query that found the row.
company, role, location	Parsed from the first line of a job post. role can hold several roles.
remote, hybrid, onsite	Workplace as the post states it.
visaSponsorship	true when the post offers it, false when it says it does not, null when it is silent.
salaryText, salaryMin, salaryMax, salaryCurrency, salaryPeriod	First plausible salary in the post. Funding rounds and equity are ignored.
employmentType	full-time, part-time, contract, internship or freelance, from the header.
companyUrl, applyUrl	Company site and the application or careers link.
applyEmailDomain	Domain of a contact e-mail. Addresses in text and header are redacted to [redacted]@domain.
techKeywords	Technologies named in the post, from a list of about 80 common ones.
header, threadId, threadTitle, threadMonth, numReplies	The first line as written, and where the post comes from.

The job fields are heuristics. Posts follow a loose convention, so a field can be empty or wrong. The original header and text are always kept so you can check.

Pricing

Pay per event. No rental, compute included.

Event	Free plan	Bronze	Silver	Gold
Story, comment or poll saved (item)	$0.0005	$0.00045	$0.0004	$0.00035
Per 1,000 items	$0.50	$0.45	$0.40	$0.35
Who is hiring job post saved (job-post)	$0.001	$0.0009	$0.0008	$0.0007
Per 1,000 job posts	$1.00	$0.90	$0.80	$0.70
Run start	$0.001	$0.001	$0.001	$0.001

• The prefilled run, 20 stories: about $0.011.
• 1,000 search results: about $0.50.
• A thread with 500 comments: about $0.25.
• The whole Who is hiring thread, about 225 job posts: about $0.23.
• Remote Python jobs of one month, about 30 posts: about $0.03.
• A brand monitor with about 5 new mentions a day: about $0.11 a month.
• Help rows for invalid input, empty results and "nothing new" runs are free. You pay only the start fee.

Comments count as items, so Include comments and thread mode add to the bill. Set Maximum cost per run in the run options to cap spend. The Actor saves only as many rows as that budget covers.

Tips

• Algolia matches whole words. scrap will not find scraping, so search for the exact words.
• Add the content type Comments to find product and brand mentions inside discussions. Most mentions live in comments.
• Popularity order returns at most 1,000 hits per query. Use sortBy: "date" with a date range to export more.
• Several queries share Max items. Unused share passes to the next query.
• A story link in storyUrls can also be a comment link. You get that comment with its replies.
• Set maxCommentsPerStory: 0 in thread mode to read only the story rows of a list of links, for example to refresh points and comment counts.
• When there are more comments than the limit, the Actor keeps the shallow ones first, so you see top-level comments before deep replies.
• Who is hiring threads exist from April 2011. The latest month fills up over its first days.

FAQ

Is it legal to scrape Hacker News?

The Actor reads two public APIs that exist for this purpose: the HN Search API by Algolia and the official Hacker News API on Firebase. It does not log in and does not scrape web pages. Posts and comments belong to their authors, so check the rules of your use case before you republish text. This Actor is not affiliated with or endorsed by Y Combinator or Hacker News.

Does it collect personal data?

Usernames and the text of public posts and comments are public data, but they can still be personal data. Handle them under GDPR if you process them further. The Actor does not look up profiles and does not save e-mail addresses from Who is hiring posts, only the domain. Comments and other text are saved as written, so an address typed into a comment is not removed there.

How accurate is the job parser?

On the October 2026 thread, 226 of 232 top-level comments were recognised as job posts. The other six were meta comments. A location was found for about 88 percent of posts and a salary for about 37 percent. Posts that ignore the usual Company | Role | Location header give weaker results. Check header and text when it matters.

Which dates and items are covered?

Search covers Hacker News from 2006 to a few minutes ago. A new item can take a few minutes to show up in search. Thread and front page modes use the Firebase API, which is live. If Algolia does not know an item yet, thread mode loads it from Firebase.

What happens if my input is invalid?

The run still ends SUCCEEDED and you pay only the small start fee. No item is charged. The dataset gets one help row instead:

{
    "error": true,
    "errorCode": "EMPTY_INPUT",
    "message": "The run got no input, so there is nothing to look up.",
    "howToFix": "Choose a mode and fill its fields. Search: {...}"
}

The same text is shown as the run status message. Codes: EMPTY_INPUT, INVALID_INPUT (a wrong value, with the field named), NO_RESULTS (the search ran but nothing matched), ITEM_NOT_FOUND (a thread link points to a missing item), HIRING_THREAD_NOT_FOUND and NOT_A_HIRING_THREAD. Fix the input as howToFix says and run again. If both APIs are down, the run fails, so you can tell outages apart from input problems.

Rate limits

The Actor makes one Algolia request at a time and at most 10 Firebase requests at once. It retries rate limit and server errors with exponential backoff.

Can I use it from an AI agent (MCP)?

Yes. The Actor works as a tool through the Apify MCP server. All inputs are optional, lists work well, and invalid input returns a readable help row instead of a failed run.

Support

Found a bug or need a field? Open an issue in the Issues tab.

Related Actors

More tools from the same developer, built with the same pricing and output style.

Actor	What it does
Website Content Crawler to Markdown for RAG & LLMs	Crawl whole websites into clean Markdown, RAG chunks and llms.txt
PDF to Markdown & Text Extractor with OCR	PDF, Word, Excel and PowerPoint files to Markdown and text, with OCR
SEO Audit Tool: Site Crawler, Broken Links & Score	Crawl a site and find broken links and on-page SEO issues
PageSpeed Insights & Lighthouse Bulk Checker	Lighthouse scores and Core Web Vitals for many URLs at once
Tech Stack Detector: Wappalyzer & BuiltWith Alternative	Detect the CMS, frameworks, analytics and hosting of any website
Bulk Domain Lookup: DNS, WHOIS, SPF, DMARC & SSL	DNS, email security and SSL checks for a list of domains
Speech to Text: Audio, Video & Podcast Transcriber	Whisper transcripts, SRT and VTT subtitles from audio, video and podcast feeds
Email Validator & Verifier: Bulk MX & Disposable Check	Check email lists for syntax, MX records and disposable domains
Email Format Finder: Company Email Pattern Lookup	Find the email format a company uses, such as first.last or flast
Work Email Finder: Find Email by Name and Domain	Find a person's work email from a name and a company domain
Website Contact Scraper: Emails, Phones & Social Links	Find emails, phone numbers and social profiles on company websites
VIES VAT Checker & EU Company Registry Lookup	Validate EU VAT numbers and look up company registry data
Companies House Scraper: UK Company Search, No API Key	Search UK companies and get profiles and officers
EU TED Tenders Scraper & Alerts: Public Procurement	Search and monitor EU public tenders with daily only-new alerts
Greenhouse, Lever & Ashby Job Scraper: 8 ATS in One Feed	Job postings from 8 applicant tracking systems in one schema
GitHub Scraper: Repo Stats, Stars, Trending & Search	Repository stats, stars, trending repos and search results