Hacker News Scraper: Stories, Comments, Search, Jobs

This Hacker News scraper returns stories and comments from Hacker News as clean rows: the live front page and the newest, best, Ask HN, Show HN and jobs lists, keyword search across all of Hacker News since 2006, full comment threads, any user's submissions, and the monthly "Who is hiring?" thread. It uses the official Hacker News API and Algolia's HN Search API, so runs are fast, cheap and need no login or API key.

• Live lists: the front page in its current order, with rank, points and comment count read at run time.
• Search: keywords with a date range, a minimum points value and relevance or newest-first order. Newest-first searches go past Algolia's 1,000-result limit.
• Comments: every comment under a story with author, depth and parent, as separate rows you can sort and filter.
• Who is hiring: one row per job post from any month's thread, with the post's first line (usually Company | Role | Location) and a remote flag. Filter by words such as rust remote.

What data you get

One item per story or comment:

Field	What it holds
id, hnUrl	Hacker News item ID and link
type, category	story, comment, job or poll; category also tells Ask HN, Show HN, Launch HN and Tell HN apart
title, url, domain	Story title, the link it points to and that link's website
author	Username
points, numComments	Story points and total comments. For a comment, numComments is the number of replies under it
createdAt, createdAtUnix	When it was posted (UTC)
text	Comment text, or the body of an Ask HN, Show HN or job post, as plain text with full link URLs
parentId, storyId, storyTitle, depth	Where a comment sits: what it replies to, the story it belongs to and how deep it is nested
rank	Position on the live list at run time (list modes)
hiringHeadline, isRemote	First line of a job post and whether it mentions remote (Who is hiring mode)
source	The list, search, user, story or thread the item came from

How to use it

1. Pick What to scrape.
2. Fill the field for that mode: nothing for live lists, a Search query for search, Story IDs or URLs for comments, Usernames for user mode, or a Month for Who is hiring.
3. Optional: set From date, To date, Minimum points, and turn on Include comments to get each story's comments after it.
4. Run it and download JSON, CSV or Excel, or call it from the API, Make, Zapier, n8n or an AI agent through the Apify MCP server.

Example input

{
  "mode": "search",
  "query": "rust",
  "dateFrom": "2024-01-01",
  "dateTo": "2025-12-31",
  "minPoints": 300,
  "maxItems": 5,
  "includeComments": true,
  "maxCommentsPerStory": 3
}

Example output (from real runs)

A story from the front page:

{
  "id": 50289418,
  "type": "story",
  "category": "story",
  "title": "We're releasing the weights of Mistral Large 4",
  "url": "https://mistral.ai/news/mistral-large-4-oss/",
  "domain": "mistral.ai",
  "author": "kuberwastaken",
  "points": 1219,
  "numComments": 341,
  "createdAt": "2026-10-29T14:13:21.000Z",
  "createdAtUnix": 1793283201,
  "text": null,
  "storyId": 50289418,
  "rank": 2,
  "hnUrl": "https://news.ycombinator.com/item?id=50289418",
  "source": "front page"
}

A comment collected with Include comments:

{
  "id": 40172273,
  "type": "comment",
  "author": "maximilianburke",
  "numComments": 3,
  "createdAt": "2024-04-26T17:55:51.000Z",
  "text": "I would love to be able to bypass the orphan rule for internal crates.",
  "parentId": 40172033,
  "storyId": 40172033,
  "storyTitle": "Leaving Rust gamedev after 3 years",
  "depth": 1,
  "hnUrl": "https://news.ycombinator.com/item?id=40172273",
  "source": "search: rust"
}

A job post from the Who is hiring thread (text shortened):

{
  "id": 49926618,
  "type": "comment",
  "author": "futohq",
  "createdAt": "2026-10-01T20:22:29.000Z",
  "text": "FUTO | https://futo.tech/ | Austin, TX | REMOTE(Worldwide) or ONSITE | Full Time\n\nFUTO is an organization dedicated to developing ...",
  "parentId": 49922569,
  "storyTitle": "Ask HN: Who is hiring? (October 2026)",
  "hiringHeadline": "FUTO | https://futo.tech/ | Austin, TX | REMOTE(Worldwide) or ONSITE | Full Time",
  "isRemote": true,
  "hnUrl": "https://news.ycombinator.com/item?id=49926618"
}

How much does it cost?

You pay per item saved: $0.0005 per story, comment or job post, plus $0.0005 once per run. 1,000 items cost $0.50 plus the start fee, so the front page (30 stories) costs about $0.02. Apify's free plan credit of $5 a month covers about 10,000 items.

Every comment is its own item, so a story with 500 comments costs $0.25 when you collect all of them. Inputs that fail (an unknown user, a missing item ID) cost nothing and are listed with the reason in the SUMMARY record of the run's key-value store.

Use cases

• Daily digest: schedule the front page or Best list with Minimum points set to 100 and send it to Slack, email or a spreadsheet.
• Brand and launch monitoring: search your product or company name newest first with From date set to 1 day.
• Job market research: pull the Who is hiring thread every month and filter by remote, a language or a city.
• Research and AI datasets: collect comment threads on a topic as plain text for sentiment analysis, summaries or RAG.

FAQ

Are points and comment counts live? In the live list modes and when a story's comments are fetched, yes: they come from the official Hacker News API at run time. In search and user modes they come from Algolia's index, which can trail the live site by a few minutes.

Why do comments have no points? Hacker News does not publish comment scores anywhere, so no tool can get them.

How far back can it search? Back to 2006, the first months of Hacker News. Relevance order stops at 1,000 results per search, a limit of Algolia's API. Newest-first order keeps going by moving the date window back, so it can return far more.

In what order are comments? Top-level comments follow Hacker News's own ranking. Replies under each comment are in the order they were posted, nested right under their parent. Deleted comments are left out.

What does Max results count? Stories for live lists and search, items per user in user mode, and job posts in Who is hiring mode. Comments mode uses Max comments per story. Comments added with Include comments do not count toward Max results, so set Max comments per story to control cost.

Does the Freelancer thread work? Yes for months up to October 2025, which is the last Freelancer thread the whoishiring account posted. The Who is hiring and Who wants to be hired threads work for every month.

Is this legal? It reads public data through the official Hacker News API and Algolia's public HN Search API. You are responsible for how you use it, especially personal data, under laws like GDPR.

Other Actors by this developer

• Google News Scraper: headlines by keyword, topic or country with the real publisher URL.
• Substack Scraper: posts, likes, comments and full text from any Substack newsletter.
• Telegram Channel Scraper: posts, views and reactions from public Telegram channels, no login.
• Career Site Jobs API: open jobs from Greenhouse, Lever, Ashby, Workable and Recruitee career sites.
• Website Tech Stack Detector: see which frameworks and tools a website runs on.