Hacker News Scraper: Stories, Comments, Search, Monitor

From The Mine Works, makers of Reddit Scraper and GitHub Trending Scraper, with over 140,000 runs across 170+ public actors.

Updated 21 Oct 2026: first release. Top, New, Best, Ask HN, Show HN and Jobs lists, keyword search with a date range and minimum points, a user's submissions, full comment threads with depth, and a monitor mode that returns only new items. Bookmark this Actor on its Store page so it is one click away next time.

Why choose this actor?

• Every list, every search, every thread. The six live lists in the order the site shows them, keyword search over stories or comments newest first back to 2007, a date range, minimum points and comments, a user's stories and comments, and any story or comment with the replies under it. One input field picks which.
• Comments as real rows, not a nested blob. With includeComments on, each story is followed by its comments in thread order, each with depth, parent_id, story_id, reply_count and its own link. You choose how many levels deep and how many per story. Filter on depth: 1 for direct replies or rebuild the tree from parent_id.
• Monitor mode for scheduled runs. Watch a keyword (your product, a competitor, a library) or the front page, and each run returns only the stories and comments no earlier run delivered. Items skipped this way are not charged.
• Cheap and fast, nothing to set up. $0.18 per 1,000 rows on every plan, read from Hacker News's own free public APIs. No account, no API key, no cookies.

Part of The Mine Works Social media and video family: Threads Scraper, Reddit Scraper, Threads Search Scraper, Instagram Profile Scraper, Instagram Followers & Following, Reddit Search Scraper.

Try it in one minute

Paste this into the input (JSON view) and press Start. It returns the first 50 stories of the Hacker News front page, one row each. It took 3 seconds in our test run:

{
  "mode": "top",
  "maxItems": 50
}

Switch mode to search and put words in keywords to search instead, or turn on includeComments to get the discussion under each story.

Apify's free plan includes $5 of credit every month, which covers about 27,000 rows at this actor's price.

Copy to your AI assistant

themineworks/hacker-news-scraper on Apify. Reads Hacker News through its free public APIs and returns one flat row per story and per comment, no login. Call ApifyClient("TOKEN").actor("themineworks/hacker-news-scraper").call(run_input={"mode": "search", "keywords": ["rust"], "dateFrom": "2026-10-01", "maxItems": 100}), then client.dataset(run["defaultDatasetId"]).list_items().items. mode is one of top, new, best, ask, show, jobs (live lists), search (needs keywords, or a date range or minPoints), user (needs usernames), item (needs itemIds, ids or news.ycombinator.com/item?id= links). Search options: searchIn (story, ask_hn, show_hn, comment, all; default story), sortBy (date or relevance; default date), dateFrom and dateTo (YYYY-MM-DD UTC or relative like "7 days"), minPoints, minComments. User option: userContent (stories, comments, all). maxItems (default 100, 1 to 10000) is per list, keyword or user. includeComments (default false; always on in item mode), maxCommentDepth (default 5), maxCommentsPerStory (default 100). monitorMode (default false) returns only items no earlier monitor run with the same mode and keywords, users or items delivered; then maxItems is the window checked. Rows have record_type "story" or "comment". Story rows: id, title, url, domain, points, author, comment_count, created_at, created_at_unix, type (story, ask, show, launch, job, poll), hn_url, text, text_html, rank, position, source, source_query, comments_fetched. Comment rows: id, parent_id, story_id, story_title, story_url, author, text, text_html, created_at, created_at_unix, depth, reply_count, thread_position, hn_url, position, source, source_query. Each run also writes one row with _type "summary", one with _type "info" when rows were delivered, and a row with _type "error" for each input that could not be read; none of these is charged. Full spec: GET https://api.apify.com/v2/acts/themineworks~hacker-news-scraper/builds/default (Bearer TOKEN), which returns inputSchema and readme. Token: https://console.apify.com/account/integrations?fpr=ymnoit&utm_source=apify-readme&utm_medium=referral

Key features

• Six live lists. top (the front page and beyond, 500 stories), new (500), best (200), ask, show and jobs, in the order Hacker News shows them, with rank on each row.
• Keyword search with dates. Search titles, links and story text, or comments. Newest first goes back as far as Hacker News does, past the usual 1,000 result limit of the search API: in our test one keyword returned 2,500 stories in 6 seconds. Most relevant first is also there, up to 1,000 per keyword. Typo matching is off, so rust does not also return posts about just.
• Filters that matter. dateFrom and dateTo as dates or as "7 days", minPoints and minComments, and searchIn to search only Ask HN, only Show HN or only comments.
• Users and threads. user mode returns a user's stories, comments or both, newest first. item mode takes any story or comment id or link and returns it with every reply under it, up to your limits; for a comment it also finds the story and the comment's depth.
• Readable text. text is plain text with paragraphs and links written out; text_html keeps the original Hacker News HTML.
• Stops cleanly. A list or search ends when it runs out, when maxItems is reached, or when two pages in a row bring nothing new. Eight failed requests in a row end the run, and the run stops before its time limit and keeps every row. The summary row says why each source stopped.

How to use it

Front page with the discussion

{
  "mode": "top",
  "maxItems": 30,
  "includeComments": true,
  "maxCommentDepth": 2,
  "maxCommentsPerStory": 50
}

30 stories, each followed by up to 50 comments, direct replies and the replies to them. In our test, best 200 stories with up to 50 comments each, 3 levels deep, gave 5,221 rows in 47 seconds.

Everything said about a topic last month

{
  "mode": "search",
  "keywords": ["rust", "postgres"],
  "dateFrom": "2026-09-01",
  "dateTo": "2026-09-30",
  "maxItems": 1500
}

Each keyword is its own search, labelled in source_query; a story both keywords find is delivered once. In our test this returned all 590 matching stories in 3 seconds. Set searchIn to comment to get what people wrote in the comments instead.

Popular posts only

{
  "mode": "search",
  "keywords": ["linux"],
  "dateFrom": "7 days",
  "minPoints": 50
}

Stories about Linux from the last 7 days with 50 points or more. Leave keywords empty with a date range and minPoints to list every popular story of a period.

A user's history

{
  "mode": "user",
  "usernames": ["pg", "https://news.ycombinator.com/user?id=dang"],
  "userContent": "all",
  "maxItems": 500
}

Stories and comments of each user, newest first. The summary row also gives each user's karma and when the account was made.

One thread, all of it

{
  "mode": "item",
  "itemIds": ["https://news.ycombinator.com/item?id=8863"],
  "maxCommentDepth": 100,
  "maxCommentsPerStory": 5000
}

The story row, then every comment in thread order. Rebuild the tree with parent_id, or sort by thread_position to read it as the page does.

Keyword monitor on a schedule

{
  "mode": "search",
  "keywords": ["your product name"],
  "searchIn": "all",
  "maxItems": 200,
  "monitorMode": true
}

Save it as a task and schedule it every hour. The first run returns the 200 newest matches; each later run checks the 200 newest again and returns only the ones not delivered before, stories and comments. Nothing new means nothing charged, not even the run fee.

Input parameters

Parameter	Type	Default	What it does
mode	string	top	top, new, best, ask, show, jobs, search, user or item
keywords	array of strings	empty	Search mode: one search per line, all words of a line must match
searchIn	string	story	Search mode: story, ask_hn, show_hn, comment or all
sortBy	string	date	Search mode: date (newest first, no depth limit) or relevance (up to 1,000 per keyword)
dateFrom, dateTo	string	empty	Search and user modes: YYYY-MM-DD in UTC, or relative such as 7 days. dateTo includes the whole day
minPoints, minComments	integer	empty	Search and user modes: only stories with at least this many points or comments
usernames	array of strings	empty	User mode: names or profile links, case sensitive
userContent	string	stories	User mode: stories, comments or all
itemIds	array of strings	empty	Item mode: story or comment ids or links
maxItems	integer	100	Most stories (or comments when you search comments) per list, keyword or user, 1 to 10,000
includeComments	boolean	false	Adds the comments under each story as rows. Always on in item mode
maxCommentDepth	integer	5	Levels of replies to read, 1 = direct replies only
maxCommentsPerStory	integer	100	Most comment rows under one story or item, 1 to 10,000
monitorMode	boolean	false	Only items no earlier monitor run with the same watch delivered

Up to 100 keywords, users or items per run.

What data do you get?

One row per story and one per comment, told apart by record_type. Data rows have no _type field; the summary, info and error rows do, so a filter on _type separates them.

Story rows (record_type: "story")

Field	What it is
id	Hacker News item id
title	Story title
url, domain	Link the story points to and its domain; text posts point to their own page
points	Points (score)
author	Username of the poster
comment_count	Comments on the story, all levels
created_at, created_at_unix	When it was posted, ISO 8601 UTC and unix seconds
type	story, ask, show, launch, job or poll
hn_url	Discussion page on news.ycombinator.com
text, text_html	Body of Ask HN, Show HN and text posts, as plain text and as HTML
rank	Place on the live list when it was read (list modes)
comments_fetched	Comment rows delivered under it (with includeComments)

Comment rows (record_type: "comment")

Field	What it is
id	Hacker News item id of the comment
parent_id	The story or comment it replies to
story_id, story_title	The story it belongs to (story_url too for search results)
author	Username of the commenter
text, text_html	The comment as plain text and as HTML
created_at, created_at_unix	When it was posted
depth	Level in the thread, 1 = direct reply to the story (empty for comments found by search)
reply_count	Direct replies to it
thread_position	Place in the thread as the page shows it
hn_url	Link to the comment

On every row: source (top, new, best, ask, show, jobs, search, user or item), source_query (the keyword, username or item id), position (place within that source, stories and searched comments) and scraped_at.

Summary row (_type: "summary", never charged): sources_requested, sources_read, sources_failed, pages_read, stories_found, stories_delivered, comments_delivered, items_delivered, charged_for, duplicates_skipped, known_from_earlier_runs, comments_dead_or_deleted, comment_threads_cut_at_cap, monitor, http_requests, http_megabytes, ended_on_deadline, stopped_by_max_cost, shortfall_reason when fewer rows came back than asked, and per_source, one entry per list, keyword, user or item with pages, found, delivered, total_matches for searches, karma for users, and why it stopped. The same summary is saved as the OUTPUT record of the run.

Error rows (_type: "error", never charged): one per input that could not be read, with error_type (invalid_item, invalid_username, invalid_date, not_found, rate_limited, fetch_error), identifier and reason.

Stable fields for automations

These were present in every story row of our test runs (4,074 stories across all modes), and their names will not change: record_type, id, title, url, author, created_at, created_at_unix, type, hn_url, source, position, scraped_at. Every comment row (5,601 in our tests) had record_type, story_title, id, parent_id, story_id, author, text, created_at, hn_url, source and scraped_at; comments read from a thread also had depth, reply_count and thread_position.

Optional, present only when Hacker News has a value: points and comment_count (job posts have no comments), text on stories (only text posts have a body), rank (list modes only), depth on comments found by search, and domain when a link cannot be read as a web address.

Output examples

The first story of the front page (test run, 21 Oct 2026):

{
  "record_type": "story",
  "id": 50170593,
  "title": "SimCity, but with Permits",
  "url": "https://permit.petercai.com/",
  "domain": "permit.petercai.com",
  "points": 366,
  "author": "pcai",
  "comment_count": 196,
  "created_at": "2026-10-20T18:29:09.000Z",
  "created_at_unix": 1792520949,
  "type": "story",
  "hn_url": "https://news.ycombinator.com/item?id=50170593",
  "rank": 1,
  "source": "top",
  "source_query": "top",
  "position": 1,
  "scraped_at": "2026-10-20T22:15:36.690Z"
}

A reply two levels deep in the Dropbox launch thread, from item mode (same day, text trimmed):

{
  "record_type": "comment",
  "id": 9272,
  "parent_id": 9224,
  "story_id": 8863,
  "story_title": "My YC app: Dropbox - Throw away your USB drive",
  "author": "dhouston",
  "text": "1. re: the first part, many people want something plug and play. and even if they were plug and play, the problem is that the user experience (on windows at least) with online drives generally sucks...",
  "created_at": "2007-04-05T16:47:01.000Z",
  "created_at_unix": 1175791621,
  "depth": 2,
  "reply_count": 1,
  "thread_position": 2,
  "hn_url": "https://news.ycombinator.com/item?id=9272",
  "source": "item",
  "source_query": "8863"
}

The summary row of a relevance search for "claude code" with 5 comments per story, trimmed:

{
  "_type": "summary",
  "mode": "search",
  "sources_read": 1,
  "stories_delivered": 30,
  "comments_delivered": 150,
  "items_delivered": 180,
  "comment_threads_cut_at_cap": 30,
  "http_requests": 241,
  "run_seconds": 7,
  "per_source": [
    { "source": "claude code", "pages": 1, "found": 50, "delivered": 30, "rows_delivered": 180, "stopped": "max_items_reached", "total_matches": 347 }
  ]
}

An error row for an id that does not exist (nothing charged):

{ "_type": "error", "error_type": "not_found", "identifier": "9999999999", "reason": "Hacker News has no such item.", "http_status": 404 }

Pricing

Pay per event: you pay for the rows you receive, plus a flat $0.003 per run, whatever memory you choose. Story rows and comment rows cost the same.

Event	When it is charged	Free plan	Bronze (Starter)	Silver (Scale)	Gold (Business)
item-scraped	Once per story or comment row stored in your dataset	$0.18 per 1,000	$0.18 per 1,000	$0.18 per 1,000	$0.18 per 1,000
run-start	Once per run, after the first page returned rows	$0.003	$0.003	$0.003	$0.003

The price of one row is the price per 1,000 divided by 1,000: all 500 top stories with 50 comments each, about 25,500 rows, cost about $4.59.

Never charged:

• inputs that are not a Hacker News id, link, username or date (each gets an error row);
• a story or comment that a second keyword of the same run finds again;
• items an earlier monitor run already delivered;
• deleted and flagged comments (no row is written for them);
• rows past your maxItems or maxCommentsPerStory;
• the summary, info and error rows;
• a run with no valid input, or one that finds nothing: the start fee is charged only once a page has returned rows;
• anything past your "Max cost per run". The actor checks the budget before each batch of rows and stops while the total is still below your limit, so the run ends SUCCEEDED with a partial result and a status message saying so.

Run it on a schedule

1. Open the actor in Apify Console, set your input, turn on monitorMode and click Save as a new task.
2. In the task, open Schedules, click Add schedule and pick how often (every hour, every morning at 08:00 and so on).
3. Set maxItems above the number of new items you expect between two runs; each run checks that many of the newest items and returns the ones not delivered before.
4. Optional: add an integration (Slack, Google Sheets, a webhook) so each new mention lands where your team works.

FAQ

What does this actor read?

Hacker News's two free public APIs: the official Firebase API (hacker-news.firebaseio.com) for the live lists, items, users and comment threads, and the Algolia HN Search API (hn.algolia.com) for keyword, date and user searches. It does not log in, vote, post or read anything that needs an account.

How far back can a search go?

To the first Hacker News posts of October 2006. With newest first (the default), the actor pages backwards through time, so one keyword can return up to 10,000 stories per run and more over several runs with dateTo. Most relevant first stops at 1,000 per keyword, a limit of the search API.

Why does a story say 196 comments and I got fewer?

comment_count counts every comment on the story. You get at most maxCommentsPerStory rows, only maxCommentDepth levels deep, and no rows for deleted or flagged comments. Raise both limits to get the whole thread; when a thread is cut, the higher levels are kept first and the summary counts it in comment_threads_cut_at_cap.

How does monitor mode decide what is new?

It remembers the id of every story and comment it delivered, per watch: the same mode with the same keywords, users or items and the same searchIn or userContent. Two tasks with different keywords keep separate memories. Each run checks the first maxItems items of each list or search and returns those not delivered before. Changing maxItems, dates or comment settings keeps the memory; changing the keywords starts a new one.

Are the points and comment counts live?

List and item modes read them straight from Hacker News at run time. Search and user modes read them from the Algolia index, which can lag the site by a few minutes for very new posts.

How long does a run take?

In our tests: 3 seconds for the "Try it" input (50 front page stories), 8 seconds for all 500 top stories, 6 seconds for 2,500 search results, and 47 seconds for 200 best stories with 5,021 comments. The default time limit is 3,600 seconds; the actor stops before it and keeps every row.

Do I need an account, an API key or a proxy?

No. Both APIs are free and keyless. If the search API ever answers "too many requests", the actor moves its search requests to Apify Proxy on its own, at no cost to you.

Which formats can I download?

JSON, CSV, Excel, XML, RSS and HTML from the run's Storage tab or through the API. The dataset has three ready views: Stories, Comments and Links.

Can I use it from Claude, ChatGPT or Cursor (MCP)?

Yes, through Apify's MCP server. Connector URL: https://mcp.apify.com/?tools=themineworks/hacker-news-scraper

• Claude: Settings > Connectors > Add custom connector, paste the URL, sign in with Apify.
• ChatGPT: turn on developer mode, add an MCP connector with the URL, sign in with Apify.
• Cursor or VS Code: add it as an HTTP MCP server with that URL.
• Claude Code: claude mcp add -t http hacker-news-scraper "https://mcp.apify.com/?tools=themineworks/hacker-news-scraper"

Is it legal to scrape Hacker News?

The actor reads only public posts through the APIs Hacker News and Algolia publish for this purpose. Comments are written by people and carry usernames; you are responsible for how you use the data, including data protection law such as GDPR and CCPA. This is general information, not legal advice.

Integrations

• Slack and email: schedule a monitor task and send each run's new rows to a channel with Apify's Slack integration or a webhook, for example "every new mention of our product on Hacker News".
• Google Sheets: send stories to a sheet; one row per story fits a sheet as is.
• Make, Zapier and n8n: start a run and read its dataset with the Apify apps and nodes.
• API and SDKs: run it from Python or JavaScript with the Apify client, or with a plain HTTP call (see "Copy to your AI assistant" above).
• MCP clients: Claude, ChatGPT, Cursor and VS Code can run it as a tool through the connector URL in the FAQ.

More from The Mine Works

Social media and video

• Threads Scraper
• Reddit Scraper
• Threads Search Scraper
• Instagram Profile Scraper
• Instagram Followers & Following
• Reddit Search Scraper
• Twitter / X Scraper
• YouTube Transcript
• Xiaohongshu (RED) Scraper
• Telegram Channel Scraper
• Telegram Channel Finder
• Pinterest Profile Scraper

Leads and business directories

• B2B Leads Finder
• Skip Trace Lookup
• Google Maps Email Scraper
• 2GIS Places Scraper

Marketing, SEO and reviews

• Facebook Ad Library Scraper
• Google Ads Transparency Scraper
• Similarweb Scraper
• Google News Scraper

LinkedIn

• LinkedIn Company Scraper
• LinkedIn Post Scraper
• LinkedIn Employees Scraper
• LinkedIn Profile Scraper

Real estate

• Zillow Rentals Scraper
• Zillow Sold Comps Scraper
• India Real Estate MCP
• Housing.com Scraper

Science, health and government data

• Academic Research MCP
• CourtListener Scraper
• Socrata Open Data Scraper
• OpenAlex Scraper

Jobs and hiring

• Foundit Monster India Jobs
• Hirist Jobs Scraper
• India Jobs MCP
• Naukri Jobs Scraper

Company and business data

• Company Domain Finder
• GST Taxpayer Lookup
• Company KYB Resolver
• SEC EDGAR Filings Scraper

E-commerce and marketplaces

• Ozon.ru Scraper
• ⭐ Amazon Reviews Scraper
• Amazon Product Scraper
• Etsy Listings Scraper

Food and local services

• NoBroker Scraper
• Swiggy Restaurant Scraper
• Zomato Scraper

Developer and AI tools

• Website to Markdown Crawler
• GitHub Skill Finder
• GitHub Repo Scraper
• GitHub Trending Scraper

More tools

• G2 Reviews Scraper
• Naver Blog Scraper
• WeChat Article Scraper 微信公众号文章
• Taobao Products Scraper 淘宝 天猫

Support

Found a story or thread that comes back with fewer rows than the site shows, or a field that is empty? Open an issue in the Issues tab with the input and the run ID, and we will look at it. For a field you need or a new source, email dmineworks@gmail.com.

Get Hacker News stories and comments as clean rows: the front page and every live list, keyword search with dates and minimum points, users and full threads, and only new items on scheduled runs.