Hacker News Scraper

This Hacker News scraper searches stories, Show HN launches, Launch HN posts, comments and monthly "Who is hiring" threads. Export consistent rows with author, points, dates, links and domains for launch tracking, developer research and brand monitoring.

What the Hacker News scraper collects

Use the actor to monitor product launches, find developer discussions, track mentions of a company domain, or collect comments from a specific thread. It reads public data from the official HN Search service provided by Algolia and the official Hacker News item API. No source account or source API key is required.

Each delivered item becomes one dataset row. Stories, jobs, polls and poll options use the post row type. Replies use the comment row type. The output keeps the original Hacker News item type as well, so you can distinguish a story from a job without inspecting its title. Deleted and missing items do not become billable rows. Explicitly requested flagged items retain a dead flag and any data the source still exposes.

Use cases for Hacker News data

• VC and startup scouting. Collect every Launch HN and Show HN post for a period, sort by points and comment count, and join the external domain to company, repository or product launch datasets to see which launches drew attention.
• Devtool marketing and brand monitoring. Search a product name in stories and comments, or run domain mode on your own website, to find every discussion that linked to you. Schedule the actor to catch new mentions every day.
• Competitive research. Run the same domain search for competitors and compare how often their posts reached a high score, which titles worked and what commenters asked.
• Recruiting and job market research. Export the monthly Who is hiring, Who wants to be hired or Freelancer thread as one row per listing, with the original text and HTML for your own parsing.
• Trend and sentiment analysis. Pull all stories or comments that mention a technology for a date range and feed the plain text into your own analysis or language model pipeline.

Input

Provide at least one selector: a search query, domain, author, tag list, item ID list, feed, story ID or hiring month. For a text search, the actor defaults to story results sorted by relevance. Other searches default to newest first. You can explicitly select relevance or date sorting.

These inputs were used to retrieve the examples included with this product:

{"query":"rust","min_points":100,"max_results":5}

{"tags":["launch_hn"],"max_results":3}

{"hiring_month":"2026-10","max_results":5}

The available tags are story, comment, poll, job, show_hn, ask_hn, launch_hn and front_page. Multiple tags are combined with OR. An author filter is combined with the selected tags using AND. Show HN and Launch HN use their own source tags; searching for their title prefixes is unnecessary.

Use date_from and date_to as YYYY-MM-DD dates. Both endpoints of the range are inclusive UTC calendar days. The min_points filter requires a minimum story score. The min_comments filter requires a minimum story comment count. Hacker News does not publish comment scores, so these score and count filters cannot be used for a comment-only request.

A domain input accepts either a hostname or an HTTP URL. The actor removes the path and a leading www, then matches the exact hostname or its subdomains. The domain belongs to the external story URL, which makes it useful for joining launch discussions to company or repository datasets. Domain and query are alternative search selectors and cannot be combined.

For direct lookups, item_ids accepts numeric IDs, decimal strings or Hacker News item URLs. Duplicate IDs are removed and the original request order is preserved. Available feeds are top, new, best, ask, show, job and front_page. Feeds preserve their source order. Item lists, feeds, story IDs and hiring months are alternative modes: choose one. Text and domain searches cannot be combined with an item list or feed, while author, tag, date and score filters remain available.

Comments and hiring threads

Set story_id to retrieve comments belonging to one discussion. The post_id on every comment identifies the original story, including replies several levels down. Set top_level_only to retain comments directly on the story.

For monthly threads, set hiring_month to YYYY-MM. hiring_kind chooses hiring, wants_to_be_hired or freelancer; the default is hiring. The actor locates the thread posted by whoishiring and then collects its comments. Hiring requests default to top-level comments because replies usually discuss a listing rather than introduce another listing. Set top_level_only to false to include those replies. A query or author can further filter the chosen discussion. Hiring output includes the selected thread's ID, title, link, publication time and comment count alongside the item rows.

Output

Rows include id, type, kind, _type, source, platform, fetched_at, post_id, post_url, hn_url, author_handle, text, text_html and posted_at. Stories add title, external url, domain, likes and comments_count. The likes field represents Hacker News points; comments_count represents the source's story comment count. Comments add comment_id, comment_url, parent_id and parent_comment_id. A top-level comment has a null parent_comment_id. A comment score stays null because it is not public.

The post_url is the Hacker News discussion page. The url is the external page linked by a story. Keeping these links separate lets you open the discussion while joining the external domain to other datasets. Original HTML remains available in text_html; text contains readable plain text. Posts combine their title and body with a blank line.

The dataset uses normalized names: author becomes author_handle, points becomes likes, num_comments becomes comments_count and created_at becomes posted_at. Publication and retrieval times are ISO 8601 UTC timestamps. The four is_show_hn, is_ask_hn, is_launch_hn and is_front_page flags describe special post categories. Source-only comment trees and search highlights are omitted.

Example output row

A real row from the search example above (query rust, at least 100 points), as delivered to the dataset:

{
  "source": "hacker-news",
  "platform": "hacker_news",
  "fetched_at": "2026-10-05T02:04:26.350Z",
  "kind": "post",
  "_type": "post",
  "id": "22238335",
  "type": "story",
  "post_id": "22238335",
  "post_url": "https://news.ycombinator.com/item?id=22238335",
  "hn_url": "https://news.ycombinator.com/item?id=22238335",
  "author_handle": "Sikul",
  "title": "Why Discord is switching from Go to Rust",
  "url": "https://blog.discordapp.com/why-discord-is-switching-from-go-to-rust-a190bbca2b1f",
  "domain": "blog.discordapp.com",
  "text": "Why Discord is switching from Go to Rust",
  "text_html": null,
  "posted_at": "2020-02-04T17:30:40.000Z",
  "likes": 1582,
  "comments_count": 642,
  "parent_id": null,
  "story_title": null,
  "story_url": null,
  "is_show_hn": false,
  "is_ask_hn": false,
  "is_launch_hn": false,
  "is_front_page": false,
  "dead": false
}

Limits and continuation

max_results defaults to 100 and accepts values from 1 to 10000. Large requests are collected in continuation steps. The actor follows those steps automatically and removes duplicate item IDs before delivery. API consumers can use the returned cursor with the same selectors and filters to fetch the remaining data.

Relevance searches have a source limit of 1000 hits for one query. Use date sorting for larger collections. Timestamp continuation keeps items posted in the same second and avoids returning the same item twice. If the source cannot advance beyond its timestamp boundary, the output reports the limitation instead of silently skipping that second. Domain and top-level comment filters may produce fewer rows than the source hit estimate. The nb_hits value is an estimate before those local filters, not a guaranteed number of deliverable rows.

Warnings report missing IDs, source limits and unfinished collection. Comments whose story cannot be resolved are skipped with a warning, and later item IDs remain available. A partial result preserves rows already retrieved. An empty search returns no dataset rows. Counts and scores from Algolia may lag the item API by approximately one or two minutes, so these values describe a retrieval snapshot rather than a permanent total.

A Hacker News API alternative without writing code

The official Hacker News API returns one item per request and stores comments as a tree of item IDs, while the Algolia HN Search API returns search hits in pages with its own field names. Using either directly means writing pagination, ID fan-out, retries and field mapping yourself. This actor combines both official sources behind one input and returns one flat row per item, so you can export to JSON, CSV or Excel, or call the actor from the Apify API, without maintaining that code.

	This actor	Official item API	Algolia HN Search API	Manual copy from the site
Full-text and domain search	Yes	No	Yes	Site search only
Comments with parent and story IDs	Yes, one row per comment	Item by item	Search hits only	No
Monthly hiring threads by month	Yes	Manual lookup	Manual lookup	Manual
Continuation beyond 1000 hits	Automatic, by date	Not applicable	Manual paging	No
Uniform field names across sources	Yes	No	No	No
Export to CSV, Excel, JSON	Yes	Own code	Own code	No

Other Hacker News actors in the Store usually cover one part of this: front page stories, a keyword search or the hiring thread alone. This actor covers search, Show HN and Launch HN, comments, feeds, item lookups and hiring threads with the same row format, which makes the output easy to combine in one dataset.

Pricing

Billing is based on delivered dataset rows through the result event. Missing and deleted items, empty searches and input errors produce no result charges. There is no separate run-start charge for this product. The current price displayed on the actor's Pricing tab is the authoritative price when you start a run.

Non-paying users receive up to ten rows per run. max_results is capped accordingly. For both free and paid users, only rows actually delivered to the dataset count toward the result event. Requesting a larger limit does not create charges for rows that were not found or could not be collected.

FAQ

Do I need a Hacker News account or an API key?

No. Both source services expose public data without requiring your own account or key. The actor collects public items and discussion text.

Can I search for every mention of a company domain?

Domain mode searches external story URLs and checks their hostnames. For mentions inside comment text, use a query together with the comment tag. These are separate searches because a linked domain and a textual mention are different signals.

Why are comment points null?

Hacker News does not expose comment scores in its public APIs. The actor preserves that absence instead of estimating or inventing a score.

Can I collect more than 1000 items?

Yes, with date sorting and max_results up to 10000, subject to the available source data. Relevance sorting remains limited to the first 1000 source hits for a query.

Are hiring comments parsed into companies, locations and remote flags?

The actor delivers the original listing text and its parent relationships. It does not infer company names, locations or employment terms. You can apply your own parser to the plain text or original HTML after export.

Why can the same story have slightly different scores between runs?

Votes and replies change continuously, and the search index updates after the item API. Each row includes its retrieval timestamp so you can compare snapshots and choose which one to retain.

How is this different from the official Hacker News API?

The actor uses the official sources and does not scrape the website pages. It adds search filters, automatic continuation, comment collection for a whole thread, hiring thread lookup by month and one consistent row format. If you only need a single item now and then, the official API is enough.

Can I export Hacker News data to CSV or Excel?

Yes. Every run stores its rows in an Apify dataset, which you can download as JSON, CSV, Excel, XML or HTML, or read through the Apify API and integrations.