Hacker News Search Scraper

Hacker News Search Scraper runs full-text searches over Hacker News and returns one row per matching story, Ask HN, Show HN, job or poll: title, link and its domain, the text of text posts, the submitter's username, points, comment count, date and the story's place in the results. Type a term such as web scraping and click Start. Filters cover dates, minimum points and minimum comments, and each story's comment thread can come along. One search term returns at most 1,000 stories, so long periods need splitting by date. It costs $0.50 per 1,000 stories on the Free plan ($0.25 on Business), and the $5 of monthly credit in Apify's Free plan covers about 10,000 stories.

What data does it return?

Field	Description
id, hnUrl	Hacker News item ID and its discussion page, such as https://news.ycombinator.com/item?id=31075396.
title	Title as submitted, with prefixes such as Ask HN: or Show HN: kept.
kind	story for a link, ask for Ask HN and Tell HN, show for Show HN, job or poll.
url, domain	The page the story links to and its host without www., such as techcrunch.com. Both are null for text posts.
text	Body of a text post (Ask HN, a Show HN without a link, a job ad) as plain text, null for link stories. 9 of the 100 stories for web scraping had one.
author	Hacker News username of the submitter.
points, commentsCount	Score and number of comments when the search ran.
createdAt	When the story was posted, ISO 8601 in UTC.
rank, matchedQuery	Position in the results of the search term, 1 first, and the term itself. matchedQuery is null for a search with filters only.
type, source, key	story, search, and the key story:<id> that new-only mode compares between runs.

With Include comments on, each story is followed by its comments as rows of type comment, with storyId, storyTitle, parentId (null for a top-level comment), depth (0 at the top), author, createdAt, text and hnUrl. The story row then also has commentsRead, the number of its comments returned, while commentsCount stays the total of the thread.

How to search Hacker News

1. In Search terms, add one term per line, such as postgres or web scraping. The search looks at titles, links and the text of text posts. An empty term together with a filter returns everything that matches the filter, for example every Show HN since a date, always newest first.
2. Under What to search, pick All stories, Ask HN, Show HN, Jobs, Polls or Front page stories, which are stories that reached the front page.
3. Choose the Order: Relevance ranks by match and popularity, Newest first by date. Narrow the results with Published after and Published before (YYYY-MM-DD), Minimum points and Minimum comments.
4. Set Maximum results per search term (100 by default). Keywords (any of) keeps only stories whose title, text or link contains one of the words.
5. Turn on Include comments for the threads, capped by Maximum comments per story (100 by default), and click Start.
6. The rows appear in the Output tab, ready to export as CSV, Excel or JSON. The log gives the number of matches Hacker News holds for each term.

From the API, this input asks for well-scored stories about two databases since the start of 2026, newest first:

{
  "queries": ["postgres", "sqlite"],
  "searchTags": "story",
  "sortBy": "date",
  "publishedAfter": "2026-01-01",
  "minPoints": 50,
  "maxResultsPerQuery": 500
}

Example output

The second result for web scraping by relevance in a run on 11 October 2026. The submitter's username is replaced with hn_user; everything else is as returned.

{
  "type": "story",
  "key": "story:31075396",
  "id": 31075396,
  "kind": "story",
  "title": "Web scraping is legal, US appeals court reaffirms",
  "url": "https://techcrunch.com/2022/04/18/web-scraping-legal-court/",
  "domain": "techcrunch.com",
  "text": null,
  "author": "hn_user",
  "points": 1040,
  "commentsCount": 237,
  "createdAt": "2022-04-18T19:37:55.000Z",
  "hnUrl": "https://news.ycombinator.com/item?id=31075396",
  "source": "search",
  "rank": 2,
  "matchedQuery": "web scraping",
  "commentsRead": null
}

How much does it cost to search Hacker News?

	Free plan	Starter	Scale	Business
1,000 stories	$0.50	$0.40	$0.325	$0.25
Comment, per 1,000	$0.20	$0.16	$0.13	$0.10

You pay only for the events in the table. There is no start fee, and compute time and proxies are included.

The first 1,000 stories for a term cost $0.50 on the Free plan and $0.25 on Business. A search for Show HN posts about web scraping since 2025 with at least 10 points found 37 stories, and with up to 50 comments each it returned 608 comments: $0.0185 plus $0.1216 on the Free plan. Stories that Keywords (any of) removes and rows a watch has already returned in new-only mode are not charged.

Limits

• Hacker News search returns at most 1,000 results per query. On 11 October 2026 llm had 53,946 matching stories; a run with a higher maximum returned the first 1,000 by relevance, posted between January 2023 and October 2026. To get more, run the term with several Published after and Published before ranges or a higher Minimum points.
• The search does not cover comments on their own: comments arrive only as the threads of the stories found.
• Points and comment counts change while a story is on the front page. Each row holds the values from the moment of the run.
• Deleted comments are left out. Replies to a deleted comment keep its ID in parentId, so a few threads have a parent that is not in the dataset.
• Text arrives as plain text: paragraphs are kept, while links and formatting are reduced to their visible text.
• A run is fast: 1,000 stories took 7 seconds, and 37 stories with 608 comments 9 seconds, with 4 threads read at a time.

Keyword alerts

Set Order to Newest first, turn on Return only rows new since the last run, give a Watch name or save the input as a task, and schedule it, for example every hour. Each run returns only the stories that no earlier run of the watch returned, and only those are charged. With Relevance order an older story that rises into the top results also counts as new, so alerts work best by date. Stories that Maximum rows or your spending limit cut off are not recorded as seen, so the next run returns them. Every run with a watch also writes a CHANGES record with the stories that appeared, the ones that dropped out of the results and the stories whose points changed.

FAQ

Is it legal to scrape Hacker News?

The actor reads the public Hacker News search API at hn.algolia.com, the search linked from news.ycombinator.com, without logging in. Stories, comments and usernames are posted publicly; the actor does not collect user profiles, karma, email addresses or anything behind an account. You are responsible for how you use the data.

How do I get every story about a topic?

Split the period. For a busy term, run it once per year or per month with Published after and Published before, so that each range holds fewer than 1,000 stories. Rows from several runs merge by id.

Can I get whole comment threads for stories I already have?

Yes. Hacker News Comments Scraper takes story links or IDs and returns their full threads, up to 1,000 comments per story by default.

Related scrapers

• Substack Scraper

Support

This actor is built and maintained by deriverge s.r.o., a software company based in the Czech Republic. If a run fails or a field you need is missing, please open an issue in the Issues tab or write to us at info@deriverge.com. We respond in English and Czech. Runs can be scheduled in Apify Console or started from the API tab, which has examples for Python, JavaScript and cURL and works with Make, Zapier, n8n and the Apify MCP server. If the actor saves you time, a short review helps other people find it.