Hacker News Scraper (Stories, Comments & Search)

Collect Hacker News stories, items, nested comments, user profiles and search results using only official/public APIs: the Firebase Hacker News API and the Algolia Hacker News Search API. This Actor does not scrape HTML pages and is not affiliated with Y Combinator.

Use cases

• Track top, new, best, Ask HN, Show HN and job stories.
• Search Hacker News by query, tag, author, date range and minimum points.
• Export selected item IDs with normalized timestamps, domains and plain-text story/comment text.
• Enrich stories with nested comments for research, moderation, sentiment analysis or LLM workflows.
• Collect public HN user profile metadata such as karma and submitted count.

Input

Field	Default	Description
mode	lists	lists, search, items, or user.
lists	["top"]	Lists to read: top, new, best, ask, show, job.
searchQuery	–	Query for Algolia search mode.
searchSortByDate	false	Use /search_by_date instead of /search.
searchTags	–	Algolia tags such as story, comment, ask_hn, show_hn, poll, author_pg.
minPoints	–	Adds points>N numeric filter in search mode.
dateFrom, dateTo	–	ISO dates converted to Algolia created_at_i numeric filters.
itemIds	–	Item IDs for items mode.
usernames	–	Usernames for user mode.
maxItems	100	Maximum dataset items to output (1–1000).
includeComments	false	Fetch comments and include them in each story item.
maxCommentsPerStory	50	Maximum comments fetched per story.
maxCommentDepth	100	Nested comment depth. 0 means top-level comments only.

{
  "mode": "lists",
  "lists": ["top", "show"],
  "maxItems": 10,
  "includeComments": true,
  "maxCommentsPerStory": 5
}

Output

One dataset item is produced per story, item, search hit or user profile. Deleted and dead Firebase items are skipped. Comments are nested in comments[] when requested.

{
  "id": 49977979,
  "type": "story",
  "title": "Mistral Large 4",
  "url": "https://docs.mistral.ai/models/mistral-large-4-0",
  "domain": "docs.mistral.ai",
  "by": "Philpax",
  "score": 822,
  "descendants": 481,
  "createdAt": "2026-10-06T13:15:49.000Z",
  "hnUrl": "https://news.ycombinator.com/item?id=49977979",
  "rank": 1,
  "list": "top",
  "source": "firebase",
  "comments": [
    {
      "id": 49978211,
      "by": "eigenspace",
      "text": "Its quite interesting to see that at least the early days of AI so far have not been a winner-take-all runaway acceleration game where catchup is impossible...",
      "createdAt": "2026-10-06T13:33:08.000Z",
      "parentId": 49977979,
      "depth": 0
    }
  ],
  "commentCount": 1,
  "scrapedAt": "2026-10-06T16:30:13.700Z"
}

Pricing

Pay per event: you only pay for results that are stored.

Event	Title	Price	When charged
item-scraped	Item	$0.0005 ($0.50 per 1,000 items)	Each story, item, search hit or user profile pushed to the dataset.
comment-scraped	Comment	$0.0001 ($0.10 per 1,000 comments)	Each fetched comment when includeComments=true.

Comments are included inside the story dataset item. They are not pushed as separate dataset rows; the optional comment-scraped event only accounts for the extra API work of fetching comments. The Actor respects your maximum cost per run and stops cleanly when the platform budget is reached.

FAQ

Does this scrape Hacker News HTML? No. It uses only https://hacker-news.firebaseio.com/v0/ and https://hn.algolia.com/api/v1/.

Are deleted or dead items returned? No. Firebase items marked deleted or dead are skipped.

Can I search by author or tag? Yes. Use Algolia tags such as author_pg, story, comment, ask_hn, show_hn, or poll.

Are comments charged separately? Story/user/search items are charged as item-scraped. When comments are enabled, fetched comments are charged as comment-scraped, but they remain embedded in the story item.

Changelog

• 0.1: Initial release. Lists, search, item IDs, user profiles, nested comments, budget-aware PPE charging.