Hacker News Stories, Comments & Users Scraper

Extract stories, comments, and user data from Hacker News using the official Algolia HN Search API and Hacker News Firebase API. No authentication or proxy required — 100% reliable.

Features

• Search stories by keyword with relevance or date sorting
• Search comments by keyword across all HN discussions
• Top / New / Best stories — real-time front page and trending feeds
• User submissions — all stories and comments by any HN user
• Item lookup — fetch any item by ID or URL
• Filter by date range, minimum points, and story type (Ask HN, Show HN, Job)

Input

Field	Type	Description
mode	select	What to scrape (see modes below)
searchQuery	string	Keyword to search (modes: searchStories, searchComments)
username	string	HN username (mode: byUser)
startUrls	array	Item IDs or URLs (mode: getItem)
storyType	select	story, ask_hn, show_hn, job, all
sortBy	select	relevance or date
dateFrom	string	Filter from date (YYYY-MM-DD)
dateTo	string	Filter to date (YYYY-MM-DD)
minPoints	integer	Minimum points filter
maxItems	integer	Maximum records to return (default 50)

Modes

Mode	Description
searchStories	Full-text search across all HN stories
searchComments	Full-text search across all HN comments
topStories	Current front page top stories
newStories	Latest submitted stories
bestStories	All-time best stories
byUser	All submissions (stories + comments) by a username
getItem	Fetch specific items by ID or URL

Output

Story record

{
  "storyId": 39876543,
  "type": "story",
  "title": "Show HN: I built a faster way to index large codebases",
  "url": "https://example.com/faster-indexing",
  "hnUrl": "https://news.ycombinator.com/item?id=39876543",
  "author": "johndoe",
  "points": 312,
  "commentsCount": 87,
  "createdAt": "2024-03-15T14:22:00+00:00",
  "recordType": "story",
  "scrapedAt": "2024-03-15T15:00:00+00:00"
}

Comment record

{
  "commentId": 39876999,
  "type": "comment",
  "text": "This is really impressive. How does it handle...",
  "hnUrl": "https://news.ycombinator.com/item?id=39876999",
  "author": "janedoe",
  "points": 42,
  "storyId": 39876543,
  "storyTitle": "Show HN: I built a faster way to index large codebases",
  "storyUrl": "https://example.com/faster-indexing",
  "storyHnUrl": "https://news.ycombinator.com/item?id=39876543",
  "createdAt": "2024-03-15T14:45:00+00:00",
  "recordType": "comment",
  "scrapedAt": "2024-03-15T15:00:00+00:00"
}

Use Cases

• Market research — track mentions of products, technologies, or companies
• Trend analysis — monitor top stories over time by category
• Competitor intelligence — search for mentions of competitors or alternatives
• Content research — find highly-upvoted discussions on specific topics
• Lead generation — identify users active in your target domain

FAQ

Does this require authentication?
No. Both the Algolia HN API and Firebase API are fully public. No API key needed.

Is there a rate limit?
The Algolia API allows generous usage for non-commercial use. This actor respects rate limits with automatic retries.

How far back does the search history go?
The Algolia HN Search API indexes all HN content since the site's founding in 2006.

Can I search for Ask HN or Show HN posts specifically?
Yes — set storyType to ask_hn or show_hn in searchStories mode.

How many results can I fetch?
Up to 5,000 records per run. The Algolia API paginates results automatically.