Hacker News Scraper: Stories, Comments, Users & Search

What does Hacker News Scraper do?

Hacker News Scraper extracts stories, comments, user profiles, and full-text search results from Hacker News — the legendary tech community run by Y Combinator. It uses the official Firebase API (no rate limits) and the Algolia search API (full archive search) to pull clean, structured JSON data. No API key required. No browser. No proxy needed.

Use it to monitor trending discussions, research historical topics, track user activity, analyze YC startup trends, or build datasets for LLM fine-tuning and sentiment analysis.

Why use Hacker News Scraper?

• Research: Find every HN discussion about a technology, company, or topic across all time
• Monitoring: Track daily top/new/best stories and alert on keywords
• Data science: Build training datasets from high-quality technical discussion
• Competitive intel: Monitor what the dev community says about your product or competitors
• Hiring & HR: Find Ask HN job threads and talent signals
• Podcast & newsletter: Auto-curate top HN content for weekly digests

How to use Hacker News Scraper

1. Go to the Input tab and select a Scrape Mode
2. For Top/New/Best/Ask/Show/Jobs: set Max Items and optional Minimum Score filter
3. For Keyword Search: enter a query, choose sort (relevance or date), and optionally restrict to stories or comments
4. For Thread: paste a story ID or HN URL to get the story + all comments
5. For User: enter a username to get their profile and submissions
6. Click Start and find your results in the Output tab as clean JSON

Input

Field	Type	Description
scrapeMode	enum	topStories, newStories, bestStories, askStories, showStories, jobStories, search, thread, user
searchQuery	string	Keywords to search (Algolia full-text). Used with search mode
searchSortBy	enum	relevance or date (newest first)
searchType	enum	story, comment, or all
storyId	string	Story ID or HN URL for thread mode
username	string	HN username for user mode
maxItems	integer	Max items to return (0 = no limit, default: 100)
includeComments	boolean	Fetch full comment trees for each story in feed/search modes
maxCommentsPerStory	integer	Max comments per story (default: 50)
minScore	integer	Filter stories below this score (default: 0 = no filter)

Output

Each scraped item is a JSON object. You can download the dataset in JSON, CSV, HTML, or Excel format.

Story example

{
  "type": "story",
  "id": 40123456,
  "title": "Show HN: I built a tool that does X",
  "by": "username",
  "score": 342,
  "descendants": 87,
  "url": "https://example.com/article",
  "text": null,
  "createdAt": "2026-06-01T14:30:00+00:00",
  "hnUrl": "https://news.ycombinator.com/item?id=40123456",
  "commentIds": [40123457, 40123458]
}

Comment example

{
  "type": "comment",
  "id": 40123457,
  "parentId": 40123456,
  "storyId": 40123456,
  "by": "commenter",
  "text": "<p>This is really interesting because...</p>",
  "depth": 0,
  "createdAt": "2026-06-01T14:45:00+00:00",
  "hnUrl": "https://news.ycombinator.com/item?id=40123457"
}

User example

{
  "type": "user",
  "id": "pg",
  "karma": 184923,
  "about": "<p>Co-founder of Y Combinator.</p>",
  "createdAt": "2006-10-09T18:00:00+00:00",
  "submittedCount": 1245,
  "hnUrl": "https://news.ycombinator.com/user?id=pg"
}

Data fields

Field	Description
type	story, comment, user, or job
id	HN item/user ID
title	Story headline
by	Author username
score	Story points (upvotes)
descendants	Total comment count
url	External article URL
text	Self-post body or comment text (HTML)
createdAt	ISO 8601 timestamp
hnUrl	Direct HN link
commentIds	IDs of top-level replies
depth	Comment nesting depth
karma	User karma score (user items only)

Pricing

This actor uses the Pay Per Event model. You are charged per item scraped:

• ~500 stories from top feed: ~$0.05
• 1,000 search results: ~$0.05
• Full thread with 200 comments: ~$0.02

Hacker News uses an open public API with no rate limits, so runs complete quickly and cheaply.

Tips

• Monitoring: Use topStories with minScore: 50 to get only high-signal stories
• Research: search mode with searchSortBy: date gives you chronological archives
• Comment depth: includeComments: false with feed modes keeps costs minimal while capturing discussion
• Thread analysis: thread mode gives the full discussion tree including nested replies

FAQ & disclaimers

Is this legal? Yes. This actor uses the official public Hacker News Firebase API provided by Y Combinator/Firebase and the public Algolia search API. No ToS violation occurs.

Can I get all HN data ever? HN has over 40 million items. The Algolia search API covers the full archive. For bulk exports use search mode with broad queries.

Something broken? Open an issue on the actor's Issues tab.