Hacker News Comments & Threads Scraper

Extract comments, threads, and user profiles from Hacker News using the official Firebase API. No authentication needed, no rate limits.

Features

• 💬 Full comment threads — all comments with nested replies, depth tracking
• 📊 Flat or nested output — choose flat (one row per comment, great for analysis) or nested tree structure
• 🔥 Top & New stories — automatically scrape current front page or newest stories
• 👤 User profiles — optionally fetch karma, account age, and bio for each commenter
• 🔗 URL or ID input — paste HN URLs directly or use numeric story IDs
• ⚡ Official API — uses HN's Firebase API, no scraping/blocking issues
• 🧹 Clean text — HTML tags stripped, entities decoded

Input

Field	Type	Default	Description
storyIds	string[]	[]	HN story IDs (e.g. 41009141)
storyUrls	string[]	[]	Full HN URLs
scrapeTopStories	boolean	false	Scrape current top stories
scrapeNewStories	boolean	false	Scrape newest stories
maxStories	number	10	How many top/new stories
maxCommentsPerStory	number	0	Max comments (0 = all)
includeUserProfiles	boolean	false	Fetch commenter profiles
flattenComments	boolean	true	One row per comment

Example Input

{
  "storyUrls": ["https://news.ycombinator.com/item?id=41009141"],
  "scrapeTopStories": true,
  "maxStories": 5,
  "maxCommentsPerStory": 100,
  "flattenComments": true
}

Output (flat mode)

Story items:

{
  "type": "story",
  "id": 41009141,
  "title": "Show HN: ...",
  "url": "https://example.com",
  "by": "pg",
  "score": 342,
  "descendants": 156
}

Comment items:

{
  "type": "comment",
  "storyId": 41009141,
  "storyTitle": "Show HN: ...",
  "id": 41009500,
  "by": "commenter",
  "text": "This is really interesting because...",
  "date": "2026-03-30T12:00:00.000Z",
  "depth": 0,
  "childCount": 3
}

Use Cases

• Sentiment analysis — analyze community reactions to launches, technologies, companies
• Market research — track what developers and tech leaders discuss
• Content mining — find popular topics, recurring themes, expert opinions
• Competitive intelligence — monitor mentions of your product or competitors
• Academic research — study online discussion patterns and community dynamics
• Lead generation — find engaged users discussing relevant topics

Technical Details

• Uses HN's official Firebase API (hacker-news.firebaseio.com/v0)
• No authentication required, no CAPTCHA, no blocking
• Polite rate limiting (50ms between requests)
• Automatic retries with backoff on transient errors
• HTML entity decoding and tag stripping for clean text output