Hacker News Scraper — Stories, Comments & Jobs

Extract Hacker News stories, comments, and job posts with full-text search. Filter by score, comment count, and keywords. No API key needed.

Uses the official HN Firebase API and Algolia Search API for fast, reliable results.

---

Free Trial Ending April 3

This actor is free while we collect feedback. Starting April 3, 2026, it moves to $4.99/month. Add a payment method at apify.com/billing to keep access.

---

Features

Feature	Details
Front Page Stories	Scrape top, new, and best stories with scores and metadata
Full-Text Search	Search all HN content via Algolia — posts, comments, titles
Ask HN / Show HN	Filter community questions and project showcases
Job Posts	YC startup job listings from "Who is Hiring?" threads
Comments	Threaded comments with author, text, and timestamps
Filters	Minimum score, minimum comment count, keyword matching
User Profiles	Karma, about text, account age, submission history

Input Parameters

Parameter	Type	Default	Description
category	string	"top"	top, new, best, ask, show, jobs, or search
searchQuery	string	—	Keyword for search mode (Algolia full-text search)
scrapeType	string	"stories"	stories, comments, or users
maxItems	integer	100	Maximum items to return (1–500)
includeComments	boolean	false	Fetch comment threads for each story
maxCommentsPerStory	integer	50	Max comments per story
minScore	integer	0	Minimum story score filter
minComments	integer	0	Minimum comment count filter
keyword	string	—	Additional keyword filter within results

Example Inputs

Get today's top stories

{
  "category": "top",
  "maxItems": 50
}

Search by keyword

{
  "category": "search",
  "searchQuery": "large language models",
  "maxItems": 100,
  "minScore": 50
}

Get Show HN posts with comments

{
  "category": "show",
  "maxItems": 30,
  "includeComments": true,
  "maxCommentsPerStory": 20
}

Filter by keyword and comment count

{
  "category": "top",
  "keyword": "rust",
  "minComments": 50,
  "maxItems": 25
}

Output Format

Story

{
  "title": "Show HN: Open-source LLM that runs on a Raspberry Pi",
  "url": "https://github.com/example/tiny-llm",
  "score": 487,
  "by": "techbuilder",
  "descendants": 203,
  "time": 1711540800,
  "type": "story",
  "hnUrl": "https://news.ycombinator.com/item?id=12345678"
}

Comment

{
  "text": "This is impressive. I tested it on my Pi 5 and inference takes about 2s per token...",
  "by": "commenter",
  "time": 1711544400,
  "parent": 12345678,
  "storyId": 12345678,
  "type": "comment"
}

Use Cases

• Tech trend monitoring — track what developers discuss and upvote
• Startup research — find YC companies, Show HN launches, and job openings
• Content curation — build newsletters from high-scoring posts by topic
• Competitive intelligence — monitor mentions of your product or competitors
• Recruiting — extract job posts from "Who is Hiring?" threads
• Market research — analyze sentiment around technologies and frameworks
• Academic research — study developer opinions and technology adoption

Why This Scraper?

HN has two official APIs (Firebase and Algolia), but neither is optimized for bulk extraction. This Actor combines both:

• Firebase API for real-time front page, new, and best stories
• Algolia Search API for full-text search across the full HN archive

FAQ

Do I need an API key? No. Both APIs are public. No authentication needed.

How far back can I search? The full HN archive back to 2006 via Algolia search.

How fresh is the data? Real-time. Firebase reflects the live front page; Algolia indexes new posts within minutes.

What export formats are supported? JSON, CSV, Excel, XML, HTML. Connect via API, webhooks, Zapier, Make, or Google Sheets.

Integrations

• REST API — trigger runs and fetch results programmatically
• Webhooks — get notified when scraping completes
• Zapier / Make — connect to 5,000+ apps
• Google Sheets — export directly to spreadsheets
• Slack / Email — set up alerts for trending topics

Related Scrapers

• Bluesky Scraper — Posts, profiles & search
• LinkedIn Jobs Scraper — Public job listings
• Reddit Scraper — Community discussions & threads

See all scrapers by CryptoSignals