Hacker News Scraper

Scrape stories, comments, and discussions from Hacker News — extract titles, scores, URLs, authors, and full comment threads at scale. The Hacker News Scraper is the fastest way to collect structured data from all HN sections including Top, New, Best, Ask HN, Show HN, and Jobs.

What does Hacker News Scraper do?

This actor fetches data directly from the official Hacker News Firebase API, extracting every available field for each story: title, URL, author, score (points), comment count, submission text, timestamp, and a direct link to the HN discussion. When comment scraping is enabled, it walks the entire comment tree and extracts author, text, depth level, parent ID, and timestamp for each comment.

Why scrape Hacker News?

Hacker News is the most influential technology community with over 10 million monthly visitors. It's where startup founders, engineers, and investors share and discuss the latest in tech. Scraping Hacker News enables powerful use cases:

• Tech trend analysis — Track which topics, frameworks, and technologies are gaining traction in the developer community.
• Startup intelligence — Monitor Show HN and launch announcements to discover new startups and products before they go mainstream.
• Hiring insights — Scrape the monthly "Who's Hiring" threads to analyze job market trends, popular technologies, and salary ranges.
• Content marketing — Identify what types of content perform best on HN to inform your content strategy and distribution.
• Sentiment analysis — Feed story titles and comments into NLP pipelines to measure developer opinion on any technology or company.
• Competitive monitoring — Track mentions of your company, product, or competitors across HN discussions.
• Academic research — Build datasets of tech community discourse for social science, NLP, and information retrieval studies.

How to use Hacker News Scraper

1. Go to the actor's input tab.
2. Choose which sections to scrape (topstories, newstories, beststories, askstories, showstories, jobstories).
3. Set the maximum number of stories to collect per section.
4. Optionally enable comment scraping and set a comment limit per story.
5. Click Start and download results from the Dataset tab.

The actor works with zero configuration — the default input scrapes 100 top stories from Hacker News.

Input parameters

Parameter	Type	Description
sections	Array of strings	Sections to scrape: topstories, newstories, beststories, askstories, showstories, jobstories
maxStoriesPerSection	Integer	Max stories per section (0 = all available up to 500)
includeComments	Boolean	Also scrape the comment tree for each story
maxCommentsPerStory	Integer	Max comments per story when enabled
minScore	Integer	Only include stories with at least this many points (0 = all)

Output format

Each story is a JSON object in the dataset:

{
    "type": "story",
    "id": 39876543,
    "title": "Show HN: An open-source tool for building AI agents",
    "url": "https://github.com/example/ai-agent-toolkit",
    "author": "techfounder",
    "score": 342,
    "numComments": 187,
    "text": "",
    "createdAt": "2026-02-28T14:30:00.000Z",
    "hnUrl": "https://news.ycombinator.com/item?id=39876543",
    "section": "topstories",
    "dead": false,
    "deleted": false
}

When comment scraping is enabled, comments appear as separate items:

{
    "type": "comment",
    "id": 39876600,
    "storyId": 39876543,
    "storyTitle": "Show HN: An open-source tool for building AI agents",
    "author": "developer_jane",
    "text": "This is really impressive. I've been looking for something like this...",
    "createdAt": "2026-02-28T15:12:00.000Z",
    "hnUrl": "https://news.ycombinator.com/item?id=39876600",
    "depth": 0,
    "parentId": 39876543
}

Cost of usage

This actor uses lightweight HTTP requests to the free Hacker News API with no proxy needed, making it extremely cost-efficient.

• Per story: ~$0.001
• Per 1,000 stories: ~$1
• Actor start cost: ~$0.005

A typical run scraping 100 top stories costs under $0.10 in platform usage. Enabling comments increases cost proportionally to the number of API calls needed per comment tree.

Tips and advanced usage

• Multi-section analysis — Add multiple sections to compare what content performs best across different HN categories.
• Score filtering — Use minScore to focus on high-engagement stories and filter out noise.
• Job market analysis — Scrape jobstories to analyze hiring trends, popular tech stacks, and company locations.
• Ask HN insights — Scrape askstories to find common questions and pain points in the developer community.
• Schedule monitoring — Use Apify Schedules to run hourly or daily and track how the HN front page evolves over time.
• Comment sentiment — Enable includeComments and pipe comment text into sentiment analysis tools for deeper community insights.

Related actors

• Trustpilot Review Scraper — Scrape business reviews and ratings from Trustpilot.
• eBay Product Scraper — Scrape eBay product listings with prices and details.
• Podcast Review Scraper — Scrape podcast reviews from major platforms.