Hacker News Scraper

Scrape Hacker News stories, jobs, discussions, and user profiles from all major sections. Extract front page posts, Show HN launches, Ask HN discussions, and job listings with comments, scores, and full metadata.

Features

• Scrape front page, Ask HN, Show HN, and Hacker News Jobs sections
• Retrieve user profiles, karma scores, and submission history
• Pull top-level comments for any story
• Flat or nested output formats for flexible data processing
• Configurable item limits per section and total
• Timestamp conversion to human-readable or ISO format
• Filter by specific item IDs or usernames

Why Use This Actor?

Hacker News is one of the most influential tech communities, shaping trends across startups, open source, and developer tools. This scraper gives you structured access to that data for competitive intelligence, trend analysis, lead generation, and market research.

Use Cases

• Trend Monitoring — Track what's trending on the front page daily
• Startup Intelligence — Monitor Show HN launches and competitor activity
• Job Market Analysis — Scrape startup job postings and hiring trends
• Developer Community Research — Analyze user profiles and submission patterns
• Content Curation — Aggregate top tech stories for newsletters or dashboards
• AI/ML News Tracking — Filter AI and machine learning discussions on HN

Pricing

This is a pay-per-result Actor at $0.0019 per result. You only pay for the items actually scraped.

Input Parameters

Parameter	Type	Default	Description
sections	string[]	["front"]	Sections to scrape: front, ask, show, jobs
maxItemsPerSection	integer	50	Max items to fetch per section
maxItemsTotal	integer	—	Global cap across all sections
itemIds	string[]	—	Scrape specific HN item IDs directly
usernames	string[]	—	Fetch specific HN user profiles
includeCommentCount	boolean	false	Include total comment count per story
includeTopLevelComments	boolean	false	Fetch top-level comments for stories
convertTimestamps	boolean	false	Convert Unix timestamps to ISO 8601
outputFormat	string	"flat"	flat (one row per item) or nested (with replies)

Output

The Actor produces a dataset with items in the selected format. Each story item includes:

• id — HN item ID
• title — Story title
• url — Link URL (if external)
• score — Points/votes
• by — Author username
• time — Unix timestamp (converted if enabled)
• descendants — Total comment count (if enabled)
• kids — Array of child item IDs
• text — Text content (for Ask HN, comments)
• section — Source section (front, ask, show, jobs)

Dataset Views

This Actor exposes two dataset views:

• stories — Flat list of all scraped stories with metadata
• users — User profiles and karma data

Tips

• Use maxItemsPerSection: 100 to get a broader view of current trends
• Set convertTimestamps: true for easier date filtering in downstream tools
• Combine includeTopLevelComments: true with outputFormat: "nested" for full discussion threads
• Use itemIds to re-scrape specific stories for updated comment counts

Related Actors

Verified related actors on Apify that pair well with this one. All links point to real, publicly available actors.

• Reddit Scraper
• YouTube Scraper
• Twitter Scraper