Hacker News Scraper — Extract Stories, Comments & User Data

Scrape Hacker News front page stories, search results, comments, and user profiles. Get structured data from HN that's easier to work with than the raw Firebase API.

Why Use This Actor?

• Beyond the API — Get structured data that's easier to work with than the raw HN Firebase API
• Bulk extraction — Scrape front page, new stories, Ask HN, Show HN, and jobs in one run
• Complete threads — Extract full comment trees with all nested replies
• Historical data — Search and extract stories from any time period

Features

• Front Page Stories — Extract current top/best/new stories with scores, comments, and metadata
• Story Comments — Get full comment threads with nested replies and author info
• Search — Search Hacker News stories by keyword with date range filtering
• User Profiles — Extract user karma, about text, and submission history
• Ask HN / Show HN / Jobs — Dedicated scraping for each HN section
• Multiple Sort Options — Sort by points, date, or number of comments
• Configurable Depth — Control how deep to go into comment trees

Input Parameters

Parameter	Type	Required	Description
scrapeType	string	Yes	Choose what to extract from Hacker News.
searchQuery	string	No	Search term to find stories on Hacker News.
searchSort	string	No	Sort search results by relevance or date.
dateFrom	string	No	Only return stories after this date. Format: YYYY-MM-DD
dateTo	string	No	Only return stories before this date. Format: YYYY-MM-DD
storyIds	array	No	Hacker News story IDs or URLs to scrape comments from.
usernames	array	No	HN usernames to scrape profile and submission history.
maxItems	integer	No	Maximum number of stories/comments to scrape.
minPoints	integer	No	Only return stories with at least this many upvotes. Leave 0 for all.
minComments	integer	No	Only return stories with at least this many comments. Leave 0 for all.

Output Example

{
  "type": "story",
  "title": "Show HN: I built an open-source alternative to Notion",
  "url": "https://github.com/example/project",
  "author": "techfounder",
  "score": 487,
  "numComments": 156,
  "createdAt": "2026-02-18T16:45:00Z",
  "hnUrl": "https://news.ycombinator.com/item?id=12345678",
  "storyType": "show_hn",
  "domain": "github.com"
}

Use Cases

• Tech Trend Monitoring — Track what's gaining traction in the developer community
• Startup Research — Find Show HN launches and community feedback
• Content Curation — Aggregate top stories for newsletters or content feeds
• Job Market Analysis — Extract "Who's Hiring" threads for job market insights
• Developer Sentiment — Understand how developers feel about tools, languages, and frameworks
• Competitive Intelligence — Monitor when your product or competitors are discussed

Cost Estimation

Scale	Estimated Cost	Time
Front page (30 stories)	~$0.01	~15 seconds
500 stories	~$0.05	~1 minute
Story + all comments	~$0.03	~30 seconds
10,000 search results	~$0.80	~10 minutes

Tips for Best Results

1. Use search mode with specific keywords for targeted data collection
2. Limit comment depth to 3-5 levels for most use cases
3. Front page scraping is the cheapest and fastest mode
4. Combine with scheduled runs for daily tech news monitoring

Integrations

Export to Google Sheets, JSON, CSV, or connect via Apify webhooks to Zapier, Make, Slack, Notion, and more.

Support

For issues, feature requests, or custom scraping needs, contact us at fatihdagustu20@gmail.com

---

Built with Crawlee and Apify SDK. Maintained and updated regularly.