Hacker News Scraper

Scrape stories from Hacker News. Get titles, points, authors, comment counts, and links from the front page, newest stories, Ask HN, Show HN, jobs, and best stories.

What does Hacker News Scraper do?

Hacker News Scraper extracts story data from any section of Hacker News. It collects the title, URL, domain, points (upvotes), author, age, comment count, and a direct link to the comments page.

It supports all major HN sections and paginated results.

Why scrape Hacker News?

Hacker News is the most influential tech community on the web, with millions of developers, founders, and investors reading it daily. Stories that trend on HN can drive massive traffic and shape industry conversations.

Key reasons to scrape it:

• Trend detection — Spot emerging technologies and hot topics in tech
• Content curation — Build newsletters, dashboards, or feeds from HN
• Sentiment analysis — Track community interest via points and comments
• Competitive intelligence — Monitor when competitors or products are discussed
• Research — Study tech community behavior and content patterns

Use cases

• Newsletter creators curating the top tech stories weekly
• Startup founders monitoring mentions of their products
• Data scientists building tech trend datasets
• Developers creating custom HN dashboards and alerts
• Investors tracking what technologies the community is excited about
• Researchers studying tech community dynamics

How to scrape Hacker News

1. Go to Hacker News Scraper on Apify Store
2. Choose a section (front page, newest, Ask HN, Show HN, jobs, or best)
3. Set the number of pages and max stories
4. Click Start and wait for results
5. Download data as JSON, CSV, or Excel

Input parameters

Parameter	Type	Default	Description
section	string	"front"	HN section: front, newest, ask, show, jobs, best
maxPages	integer	3	Max pages to scrape (30 stories/page)
maxStories	integer	100	Max total stories to extract

Input example

{
    "section": "front",
    "maxPages": 2,
    "maxStories": 60
}

Output

Each story in the dataset contains:

Field	Type	Description
rank	number	Position on the page
id	number	Hacker News story ID
title	string	Story title
url	string	Link to the article
domain	string	Source domain (e.g., "github.com")
points	number	Upvote count
author	string	Username who submitted the story
age	string	Time since posting (e.g., "4 hours ago")
commentCount	number	Number of comments
commentsUrl	string	Direct link to HN comments page
scrapedAt	string	ISO timestamp of extraction

Output example

{
    "rank": 1,
    "id": 47225130,
    "title": "The workers behind Meta's smart glasses can see everything",
    "url": "https://www.svd.se/a/K8nrV4/metas-ai-smart-glasses-and-data-privacy-concerns",
    "domain": "svd.se",
    "points": 619,
    "author": "sandbach",
    "age": "4 hours ago",
    "commentCount": 343,
    "commentsUrl": "https://news.ycombinator.com/item?id=47225130",
    "scrapedAt": "2026-03-03T02:54:49.013Z"
}

Pricing

Hacker News Scraper uses pay-per-event pricing:

Event	Price
Run started	$0.001
Story extracted	$0.001 per story

Cost examples

Scenario	Stories	Cost
Front page (1 page)	30	$0.031
Front page (3 pages)	90	$0.091
Multiple sections	150	$0.151

Platform costs are negligible — typically under $0.001 per run.

Using Hacker News Scraper with the Apify API

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('automation-lab/hackernews-scraper').call({
    section: 'front',
    maxPages: 1,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Found ${items.length} stories`);
items.forEach(story => {
    console.log(`#${story.rank} ${story.title} (${story.points} pts, ${story.commentCount} comments)`);
});

Python

from apify_client import ApifyClient

client = ApifyClient('YOUR_API_TOKEN')

run = client.actor('automation-lab/hackernews-scraper').call(run_input={
    'section': 'front',
    'maxPages': 1,
})

dataset = client.dataset(run['defaultDatasetId']).list_items().items
print(f'Found {len(dataset)} stories')
for story in dataset:
    print(f"#{story['rank']} {story['title']} ({story['points']} pts, {story['commentCount']} comments)")

Integrations

Hacker News Scraper works with all Apify integrations:

• Scheduled runs — Scrape HN every hour or daily for automated monitoring
• Webhooks — Get notified when a scrape completes
• API — Trigger runs and fetch results programmatically
• Google Sheets — Export stories to a spreadsheet for analysis
• Slack — Post top stories to your team's Slack channel

Connect to Zapier, Make, or Google Sheets for automated workflows.

Tips

• Use "best" section to get all-time popular stories
• Schedule hourly runs to track how stories rise and fall on the front page
• Compare points and comments to gauge engagement levels
• Filter by domain in post-processing to find stories from specific sources
• The "newest" section is great for catching stories before they trend
• Job stories (section: jobs) have no points or comments — they're job postings

FAQ

How many stories does each page have? Each HN page shows 30 stories. With maxPages: 10, you can get up to 300 stories per section.

Can I search for specific topics? HN doesn't have a built-in search on its main pages. For topic search, consider using the Algolia HN Search API separately.

How often does HN update? The front page changes continuously. Running every 15-60 minutes captures most significant changes.

What about job stories? Job postings (section: jobs) don't have points or comments. The author field will be the company posting the job.