Hacker News Scraper — Stories, Comments, Search & Profiles

Scrape Hacker News (news.ycombinator.com) stories, comments, search results, and user profiles. Get titles, scores, authors, comment counts, and timestamps for any HN listing.

Features

• Front page — Top stories from the HN home page
• New stories — Latest submissions
• Best stories — Highest-ranked stories
• Ask HN — Questions and discussions
• Show HN — Project showcases and demos
• Search — Search stories by keyword via the Algolia HN Search API
• User profiles — Scrape karma, about text, and account creation date
• Comments — Optionally scrape top-level comments for each story
• Pagination — Automatically fetches multiple pages up to your max items limit
• No authentication — Uses public HN pages and the Algolia API

Use cases

• Trend monitoring — Track what's trending in tech, startups, and programming
• Content curation — Auto-collect top stories for newsletters or blog roundups
• Sentiment analysis — Scrape comments for opinion mining and discussion analysis
• User research — Profile HN users by karma, submissions, and activity
• RAG / AI — Feed HN discussions into LLM pipelines for tech Q&A
• Data analysis — Analyze story scores, comment counts, and posting patterns

Input

Field	Type	Required	Default	Description
mode	string	✅	front-page	front-page, new, best, ask, show, search, user
query	string	❌	—	Search query (search mode only)
username	string	❌	—	HN username (user mode only)
maxItems	integer	❌	30	Max items to return (1–500)
includeComments	boolean	❌	false	Scrape top-level comments for each story

Example input

{
    "mode": "front-page",
    "maxItems": 30,
    "includeComments": false
}

Search example:

{
    "mode": "search",
    "query": "self-hosted homelab",
    "maxItems": 20
}

Output

Story

{
    "itemType": "story",
    "hnId": "39501234",
    "title": "Show HN: I built a self-hosted analytics tool",
    "url": "https://example.com/analytics",
    "hnUrl": "https://news.ycombinator.com/item?id=39501234",
    "score": 245,
    "author": "johndoe",
    "postedAt": "2025-07-04T12:00:00+00:00",
    "commentCount": 87,
    "rank": 1,
    "scrapedAt": "2025-07-04T15:00:00Z"
}

Comment

{
    "itemType": "comment",
    "hnId": "39501500",
    "author": "alice",
    "postedAt": "2025-07-04T12:30:00+00:00",
    "text": "This is really cool! How does it handle...",
    "scrapedAt": "2025-07-04T15:00:00Z"
}

User

{
    "itemType": "user",
    "author": "pg",
    "karma": 150000,
    "about": "Cofounder of YC",
    "postedAt": "2007-01-01",
    "scrapedAt": "2025-07-04T15:00:00Z"
}

Pricing

$0.50 per 1,000 results

A standard front-page scrape returns 30 stories for less than $0.02. Comment scraping costs more due to additional page requests.

Notes

• Rate limiting: 1 second delay between page fetches, 0.5s between comment pages
• Search mode uses the Algolia HN Search API (api.hn.algolia.com) for fast, reliable results
• postedAt for listing pages is an approximate timestamp converted from HN's relative age text (e.g. "3 hours ago")
• For exact timestamps, use the search mode which returns precise creation dates from Algolia