Hacker News Scraper

Extract stories, jobs, and polls from Hacker News with powerful filtering. Uses the official HN Algolia API for reliable, high-speed data extraction — no browser automation required.

Features

• Full-text search: Search HN stories by keyword, topic, or author
• Type filtering: Scrape stories, job postings, or polls
• Date ranges: Filter by post date to get historical data
• Quality filters: Set minimum upvotes and comment thresholds
• Sort options: By relevance or chronologically (newest first)
• Comment extraction: Optionally fetch top comments per story
• Fast & reliable: Uses the HN Algolia API, no Playwright/browser needed
• Structured output: JSON dataset ready for analysis, CSV export

Use Cases

• Tech trend analysis: Track what topics are trending in the developer community
• Job hunting: Scrape "Who's Hiring" and "Ask HN: Who is hiring?" threads
• Competitive intelligence: Monitor what people say about products/companies
• Research: Gather HN data for academic or market research
• Content discovery: Find highly-upvoted stories on any topic

Input

{
    "searchQuery": "AI agents",
    "storyTypes": ["story"],
    "maxResults": 200,
    "sortBy": "date",
    "minPoints": 10,
    "minComments": 5,
    "dateFrom": "2024-01-01T00:00:00Z",
    "dateTo": "2025-12-31T23:59:59Z",
    "includeComments": false
}

Parameters

Parameter	Type	Default	Description
searchQuery	string	""	Search term. Leave empty for all stories
storyTypes	array	["story"]	Types: story, job, poll
maxResults	integer	100	Max stories to return (up to 10,000)
sortBy	string	"relevance"	"relevance" or "date" (newest first)
minPoints	integer	0	Min upvotes filter
minComments	integer	0	Min comments filter
dateFrom	string	""	ISO 8601 date (e.g. 2024-01-01T00:00:00Z)
dateTo	string	""	ISO 8601 date
includeComments	boolean	false	Fetch top comments per story
maxCommentsPerStory	integer	10	Max comments to fetch (if enabled)

Output

Each story is saved to the Apify dataset:

{
    "id": "39654819",
    "title": "Show HN: I built an autonomous AI agent that earns money",
    "url": "https://example.com/agent",
    "type": "story",
    "author": "opspawn",
    "points": 342,
    "comments": 87,
    "createdAt": "2024-03-15T14:22:00.000Z",
    "createdAtTimestamp": 1710511320,
    "text": null,
    "hackerNewsUrl": "https://news.ycombinator.com/item?id=39654819",
    "topComments": [
        {
            "id": "39655001",
            "author": "throwaway123",
            "text": "This is fascinating. How does the agent decide...",
            "createdAt": "2024-03-15T15:01:00.000Z"
        }
    ]
}

Example Workflows

Get top AI stories from 2024

{
    "searchQuery": "artificial intelligence",
    "maxResults": 500,
    "sortBy": "relevance",
    "minPoints": 100,
    "dateFrom": "2024-01-01T00:00:00Z",
    "dateTo": "2024-12-31T23:59:59Z"
}

Scrape HN Job Postings

{
    "storyTypes": ["job"],
    "maxResults": 200,
    "sortBy": "date"
}

Monitor a company on HN

{
    "searchQuery": "OpenAI",
    "maxResults": 1000,
    "sortBy": "date",
    "minPoints": 5
}

API

This actor uses the Hacker News Algolia API — a free, official API maintained by Algolia in partnership with YCombinator.

Support

Built by OpSpawn — autonomous AI agent infrastructure.