Hacker News Scraper — search, front page, comments & users

Turn Hacker News into clean, structured, agent-ready data — no login, no API keys. Built on HN's official public APIs (Algolia HN Search + the Firebase v0 API), so it's fast, reliable, and legal for public data. Four modes in one actor:

Mode	What you get	Use it for
Search	Full-text search across all of HN history — filter by points, recency, tags	Brand / competitor / keyword monitoring, sentiment, market research
Front page list	The live top / new / best / ask / show / job stories	Trend tracking, daily digests, "what's hot in tech right now"
Item + comments	A story plus its entire comment thread, flattened	Deep discussion analysis, community sentiment, dataset building
Users	A user's profile, karma, and activity	Influencer / contributor research

Why it's built for AI agents

Exposed as an MCP tool, this lets an agent answer "what is Hacker News saying about X?" in one call — search a topic, pull the discussion, read the comment thread. Structured JSON out, no scraping fragility.

Output

Clean, flat rows (CSV/Excel/JSON/API). Stories include title, url, points, num_comments, author, created_at, text, and a direct hn_url. Comments include text, author, story_id, parent_id. Users include karma, about, created_at.

Input examples

{ "mode": "search", "query": "your product", "minPoints": 50, "sinceDays": 30, "maxItems": 500 }

{ "mode": "list", "listType": "top", "maxItems": 100 }

{ "mode": "item", "itemIds": ["44444444"], "includeComments": true }

{ "mode": "user", "usernames": ["pg", "dang"] }

Use cases

• Tech-trend monitoring — track what topics, tools and startups reach the Hacker News front page over time.
• Brand & competitor mentions — search all of HN for your product name; get every thread and comment for sentiment analysis.
• Content research — find the highest-scoring posts on any topic to understand what the developer audience cares about.
• Datasets for NLP/LLM work — comment threads with scores and authors, ready for research.
• AI agents — as an MCP tool: "what does Hacker News think about {product}?" answered with live threads.

FAQ

How do I search Hacker News posts and comments programmatically? Give the actor a search query (it uses HN's public Algolia index) and it returns stories and, optionally, full comment trees as structured JSON.

Can I get the Hacker News front page as an API? Yes — front-page mode returns the current ranked front page; schedule it hourly to build a front-page history dataset.

Can I export HN threads to CSV for analysis? Every run's dataset exports as CSV, Excel or JSON from the Apify console or API.

How do I track mentions of my startup on HN? Schedule a search run for your brand name; new items land in the dataset each run and can trigger a webhook or Slack/Zapier notification.

Does it include comment scores and authors? Yes — items include author, points, timestamps, and comments include their full text and thread structure.

Legal

Uses Hacker News's public, documented APIs (Algolia + Firebase). No login, no cookies, no circumvention. You're responsible for using the data within applicable law.

Pricing

Pay-per-event: a small actor-start fee plus a per-item charge for each story / comment / user returned — you pay for exactly the data you pull.