Hacker News Scraper: Front Page, Search, Comments, Ask HN, Show HN & Jobs

Get Hacker News data as clean JSON, CSV or Excel through the official Hacker News API and the HN Search API (Algolia): the live front page (top), new, best, Ask HN, Show HN and job lists, full-text search of every story and comment since 2006, and complete comment threads. Each row has the title, link, domain, points, comment count, author, time, text and the HN discussion link.

No HTML scraping, no login, no proxy: fast, cheap and reliable. $0.20 per 1,000 items.

What you can use it for

• Brand and competitor monitoring: schedule a search for your product or company name with Only new items and get every new mention (stories and comments).
• Launch tracking: every Show HN and Launch HN post with points and comments, to see what developers build and what gets traction.
• Content and newsletter curation: today's front page or the best stories of the week with points above a threshold.
• Developer sentiment and research: full comment threads of a discussion for analysis or an LLM summary.
• Hiring and job research: HN job posts (YC startups) and the monthly "Who is hiring?" threads with all comments.
• Domain tracking: every HN submission linking to your site (or github.com, a news outlet, a competitor).
• AI agents (MCP) and RAG: structured HN data an agent can query on demand.

How to use it

1. Choose a mode: list (front page and other live lists), search (full-text search with filters) or items (specific IDs/URLs).
2. Add filters: date range, minimum points/comments, domains, authors.
3. Optional: turn on Include comment threads (charged per comment).
4. Click Start, or schedule it with Only new items for alerts.

Why this scraper

• Official APIs only: the Firebase-hosted Hacker News API and HN Search by Algolia, both public and free. Nothing breaks when the HN page layout changes.
• Search all of HN history: stories, comments, Ask HN, Show HN, jobs and polls since 2006, by relevance or newest first. Newest-first search pages past HN Search's 1,000-result cap.
• Front page with rank: the top list in its live front-page order, plus new, best, ask, show and job.
• Comment threads: in thread order with depth and parent ID, limited by count and depth.
• Only new items: for scheduled keyword alerts you pay only for items you haven't seen; with newest-first search, a run stops as soon as it reaches items from the previous run.
• Cheap: $0.20 per 1,000 items, comments $0.05 per 1,000. Filtered-out, already-seen and not-found items are free. 256 MB memory.
• Privacy-minded: only public HN data. Usernames can be switched off with Include usernames.

Modes

Mode	What you get
list (default)	Live lists: top (front page ranking, up to 500), new, best, ask, show, job, with list and rank.
search	Full-text search over all HN items. searchType: story, comment, ask, show, job, poll, front_page or any. sortBy: relevance or date.
items	Specific item IDs (8863) or URLs (https://news.ycombinator.com/item?id=8863): stories, comments, jobs, polls.

Input examples

Today's front page, top 30 stories:

{ "mode": "list", "lists": ["top"], "maxItems": 30 }

Every new mention of your brand (stories and comments), as an alert on a schedule:

{ "mode": "search", "searchQueries": ["apify"], "searchType": "any", "sortBy": "date", "onlyNew": true, "stateKey": "brand-alert" }

Show HN posts from the last 7 days with 50+ points, with top-level comments:

{ "mode": "search", "searchQueries": [""], "searchType": "show", "dateFrom": "7 days", "minPoints": 50, "includeComments": true, "maxCommentDepth": 1 }

Every HN story linking to github.com this month:

{ "mode": "search", "searchType": "story", "domains": ["github.com"], "dateFrom": "2026-10-01", "sortBy": "date", "maxItems": 1000 }

A full discussion thread:

{ "mode": "items", "itemIds": ["https://news.ycombinator.com/item?id=8863"], "includeComments": true, "maxCommentsPerStory": 0 }

Output

{
  "id": 8863, "type": "story", "title": "My YC app: Dropbox - Throw away your USB drive",
  "url": "http://www.getdropbox.com/u/2/screencast.html", "domain": "getdropbox.com",
  "hnUrl": "https://news.ycombinator.com/item?id=8863", "author": "dhouston",
  "points": 104, "commentsCount": 71, "createdAt": "2007-04-04T19:16:40Z", "text": null,
  "list": "top", "rank": 1, "matchedQuery": null,
  "comments": [{ "id": 8865, "parentId": 8863, "depth": 1, "author": "dhouston", "createdAt": "2007-04-04T19:22:55Z",
                 "text": "oh, and a mac port is coming :)", "hnUrl": "https://news.ycombinator.com/item?id=8865", "storyId": 8863 }],
  "commentsIncluded": 1, "scrapedAt": "2026-10-22T07:30:00Z"
}

type is one of story, ask, show, launch, job, poll or comment. Comment search hits carry storyId, parentId and the story title. A summary (items saved, already seen, filtered, total hits, API requests) is saved as OUTPUT in the run's key-value store.

Pricing

Pay per event, no monthly rent:

• $0.0002 per item saved ($0.20 per 1,000)
• $0.00005 per comment in a thread ($0.05 per 1,000), only with Include comment threads
• $0.0003 per run start

Example	Cost
Front page, top 30 stories	$0.0063
1,000 search results	$0.20
100 stories with 100 comments each	$0.52
Daily brand alert, ~20 new mentions a day, for 30 days	$0.13

Filtered-out, already-seen and not-found items are free. You can set a maximum cost per run in the run options; the Actor stops cleanly when it is reached.

API, integrations and MCP

• API: run it from any language with the Apify API or the apify-client for JavaScript/Python, and read results from the dataset.
• No-code: use the Apify app in n8n, Make or Zapier, or send results to Google Sheets, Slack or a webhook.
• AI agents (MCP): add https://mcp.apify.com?tools=manageable_gully/hacker-news-scraper to Claude, Cursor, VS Code or any MCP client.
• Schedules: run it hourly or daily from Apify Console, with Only new items for alerts.

FAQ

Is it allowed? Yes. It uses the official Hacker News API (published by Y Combinator for public use) and the public HN Search API by Algolia, at modest request rates, without logging in. This Actor is not affiliated with Y Combinator or Algolia.

How fresh is the data? Lists come live from the official API. HN Search indexes new items within minutes.

Why do points differ slightly from the website? Points and comment counts change constantly; each row reflects the moment it was fetched.

Are deleted or dead items included? No. They are skipped (and not charged).

Can I get user profiles or karma? No. The Actor returns public usernames on items only, and you can switch them off.

More tools from WebIntel Tools

• RSS Feed Reader & Monitor: read and monitor RSS, Atom and JSON feeds, only new items
• Product Hunt Scraper: today's Product Hunt launches and leaderboards with websites
• Web Page to Markdown: clean Markdown and RAG chunks from any URL, for LLMs
• arXiv & PubMed Paper Search: research papers through the official APIs