🟧 Hacker News Scraper — Stories, Comments, Search & Front Page

Scrape Hacker News (news.ycombinator.com): the front page, new, best, Ask HN, Show HN and jobs lists in their live order, keyword search across the whole HN history, every story that links to a domain, full comment threads with depth, and user profiles with their latest submissions. Schedule it with "only new" to get an alert-ready list of fresh mentions of your product, company or competitors.

$0.20 per 1,000 results (stories or comments). User profile rows and errors are free. No login, no API key.

• ✅ Live lists: Top (front page), New, Best, Ask HN, Show HN, Jobs — with the rank of each story
• ✅ Search stories, comments or both, by keyword, in the title only, or in the link URL (find every HN submission of yourdomain.com)
• ✅ Newest first with no 1,000-result cap: goes back through the whole HN archive, or by relevance
• ✅ Comment threads: every comment with author, text, date, parent and depth (1 = direct reply), in thread order
• ✅ Users: karma, account creation date, about text, number of submissions, plus their latest stories or comments
• ✅ Filters: last N days, minimum points, minimum comments, Ask/Show HN, jobs, polls, stories that reached the front page
• ✅ "Only new since last run": remembers what earlier runs returned — perfect for scheduled brand monitoring
• ✅ Clean text (HN's HTML converted to readable paragraphs, links kept as full URLs), ISO dates, numbers as numbers
• ✅ Uses only the official Hacker News API (Firebase) and the official HN Search API by Algolia — fast and stable

Who uses it

• Founders & marketers — track every mention of your product, company or competitors and every share of your website.
• Developer relations — collect feedback threads and Show HN launches in your space.
• Investors & analysts — follow trends, launches and "Who is hiring" posts over time.
• AI builders & researchers — high-quality tech discussions for datasets, summaries and sentiment analysis.
• Recruiters — job posts from the HN Jobs list.

How to use it

1. Pick front-page lists, and/or write search queries (apify, web scraping, github.com/yourorg).
2. To find every submission of a website, set Search in → Link URL only and use the domain as the query.
3. Add story links to download complete comment threads, or usernames to get profiles and their posts.
4. Optional: include comments of each story, last N days, min points, and only new since the last run.
5. Run, then export to CSV, Excel, JSON or send results to Slack, email, Google Sheets, Make, Zapier or n8n.

Input example

{
  "searchQueries": ["apify.com"],
  "searchIn": "url",
  "searchType": "story",
  "sortBy": "date",
  "newerThanDays": 30,
  "includeComments": true,
  "onlyNew": true
}

Output example

{
  "type": "story",
  "id": 50170593,
  "title": "SimCity, but with Permits",
  "url": "https://permit.petercai.com/",
  "domain": "permit.petercai.com",
  "author": "pcai",
  "points": 140,
  "comments": 67,
  "createdAt": "2026-10-20T18:29:09Z",
  "hnUrl": "https://news.ycombinator.com/item?id=50170593",
  "storyType": "story",
  "rank": 1,
  "source": "list:top"
}

Comment rows have "type": "comment" with storyId, storyTitle, parentId, author, text, createdAt, depth, replies and hnUrl. User rows have "type": "user" with username, karma, createdAt, about and submissions.

Pricing

$0.0002 per result ($0.20 per 1,000 stories or comments). User profiles, "not found" and input errors are free. Higher Apify plans get an automatic discount, and you can set a maximum cost per run.

Tips

• Monitoring: schedule a daily run with onlyNew on and connect it to Slack or email — you'll only get what's new.
• Clean up and combine several runs with Merge, Filter, Deduplicate & Transform Datasets.
• Read the linked articles in full with Article Text Extractor.

FAQ

Are points and comment counts live? Front-page lists and story links read the live official API, so points and comment counts are current. Search results come from the Algolia index, which can lag a few minutes behind for very fresh stories.

Can I get more than 1,000 search results? Yes — with Sort: newest first the Actor pages back through time with no limit. Relevance sorting is limited to 1,000 results per query by the search API.

Why don't comments have points? Hacker News doesn't publish comment scores to anyone except their author.

Are deleted or flagged items included? No. Deleted and dead items are skipped.

Is it legal? The Actor uses the public APIs that Hacker News provides for this purpose. Respect the authors' rights and privacy laws when you reuse the content.

Something doesn't work or you need a field? Open an issue in the Issues tab — we answer fast.