Hacker News Scraper — Stories, Comments & Jobs

Extract stories, comments, jobs, Ask HN, and Show HN from Hacker News. Filter by keyword, date, points, and author. Fetch entire comment threads. No browser, no proxies, no auth needed.

---

Who is this for?

• Developers & researchers tracking tech trends, AI news, or startup discussions
• Recruiters extracting job listings from monthly "Who is Hiring" threads
• Marketers monitoring Show HN launches and community sentiment
• Data analysts building HN datasets for LLMs, RAG pipelines, or dashboards
• Anyone who wants to schedule automatic HN monitoring without writing code

---

Why this scraper?

• 128 MB memory, 2–30 sec per run — smallest Apify tier, lowest cost possible
• No browser, no proxies — uses Algolia Search API + HN Firebase API directly
• Full comment threads — fetch every comment for any story with one toggle
• Dual mode — full-text search (Algolia) or live feeds (Firebase top/new/best)
• Clean output — HTML stripped, hnUrl always present, domain extracted

---

Input

Field	Type	Default	Description
mode	search / feed	search	Search via Algolia or fetch a live Firebase feed
query	string	—	Keyword to search (required for mode=search)
type	story / comment / ask_hn / show_hn / job / all	story	Content type
sortBy	relevance / date	relevance	Sort order
feedType	top / new / best / ask / show / job	top	Feed type (only for mode=feed)
maxItems	integer	100	Maximum results (1–1000)
minPoints	integer	—	Minimum story points filter
minComments	integer	—	Minimum comment count filter
dateFrom	string	—	Filter after this date (ISO format, e.g. 2024-01-01)
dateTo	string	—	Filter before this date
author	string	—	Filter by HN username
storyId	string	—	Fetch all comments for a specific story by ID
includeComments	boolean	false	Also fetch all comments for each story found

---

Output

Story

{
  "id": "39930210",
  "type": "story",
  "title": "Show HN: I built a HN scraper",
  "url": "https://github.com/example/hn-scraper",
  "hnUrl": "https://news.ycombinator.com/item?id=39930210",
  "domain": "github.com",
  "author": "pg",
  "points": 342,
  "commentCount": 87,
  "tags": ["story", "author_pg"],
  "publishedAt": "2024-03-15T10:30:00.000Z",
  "scrapedAt": "2026-05-25T12:00:00.000Z"
}

Comment

{
  "id": "39930300",
  "type": "comment",
  "text": "Great work! How do you handle rate limits?",
  "author": "dang",
  "storyId": "39930210",
  "storyTitle": "Show HN: I built a HN scraper",
  "hnUrl": "https://news.ycombinator.com/item?id=39930300",
  "publishedAt": "2024-03-15T11:00:00.000Z",
  "scrapedAt": "2026-05-25T12:00:00.000Z"
}

Export in JSON, CSV, Excel, XML directly from Apify Console.

---

Example use cases

Get today's top 30 stories:

{ "mode": "feed", "feedType": "top", "maxItems": 30 }

Track AI stories with 100+ points this month:

{
  "mode": "search",
  "query": "artificial intelligence",
  "type": "story",
  "sortBy": "date",
  "minPoints": 100,
  "dateFrom": "2024-06-01",
  "maxItems": 50
}

Extract all comments from a specific story (great for LLM input):

{ "storyId": "39930210", "type": "comment", "maxItems": 1000 }

Scrape the monthly "Who is Hiring" thread:

{
  "mode": "search",
  "query": "ask hn who is hiring",
  "type": "ask_hn",
  "sortBy": "date",
  "maxItems": 5
}

Monitor all new Show HN launches:

{ "mode": "feed", "feedType": "show", "maxItems": 50 }

Get all stories + their comments in one run:

{
  "mode": "search",
  "query": "rust programming",
  "type": "story",
  "minPoints": 50,
  "maxItems": 20,
  "includeComments": true
}

---

Schedule daily monitoring

Run this actor on a schedule to track any HN topic automatically.

1. Open the actor and click Schedule
2. Set frequency (daily, weekly, or custom cron)
3. Results accumulate in your dataset — ready to export or pipe into your app

No code required.

---

APIs used

API	Use case	Auth
Algolia HN Search	Search, filtering, comment threads	None
HN Firebase API	Live feeds (top/new/best/ask/show/job)	None

---

Support

Found a bug or need a feature? Open an issue and I'll get back to you quickly.