Hacker News Scraper

Extract stories, comments, jobs, Ask HN, and Show HN from Hacker News — fast, cheap, no browser required.

Features

• Full-text search via Algolia HN Search API — filter by keyword, date, points, comments, author
• Live feeds via HN Firebase API — top, new, best, ask, show, job
• Comment fetching — get all comments for individual stories or include comments in a search run
• 128 MB / ~2–30s — no browser, no proxies, no auth needed
• Clean output — HTML stripped, hnUrl always present, domain extracted

---

Input

Field	Type	Default	Description
mode	search | feed	search	Search via Algolia or fetch a live Firebase feed
query	string	—	Keyword to search (required for mode=search)
type	story | comment | ask_hn | show_hn | job | all	story	Content type to retrieve
sortBy	relevance | date	relevance	Sort order for search results
feedType	top | new | best | ask | show | job	top	Feed to fetch (only for mode=feed)
maxItems	integer	100	Maximum results (1–1000)
minPoints	integer	—	Minimum story points
minComments	integer	—	Minimum comment count
dateFrom	string (ISO)	—	Filter items after this date (e.g. 2024-01-01)
dateTo	string (ISO)	—	Filter items before this date
author	string	—	Filter by HN username
storyId	string	—	Fetch all comments for a specific story ID
includeComments	boolean	false	Also fetch comments for each story found

Example inputs

Search AI stories with 100+ points:

{
  "mode": "search",
  "query": "artificial intelligence",
  "type": "story",
  "sortBy": "date",
  "minPoints": 100,
  "maxItems": 50
}

Fetch current top 30 stories:

{
  "mode": "feed",
  "feedType": "top",
  "maxItems": 30
}

Get all Ask HN posts:

{
  "mode": "search",
  "query": "who is hiring",
  "type": "ask_hn",
  "maxItems": 20
}

Get all comments for a specific story:

{
  "mode": "search",
  "query": "",
  "type": "comment",
  "storyId": "39930210",
  "maxItems": 500
}

---

Output

Story

{
  "id": "39930210",
  "type": "story",
  "title": "Show HN: I built a HN scraper",
  "url": "https://github.com/example/hn-scraper",
  "hnUrl": "https://news.ycombinator.com/item?id=39930210",
  "domain": "github.com",
  "author": "pg",
  "points": 342,
  "commentCount": 87,
  "text": null,
  "tags": ["story", "author_pg"],
  "publishedAt": "2024-03-15T10:30:00.000Z",
  "scrapedAt": "2026-05-21T12:00:00.000Z"
}

Comment

{
  "id": "39930300",
  "type": "comment",
  "text": "Great work! How do you handle rate limits?",
  "textHtml": "<p>Great work! How do you handle rate limits?</p>",
  "author": "dang",
  "points": null,
  "storyId": "39930210",
  "storyTitle": "Show HN: I built a HN scraper",
  "parentId": "39930210",
  "hnUrl": "https://news.ycombinator.com/item?id=39930300",
  "publishedAt": "2024-03-15T11:00:00.000Z",
  "scrapedAt": "2026-05-21T12:00:00.000Z"
}

---

APIs used

API	Use case	Auth
Algolia HN Search	Search, filtering, comments	None
HN Firebase API	Live feeds (top/new/best/ask/show/job)	None

---

Performance

• 128 MB memory — smallest Apify tier
• ~2–5s for a simple keyword search
• ~10–30s for a 100-item feed run (Firebase requires one request per item)
• No proxies, no headless browsers, no authentication needed

---

Keywords

Hacker News API, HN scraper, tech trends, Ask HN, Show HN, jobs board, startup news, tech community, Hacker News stories, Hacker News comments