Hacker News Scraper — Stories, Comments & Hiring Jobs

Hacker News as clean rows: the front page in real rank order, full-text search that goes past the 1,000-result wall, whole comment trees, user profiles, and the monthly "Who is hiring" threads parsed into structured job records.

No API key. No login. Pick a mode, press start, export CSV or JSON.

---

Three things this does that other HN scrapers don't

1. It gets past the 1,000-result wall

The Algolia index behind Hacker News search refuses to page beyond 1,000 results. It does not say so — page 20 comes back HTTP 200 with an empty list, so a page-based scraper stops at exactly 1,000 rows and reports success. If you asked for a month of stories, you silently got a few days.

This Actor walks a timestamp cursor instead of paging.

Approach	Unique stories retrieved
Page-based (every other HN Actor)	1,000 — hard stop
Cursor walk (this Actor)	6,000 in 6 requests, and it keeps going

2. Its front page is the actual front page

The obvious shortcut is Algolia's front_page tag. Measured, that tag returns 171 items — not the 30 on the front page — sorted by relevance rather than rank, and 51 of the first 100 are YC job ads with points: null and num_comments: null. Half your paid rows come back empty, in the wrong order, with no rank.

This Actor reads Hacker News' own ranked lists, so every row carries a real rank field, 1…N, and a real score. All six lists are available: top, new, best, ask, show, jobs.

3. It turns "Who is hiring" into a jobs database

"Ask HN: Who is hiring?" runs on the first of every month and is one of the best sources of hiring data on the internet — hundreds of companies, written by the people doing the hiring, with contact addresses. Existing scrapers hand you the raw comment text.

This one parses it. Measured on the July 2026 thread: 276 posts, 89.9% resolved into fields, 268 companies, 232 apply links, 250 workplace classifications, 83 contact emails.

Posts that don't match a known format still return their full text — you are never billed for an empty row.

---

What you get

Stories and comments

Column	Example
rank	1 (front-page modes — real HN rank)
title	Elevators
url / domain	link target · acme.com
author	pg
points / numComments	1097 / 262
createdAt	ISO 8601
text	self-post body, HTML stripped and entities decoded
isAskHN / isShowHN	true / false
hnUrl	permalink to the item

Who is hiring / Who wants to be hired

Column	Example
company	WireScreen
title	Senior Software Engineer (Frontend)
location	NYC - hybrid
workplace	REMOTE · HYBRID · ONSITE
employmentType	full-time · contract · internship
salaryMin / salaryMax / salaryCurrency	170000 / 220000 / USD
techTags	["typescript","postgres","aws"]
emails	["jobs@acme.com"] — obfuscated forms decoded
applyUrl / urls	direct link · every link in the post
visa	mentioned · no
text	the full original post, always
commentUrl / threadTitle	permalink · "Ask HN: Who is hiring? (July 2026)"

---

Modes

Mode	What it returns
Search stories	Keyword, author, domain, points, comment count, date range
Front page / ranked lists	Top, new, best, ask, show or jobs — in rank order
Search comments	Full-text across every HN comment
One story + all its comments	The whole tree, flattened, with reply depth
User profile	Karma, bio, and their recent submissions
Who is hiring	Structured jobs from the monthly thread
Who wants to be hired	Structured candidate posts from the companion thread

---

Typical uses

• Recruiting and lead gen. Every company hiring this month, with contact emails, filtered to remote-only or to one technology. Companies hiring are companies spending.
• Salary benchmarking. Pull twelve months of hiring threads and read the bands.
• Trend tracking. Which technologies appear in job posts, month over month.
• Newsletters and aggregators. Schedule the front page daily and publish it.
• Sentiment and market research. Search comments for what people say about a product.
• Feeding an LLM or agent. One row per item, consistent field names, HTML already stripped and entities decoded.

---

A note on "Ask HN"

Hacker News files every text-only submission under "ask" — including "Tell HN" and plain untitled discussion. Neither the ask feed nor the ask_hn tag gives you only posts titled "Ask HN:" (measured: 2/10 and 3/10 respectively).

If you want real Ask HN posts, set Title must contain to Ask HN. It's applied before billing, so you don't pay for the rows it removes. Show HN has no such problem — that feed is clean.

---

Example input

Remote Python jobs with a stated salary, from the latest hiring thread:

{
  "mode": "whoIsHiring",
  "workplace": "REMOTE",
  "tech": "python",
  "withSalaryOnly": true,
  "sortBy": "salary",
  "maxItems": 200
}

Stories about Rust that actually got traction, last 30 days:

{
  "mode": "search",
  "searchQuery": "rust",
  "minPoints": 100,
  "postedWithinDays": 30,
  "sortBy": "points",
  "maxItems": 200
}

Real "Ask HN" posts — see the note above on why titleContains is the selector:

{ "mode": "search", "storyType": "ask_hn", "titleContains": "Ask HN", "maxItems": 100 }

---

Settings that matter

• titleContains — the only reliable way to select genuine "Ask HN:" posts, and it runs before rows are billed, so you don't pay for the ones you'd throw away.
• hiringThreads — 1 gives you the current month. Raise it to build a hiring time series across several months in one run.
• withSalaryOnly / withEmailOnly — the two filters that turn a hiring thread from reading material into a usable list. Both applied before billing.
• minPoints / postedWithinDays — HN search returns everything ever posted by default; these are what make a search query useful.
• allowTypos — off by default. Turn it on for name searches, off for exact technical terms where a fuzzy match just adds noise.
• maxItems — your cost ceiling. Set it deliberately.

---

Reliability

Both sources are official public APIs — the Algolia HN index and the Hacker News Firebase API. There is no HTML parsing and no bot-detection workaround anywhere in this Actor, which is why it doesn't quietly break when a page layout changes.

A 126-assertion live test suite runs against the real APIs before every release, including assertions that the 1,000-result wall is still there, that numeric filters are actually applied rather than silently ignored, and that every extracted email is a valid address.

---

Pricing

Pay per event. You are charged for the run start and for each row returned — nothing else, and platform compute is included free. Filters that can be pushed down to the API (points, comment count, date range) are, so they don't consume your row budget.

---

Questions or a bug?

Open an issue on the Issues tab of this Actor. Include your input JSON and the run ID — that's usually enough to reproduce it immediately.