Hacker News Jobs Scraper | Who Is Hiring Threads
Pricing
from $1.20 / 1,000 results
Turn Hacker News 'Ask HN: Who is hiring?' threads into structured job data - one row per posting, with company, location, remote flag, salary, detected skills and a link to the original comment. Up to 12 months in one run. Direct-from-company postings that never reach a job board.
This was the last of the seven listed actors with no zero-row contract at all, and the seventh
silent zero in the portfolio's history. Measured on the live Store actor 2026-09-10: an impossible
techStackFilter returned SUCCEEDED, [], 7.0s — a buyer's only signal being an empty Output
tab, with no way to tell too-narrow filters from a markup change.
apify-actor-kit (src/kit/, vendored) and added the failOnZeroResults input,
default true — the same contract the other five board actors already had. This actor is now a
registered consumer in apify-actor-kit/sync-to-actors.sh.commentsSeen (what HN served), postingsParsed (what survived
validation), a per-filter rejection breakdown, and a blocked count in failedRequestHandler,
which previously only logged. A zero run now reports filtered_out / blocked / none_usable /
source_empty rather than one green lie.threads.length === 0
called Actor.exit() bare. It now goes through the same contract — HN publishes a hiring thread
every month, so finding none for every month requested is a real failure.Caught by running it, not by reading it. The first cut passed
Dataset.getInfo().itemCount as the pushed count. That value is eventually consistent: a run
that had just pushed 40 rows read back 0, so a perfectly healthy run reported parsed_not_pushed
and FAILED. It was harmless while it only fed a log line and became a bug the moment it decided
whether the run failed. The count is now itemsPushed, the same synchronously-reserved counter that
enforces the spend cap.
⚠️ A value that was only ever logged has no track record. Promoting one to a control decision is a change of contract, not a change of caller.
Two deliberate non-changes, both of which would have been wrong:
includeReplies is off by default, so
counting it would make nearly every zero-row run blame a filter the buyer never set — and
diagnoseZeroRows tests the filter breakdown before every other cause. A skipped reply is simply
not a parsed posting.requested is not wired to maxItems. On the five board actors it names a board ceiling, so
a short set means the board is exhausted. Here the ceiling is however many postings N months of
threads hold — 3 months returns ~750 against a default cap of 1000 every time, which is not a
shortfall worth reporting.Breaking output change. The actor emitted one item per thread, with every posting nested in
jobPostings[]. It now emits one item per posting, with the thread fields denormalized onto each row.
The old shape was defensible on the merits — the thread is the unit HN publishes — and it stopped being defensible the moment the actor had a price. Apify bills per dataset item. The verified August 2026 thread delivered 230 postings as one billable result. There is no price that works in both directions: cover support at 230-postings-per-result and the listing reads as absurd; price per posting and the whole run bills as $0.001.
At the portfolio price of $1.50/1,000 that same run now bills as 230 results = $0.345, against $0.0018 of measured cost.
The billing unit is part of the data model, not a decision downstream of it. This shape was chosen before the actor had a price, and it was correct until the price existed.
includeReplies | default true → false. A reply is a discussion comment, not a vacancy, and it would now bill as a result. It also bypasses techStackFilter and remoteOnly entirely (HN-DOM), so a filtered run was returning unfiltered replies |
maxItems | new, default 1000. Per-result pricing means the row count is the bill, and a 12-month run over ~230-posting threads is a few thousand rows. Enforced by slicing before the push - Apify bills what reaches the dataset, so a cap applied afterwards is a cap the buyer already paid for |
title | removed. Hard-coded null on every row since the actor was written. Advertising a field that is never populated is the same defect as a filter that never narrows |
rawText | removed. It was description.substring(0, 500) - no information the full field lacks, on every row of a 230-row payload |
Added threadTotalComments (was totalComments, at thread level). Every row now carries source,
commentId, commentUrl, company, location, remote, salary, skills, description,
depth, parentCommentId, the four thread* fields and scrapedAt.
Two dataset views added - overview and full - which the flat shape makes possible; a nested
array cannot be rendered as a table.
TQM's own nightly pipeline does not consume this actor. It scrapes the same threads through its
own Python hackernews.py via the Algolia API, and is not registered in apify_ingest.ACTORS.
Checked before the reshape rather than assumed.
tsc --noEmit clean. Closes IMPROVEMENT-PLAN items 1 and 7.
Documentation only. No behavior changes; build 0.0.16 is unmodified.
docs/ARCHITECTURE.md, docs/DEVELOPMENT.md, docs/HN-DOM.md,
docs/IMPROVEMENT-PLAN.md, and CLAUDE.md..actor/dataset_schema.json — the output surface was previously undeclared, and now documents the
nested jobPostings[] structure.includeDescription; the
actual input is includeReplies. Corrected.Platform run on 2026-08-13 (monthsToScrape: 1, includeReplies: false):
| Metric | Value |
|---|---|
| Dataset items | 1 (a thread) |
| Job postings inside it | 230 |
| Thread comments | 350 |
| Cost | $0.0018 |
| Duration | 13 s |
Roughly $0.008 per 1,000 job postings — the cheapest source in the portfolio by an order of magnitude, because one HTTP fetch yields hundreds of postings.
title is hard-coded undefined — advertised, never populated, 0 of 230 postings.hits[0] without verifying the author is whoishiring, that the
title matches exactly, or that the month is right.company extractor's third pattern guesses "first three words if all capitalized" and produced
"Location: Charleston SC" as a company name. Fill rate 219/230, accuracy lower.techStackFilter and remoteOnly are applied only at depth 0, so with the default
includeReplies: true replies bypass filtering entirely.rawText is description.substring(0, 500) — pure duplication, inflating dataset size.width / 40). If HN moves to CSS
indentation every comment silently becomes depth 0, and includeReplies: false stops working.getParentCommentId is O(n²) — ~60,000 depth computations on a 350-comment thread.monthsToScrape.commentId is HN's own identifier — a genuine stable ID rather than a derived slug. The
strongest identity story in the portfolio.All findings are itemized and prioritized in docs/IMPROVEMENT-PLAN.md. The headline item is
flattening the output: it is a prerequisite for pricing the actor and it changes the shape every
other item is written against.
No changelog was kept. Build 0.0.16 is the deployed build as of 2026-08-13; the last source commit
was 2025-11-14.
Note the repo and .actor/actor.json are named hackernews-scraper, while the deployed actor is
moisecristian2/hackernews-job-scraper.