Hacker News Jobs Scraper | Who Is Hiring Threads
Pricing
from $1.20 / 1,000 results
Turn Hacker News 'Ask HN: Who is hiring?' threads into structured job data - one row per posting, with company, location, remote flag, salary, detected skills and a link to the original comment. Up to 12 months in one run. Direct-from-company postings that never reach a job board.
Breaking output change. The actor emitted one item per thread, with every posting nested in
jobPostings[]. It now emits one item per posting, with the thread fields denormalized onto each row.
The old shape was defensible on the merits — the thread is the unit HN publishes — and it stopped being defensible the moment the actor had a price. Apify bills per dataset item. The verified August 2026 thread delivered 230 postings as one billable result. There is no price that works in both directions: cover support at 230-postings-per-result and the listing reads as absurd; price per posting and the whole run bills as $0.001.
At the portfolio price of $1.50/1,000 that same run now bills as 230 results = $0.345, against $0.0018 of measured cost.
The billing unit is part of the data model, not a decision downstream of it. This shape was chosen before the actor had a price, and it was correct until the price existed.
includeReplies | default true → false. A reply is a discussion comment, not a vacancy, and it would now bill as a result. It also bypasses techStackFilter and remoteOnly entirely (HN-DOM), so a filtered run was returning unfiltered replies |
maxItems | new, default 1000. Per-result pricing means the row count is the bill, and a 12-month run over ~230-posting threads is a few thousand rows. Enforced by slicing before the push - Apify bills what reaches the dataset, so a cap applied afterwards is a cap the buyer already paid for |
title | removed. Hard-coded null on every row since the actor was written. Advertising a field that is never populated is the same defect as a filter that never narrows |
rawText | removed. It was description.substring(0, 500) - no information the full field lacks, on every row of a 230-row payload |
Added threadTotalComments (was totalComments, at thread level). Every row now carries source,
commentId, commentUrl, company, location, remote, salary, skills, description,
depth, parentCommentId, the four thread* fields and scrapedAt.
Two dataset views added - overview and full - which the flat shape makes possible; a nested
array cannot be rendered as a table.
TQM's own nightly pipeline does not consume this actor. It scrapes the same threads through its
own Python hackernews.py via the Algolia API, and is not registered in apify_ingest.ACTORS.
Checked before the reshape rather than assumed.
tsc --noEmit clean. Closes IMPROVEMENT-PLAN items 1 and 7.
Documentation only. No behavior changes; build 0.0.16 is unmodified.
docs/ARCHITECTURE.md, docs/DEVELOPMENT.md, docs/HN-DOM.md,
docs/IMPROVEMENT-PLAN.md, and CLAUDE.md..actor/dataset_schema.json — the output surface was previously undeclared, and now documents the
nested jobPostings[] structure.includeDescription; the
actual input is includeReplies. Corrected.Platform run on 2026-08-13 (monthsToScrape: 1, includeReplies: false):
| Metric | Value |
|---|---|
| Dataset items | 1 (a thread) |
| Job postings inside it | 230 |
| Thread comments | 350 |
| Cost | $0.0018 |
| Duration | 13 s |
Roughly $0.008 per 1,000 job postings — the cheapest source in the portfolio by an order of magnitude, because one HTTP fetch yields hundreds of postings.
title is hard-coded undefined — advertised, never populated, 0 of 230 postings.hits[0] without verifying the author is whoishiring, that the
title matches exactly, or that the month is right.company extractor's third pattern guesses "first three words if all capitalized" and produced
"Location: Charleston SC" as a company name. Fill rate 219/230, accuracy lower.techStackFilter and remoteOnly are applied only at depth 0, so with the default
includeReplies: true replies bypass filtering entirely.rawText is description.substring(0, 500) — pure duplication, inflating dataset size.width / 40). If HN moves to CSS
indentation every comment silently becomes depth 0, and includeReplies: false stops working.getParentCommentId is O(n²) — ~60,000 depth computations on a 350-comment thread.monthsToScrape.commentId is HN's own identifier — a genuine stable ID rather than a derived slug. The
strongest identity story in the portfolio.All findings are itemized and prioritized in docs/IMPROVEMENT-PLAN.md. The headline item is
flattening the output: it is a prerequisite for pricing the actor and it changes the shape every
other item is written against.
No changelog was kept. Build 0.0.16 is the deployed build as of 2026-08-13; the last source commit
was 2025-11-14.
Note the repo and .actor/actor.json are named hackernews-scraper, while the deployed actor is
moisecristian2/hackernews-job-scraper.