Hacker News (HN) API Scraper – Who Is Hiring, Tech News API
Pricing
from $0.10 / 1,000 items
Hacker News (HN) API Scraper – Who Is Hiring, Tech News API
Search or browse Hacker News stories, comments, Ask HN, Show HN and Who's Hiring threads via the official Algolia API: full-text Hacker News search, Hacker News jobs threads and startup news feeds. Filter by points, date and exclude keywords. Pay per item. Also a keyword monitoring API for alerts.
Pricing
from $0.10 / 1,000 items
Rating
0.0
(0)
Developer
Fetch Smith
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
4 days ago
Last modified
Categories
Features
Share
Hacker News Scraper
Search or browse Hacker News (stories, comments, Ask HN, Show HN, jobs, and monthly Who's Hiring threads) using the official Algolia Search API — fast, reliable, no scraping fragility. Pay only per item returned.
Use cases
- Track mentions of your product, company or keyword on HN
- Pull the current front page (live top stories) without a separate feed-crawling mode
- Pull the newest Who's Hiring / Who Wants to Be Hired threads for job-market research
- Feed trending tech discussions into AI agents, newsletters or dashboards
- Build datasets of Show HN launches or Ask HN discussions by topic
- Look up a founder's or user's HN karma, bio and account age (
usernames) - Run a scheduled keyword alert that only ever returns new mentions (
watchLabel) - Spot which Show HN launches or technical discussions link to a real, active GitHub repo — with star count, language, last-push date and a computed freshness/maturity classification (
enrichGithubLinks)
Input
| Field | Type | Description |
|---|---|---|
queries | array | Keywords to search. Leave empty to browse by tag/date, or when using author/usernames/domainFilter/excludeKeywords/a points-comments-date filter to browse by those instead of a text query — any of those alone now runs with no keyword filter. The Console form still shows and submits ["apify"] as a safety-net example so a run started with nothing configured at all doesn't silently scan HN's entire ~46M-item history; that example only applies in code when queries AND every other filter are absent (verified live 2026-10-18: a bare run still returns real "apify" rows; {"domainFilter":["simonwillison.net"]}/{"tags":["job"]}/{"usernames":[...]} alone each now return their real, non-empty population instead of 0 rows from the keyword being silently ANDed in). To force an empty keyword even on a bare run, pass an explicit "queries": []. |
tags | array | story, comment, poll, ask_hn, show_hn, job, front_page (default ["story"]). Several tags are OR-ed: ["story","comment"] returns both |
includeComments | boolean | Fetch comment text when tags includes comment |
sortBy | string | relevance or date (newest first) |
minPoints | integer | Only stories with at least this many points — jobs and comments have no points in HN's own data (always null), so combining this with tags: ["job"] or tags: ["comment"] matches nothing, at any threshold |
minComments | integer | Only stories with at least this many comments (find high-engagement discussions) — same caveat: jobs and comments have no comment count of their own |
excludeKeywords | array | Drop any story/comment whose title or text contains any of these words/phrases (case-insensitive) — HN's search has no negative-term syntax, so this is applied client-side after fetching, before you're charged |
domainFilter | array | Only keep stories (and comments on those stories) whose linked URL's domain matches one of these, e.g. ["github.com","arxiv.org"] — matches the domain or any subdomain of it, applied client-side after fetching, before you're charged. A comment is matched on its parent story's domain; items with no resolvable URL (text-only posts, jobs) are dropped when this is set |
maxCommentDepth | integer | Only keep comments this many hops or fewer from their story (1 = a direct reply to the story, 2 = a reply to a top-level comment, etc). HN's search index carries no depth field, so each comment's depth is resolved by walking its parentId chain via HN's own Items API, cached across the run and shared between comments in the same thread — leave empty to return comments at any depth, with no lookup and no commentDepth populated. Stories/jobs/polls are never affected |
author | string | Only items posted by this exact HN username |
postedAfter / postedBefore | string | ISO date bounds |
maxItemsPerQuery | integer | Cap per query. Up to 1000 — HN's own hard limit per query — or up to 50,000 with autoSplitLargeQueries on (see FAQ) |
autoSplitLargeQueries | boolean | Go past HN's 1,000-hit-per-query ceiling: any query that truncates is automatically re-searched in halved date windows until each one fits under the ceiling, so you get the whole match set without working out the slice width yourself (default false) — see below |
maxSplitWindows | integer | With autoSplitLargeQueries: most date windows one query may be halved into, bounding a very broad query's request count and run time (default 24, minimum 3) |
maxResults | integer | Overall cap |
usernames | array | HN usernames to fetch profile data for (karma, about, account age) — a separate lookup, not a story filter. To fetch ONLY profiles with no story search, also set queries and tags to []. |
watchLabel | string | Name a saved search to get only story/comment/job hits new since its last run — see below |
watchChanges | boolean | With watchLabel: also re-deliver an already-seen item when its points or comment count crosses a milestone (default false) — see below |
watchPointMilestones | string | Comma-separated point thresholds for watchChanges (default 25,50,100,250,500,1000,2500,5000; empty disables points alerts) |
watchCommentMilestones | string | Comma-separated comment-count thresholds for watchChanges (default 25,50,100,250,500,1000; empty disables comment alerts) |
enrichGithubLinks | boolean | When a result links to a GitHub repo, add star count, primary language, last-push date and open-issue count (default false) |
webhookUrl | string | Optional. POST a small JSON completion summary (items pushed, hits scanned, dataset ID, watch new/skipped counts, per-query completeness) here when the run finishes — see FAQ |
Output (one item per story/comment)
{"id": "41930211","type": "story","title": "Show HN: FetchSmith – pay-per-use scraper APIs","url": "https://fetchsmith.com","hnUrl": "https://news.ycombinator.com/item?id=41930211","author": "someuser","points": 142,"numComments": 38,"engagementScore": 161,"createdAt": "2026-09-01T12:00:00.000Z","query": "fetchsmith","parentId": null,"commentDepth": null,"githubRepo": null,"githubStars": null,"githubLanguage": null,"githubPushedAt": null,"githubOpenIssues": null,"githubCreatedAt": null,"githubArchived": null,"githubFreshness": null,"githubMaturity": null}
githubRepo/githubStars/githubLanguage/githubPushedAt/githubOpenIssues/githubCreatedAt/githubArchived are only populated when enrichGithubLinks: true and the item actually links to a GitHub repo (common on Show HN); otherwise they stay null. githubFreshness and githubMaturity are computed classifications off those same fields, at no extra request: githubFreshness is "active" (pushed in the last 30 days), "maintained" (31-180 days), "stale" (181-365 days) or "dormant" (over a year), overridden to "archived" whenever GitHub's own archived flag is set regardless of push recency; githubMaturity is "new" (repo under 90 days old), "young" (90 days-1 year), "established" (1-3 years) or "mature" (over 3 years), off the repo's created_at. Both are null whenever the underlying date is null (same cases as above). Thresholds are in classifyGithubFreshness()/classifyGithubMaturity() in src/main.js — a documented bucketing of fields already fetched, not a model call.
parentId is a comment's immediate HN parent (another comment, or the story itself) — null on stories/jobs/polls. commentDepth is that comment's distance from its story (1 = direct reply) — it stays null on every row, comment or not, unless you set maxCommentDepth on this run; set it and the field is resolved for every comment that matches, not just the ones the limit would drop.
engagementScore is points + numComments × 0.5, rounded to 1 decimal — a single number for sorting/filtering a result set by engagement without hand-weighing two columns yourself. It is not decayed by age (unlike HN's own front-page ranking): an age-decayed score collapses to ~0 for anything older than a few days, which would make it useless on relevance search or sortBy:"date" results spanning years — the common case for a keyword search. null on comment rows, which never carry points/numComments.
A usernames lookup returns one row per user, type: "user":
{"id": "pg","type": "user","author": "pg","hnUrl": "https://news.ycombinator.com/user?id=pg","karma": 157316,"about": "Bug fixer.","accountCreatedAt": "2006-10-09T19:41:32.000Z","submissionCount": 15565,"authorInfluenceScore": 100}
submissionCount is the length of the HN API's own submitted array — stories and comments combined, there is no way to split them without fetching every one. authorInfluenceScore is a transparent, documented 0-100 score off the only three signals the public HN user API ever exposes: karma and submissionCount are log-scaled (both have long right tails — a handful of accounts sit orders of magnitude above the median) and capped at 100,000 / 5,000 respectively, account age is linear-scaled and capped at 15 years, and the three combine as karma×0.5 + age×0.2 + submissions×0.3, rounded to the nearest integer. null when karma itself is unavailable. Same "formula, not a model call" spirit as engagementScore above — the exact scaling and weights are in authorInfluenceScore() in src/main.js, not a black box.
Watch mode (watchLabel)
Name a saved search — watchLabel: "my-launch-watch" — and the Actor keeps a per-label record of every story/comment/job hit it has already delivered under that name, so a scheduled run returns only what is new since last time and you are charged for nothing else. The first run on a new label is a free baseline run: it records what already matches and returns zero results. HN's search API never serves more than 1000 hits for one query (see FAQ), so a query with more than 1000 current matches cannot be fully baselined — the un-recorded tail would come back as "new" and be charged on your next run. The baseline run detects this and says so explicitly in its status message, naming each query and its declared match count; narrow those queries (tighter keywords, minPoints, or a postedAfter window) until each fits under 1000, then re-seed under a fresh watchLabel. Every run after that returns only new items. Change queries, tags, author, sortBy, the date range or the point/comment thresholds and the label starts a fresh baseline, instead of dumping everything the old narrower filter excluded as "new". usernames profile lookups are unaffected — they run and are charged normally every time, since a profile snapshot isn't a discrete new item.
Never pin a watch to postedBefore. A newly-posted story/comment/job's own created_at is its post time, so once today's real date passes a fixed postedBefore, nothing new can ever fall on or before that cutoff again — the watch goes permanently quiet while still reporting a healthy "nothing new" every run. There is no escape hatch: postedBefore becomes a server-side Algolia filter (not a client-side check on a sometimes-missing field), so every new hit is blinded without exception. A run in that state logs two warnings and sets watchDeadDateBound: true in the RUN_SUMMARY key-value record and the webhookUrl payload. postedAfter alone is safe to keep in a watched query — it only trims old matches, it never excludes new ones.
A story's id never changes when it takes off — set watchChanges to hear about that too. By default watch mode only ever tells you about brand-new items: a story you were already delivered goes silent forever, even on the day it hits the front page, because HN keeps the same item id for its whole life. Set watchChanges: true and an already-delivered item is re-delivered — charged again, same as a new row — when its points or numComments crosses a milestone it hadn't reached when you last saw it, tagged with _watchChangeType (["pointsMilestone"], ["commentsMilestone"], or both), _watchPrevious ({"points": 99} — where it was last time) and _watchMilestone ({"points": 100} — the rung it cleared).
Why milestones instead of a plain "the number changed" diff: points and comments tick up on nearly every poll of an active story, so a bare diff would re-charge you on essentially every scheduled run for as long as a story stays warm. With milestones each rung pays out at most once per item — a story crawling from 51 to 99 points across twenty runs returns nothing, then returns once at 100. A story's entire life from 0 to 3,000 points costs you 7 extra rows on the default ladder, not 3,000. A score that falls (a flagged or downvoted post) never fires, and a jump straight from 10 to 600 fires once, at the highest rung cleared (500), not four times.
Tune the ladders to your queries with watchPointMilestones / watchCommentMilestones (comma-separated). A Show HN or niche-tag watch rarely passes 100 points, so 25,50,100 is the whole useful range; a front-page watch spends most of its life above 100 and wants a coarser ladder. Leave either field empty to switch that signal off entirely and keep the other. A baseline seeded before this option existed keeps working unchanged — it just has no engagement snapshot for its existing ids yet, so those start reporting milestones from the first run that re-sees each of them, never retroactively. RUN_SUMMARY and the webhook payload split the two: watchNewCount counts first-time items, watchChangedCount counts milestone re-deliveries.
minPoints/minComments and the milestone ladders — WATCH MODE WARNING. minPoints/minComments are lower bounds on the exact same fields the milestone ladders watch. Every item a bounded query can ever return already has points/numComments at or above that bound the moment it first enters the baseline (whether via seeding or first sight on an incremental run), and crossedMilestone() only fires for a rung strictly above that starting value — so any rung at or below the bound can never fire, for any item, for as long as the bound and the rung both stand. This isn't a timing fluke (unlike a story that just happens to already be at 80 points the first time any watch sees it) — it's a guaranteed consequence of the filter. Example: minPoints: 100 with the default ladder 25,50,100,250,500,1000,2500,5000 permanently disables the 25/50/100 rungs; 250 and up still work. The run warns explicitly and names the dead rungs, and they're also surfaced in RUN_SUMMARY/the webhook payload as watchDeadPointRungs/watchDeadCommentRungs. Fix: drop those rungs from watchPointMilestones/watchCommentMilestones, or lower the threshold.
Baseline size cap. The delivered-ids record itself is capped at 20,000 entries; once a label's baseline grows past that, the oldest ids are dropped to bound the record's size. If that happens, a later run will treat those dropped ids as "new" again and charge for them a second time — the run's log and status message warn explicitly when this occurs, and RUN_SUMMARY's baselineTruncated/baselineTruncatedTotal fields give the exact counts. It only matters for a label with a very high cumulative volume of matches over many runs; narrow the query/tags to keep the baseline well under the cap.
Going past the 1,000-hit ceiling (autoSplitLargeQueries)
HN's search API hands over at most 1,000 hits for one query, however many matches it declares — a search for ai declares nearly 2 million and still returns 1,000 (see FAQ). The documented workaround is to split the query into consecutive postedAfter/postedBefore windows yourself, and the hard part is the width: monthly slices keep python under the ceiling (256–447/month) but not ai (3,368 in a single month), so one fixed width either over-splits the quiet stretches or still truncates the busy ones.
Set autoSplitLargeQueries: true and the run does it for you, adaptively. It searches your query normally; if that pass truncates, it halves the date range and re-searches both halves, and keeps halving only the windows that are still truncating until each one fits under the ceiling. Measured live (2026-10-21): python over calendar 2025 declares 3,692 matches and returns exactly 1,000 with the option off. With it on, the same input returned 3,692 of 3,692 — complete — in 9 windows, the run halving 2025 → half-years → quarters only where the volume required it.
Four things worth knowing before you switch it on:
- You are not charged twice. The halves re-scan hits the wider pass already returned (4,000 re-scans in that measured run), and every one is deduped before delivery —
pushedwas 3,692, not 7,692.RUN_SUMMARYreports both:scannedis every hit Algolia handed back,uniqueScannedis the distinct items, and only the latter counts againstmaxItemsPerQueryand decides completeness. - Raise
maxItemsPerQuerytoo, and watch your charge limit. It is clamped to 1,000 without this option (nothing above it was ever reachable) and to 50,000 with it. A query you only ever saw 1,000 rows of can now return its real volume, and every row is charged. - Windows are half-open and non-overlapping (
created_at_i >= from,< to), computed inside your ownpostedAfter/postedBeforerange, so a split can neither lose an item at a boundary nor count one twice. Splitting stops at a 1-hour floor or whenmaxSplitWindowsruns out, and then says so rather than quietly reporting a complete run:RUN_SUMMARYcarriessplitExhausted: "max-split-windows"(raise the budget) or"min-window-width"(the query is too broad to enumerate by date alone — narrow the keywords or addminPoints), alongside the usualhitPaginationCeiling: trueandincompleteReason: "algolia-pagination-ceiling". - A complete run means complete against that run's own scan, not a permanently fixed number. Measured live (2026-10-22): two back-to-back runs of the identical
kubernetesquery over a fixed, fully-past 2024–2026 range returneddeclaredMatches: 1388/delivered: 1389(complete: true) on the first anddeclaredMatches: 1389/delivered: 1388(complete: false,incompleteReason: "scan-short-of-declared") on the second — Algolia's own hit count for a historical query shifted by 1 between the two calls, seconds apart, with identical window splitting (3 windows searched, 2 split) both times. That rules out our own boundary logic as the cause; it is Algolia's index being off by a row or two at the margin, not something this Actor can correct for. If a run reportsscan-short-of-declared, re-running it is more likely to resolve the discrepancy than to repeat it.
Off by default: it re-searches the same query several times over, which costs requests and run time an ordinary capped pull neither needs nor wants.
GitHub enrichment (enrichGithubLinks)
When a story or comment's URL or text links to a GitHub repo, set enrichGithubLinks: true to look it up on GitHub's public API and add githubStars, githubLanguage, githubPushedAt, githubOpenIssues, githubCreatedAt, githubArchived, and two computed classifications, githubFreshness and githubMaturity — handy for triaging Show HN launches or "what got built" threads by real traction and health rather than just HN points. Off by default (adds one extra request per distinct repo found). Bounded to 200 lookups per run against GitHub's unauthenticated 60/hour rate limit; a repo linked by multiple items in the same run is only looked up once. If the limit is hit mid-run, later items still get githubRepo (the match itself is free) but not the star/language/push/classification data.
githubFreshness buckets how recently the repo was pushed to — active (<=30 days), maintained (31-180), stale (181-365), dormant (>365), or archived whenever GitHub's own archive flag is set, which overrides every other bucket since an archived repo won't get a new push regardless of how recent its last one was. githubMaturity buckets the repo's age since creation — new (<=90 days), young (91-365), established (1-3 years), mature (>3 years). Both are a fixed, documented bucketing of fields the same API call already returns (created_at/archived cost nothing extra), not a separate lookup or a model call — see the Output section above for the exact thresholds and source link.
Comment rows check the parent story's link first. A comment has no URL of its own — HN attaches the story's url to it — so on a tags: ["comment"] search, githubRepo reports the story's linked repo whenever the story links to GitHub, even if the comment's own text names a different repo. Only when the story has no GitHub link (or no link at all) does the comment's own text get scanned. Verified live: a comment on "Modern ClojureScript" (storyUrl: https://github.com/magomimmo/modern-cljs) that itself links to two unrelated repos (omcljs/om, reagent-project/reagent) came back with githubRepo: "magomimmo/modern-cljs" — the story's repo, not either repo the comment actually names. If you want the repo(s) a comment itself links to, read them out of its text field rather than relying on githubRepo when the parent story also has a GitHub link. How often this matters, measured on 500 live comment hits (2026-10-20): 50 had a GitHub repo in story_url; of the 33 of those whose own text also linked a repo, 30 linked a different one — and a further 191 of the 500 linked a repo their parent story did not. Full method and numbers: a Hacker News comment has no URL of its own.
Pricing
result — charged per item returned. Empty queries and failed pages are free. A watchLabel baseline run always returns 0 rows and is charged nothing. GitHub enrichment adds no separate charge.
The niche's biggest listing by lifetime users is epctex (202 users as of 2026-11-06, epctex/hackernews-scraper) — but what is dormant about it is the product, not its traffic: its build is still version 0.0 and unchanged since 2021, while it did pick up 16 new users in the last 30 days (an earlier version of this paragraph said 0, true when written on 2026-10-02 and no longer true — so read it as a stale codebase that still draws installs on name recognition, not as an abandoned listing). It rented for a flat monthly fee until 2026-10-01, when Apify's rental-pricing deprecation converted it to $0.0003 per result plus a $0.00005 start fee — 1.5x our Free-tier rate and 3x our Gold rate, with a start fee we don't charge. Its 7 inputs are startUrls-driven (paste HN list/item URLs), with a comment-hierarchy toggle, a page-range cap and an extendOutputFunction; no keyword search, no tag filter, no thresholds, no date window, no user lookups, no watch mode, no webhook. Verified live 2026-10-02.
The actively-growing leader is gentle_cloud (243 users as of 2026-11-06, 63 of them in the last 30 days — still the fastest-growing listing in this niche; pricing re-verified live 2026-10-16 — gentle_cloud/hacker-news-scraper), charging the same $0.0002 per result as we do on the Free tier, more than us on Bronze ($0.0002 vs our $0.00017), $0.0015 per result on Silver (11x our $0.00013 there), and the same $0.0001 as us on Gold and above — we are not cheaper than them at the top three tiers, we match them. Their input surface, read off their current build (0.0.1, unchanged since March 2026), is 7 fields: a mode selector (top/new/best/ask/show/job feeds, keyword search, and a user mode that returns one username's submitted stories), relevance/date sort, a max-results cap, top-level comments per story, and an HTTP timeout. No minPoints/minComments thresholds, no excludeKeywords, no postedAfter/postedBefore date windows, no multi-query batching, no comment-only search, no GitHub-link enrichment, no profile fields (their user mode returns a user's stories, not their karma/about text/account age), no watch mode with milestone re-alerts, and no completion webhook. Verified live 2026-10-03.
Two rivals ship point/comment thresholds and a date window. automation-lab (33 users as of 2026-11-03, automation-lab/hackernews-scraper) charges $0.001 per run start plus $0.00115 per story on the Free tier: a 100-story run bills $0.116 there against $0.02 here, and we charge nothing to start a run. It has no comment search, no user lookups, no excludeKeywords, no multi-query batching, no watch mode and no webhook. The more capable one is constructive_calm (29 users as of 2026-11-09, constructive_calm/hacker-news-scraper, 15 inputs): it charges a $0.01 Actor-start fee plus $0.0004 per story (2x our Free rate) — but only $0.00015 per comment, below our flat $0.0002 on the Free tier. On a comment-heavy run that start fee is what decides it: their $0.01 + $0.00015/comment crosses our $0.0002/comment at about 200 comments, so for comment-only pulls larger than that they are genuinely cheaper than us on the Free tier, and we only win below that size. From Silver down ($0.00013 and lower) we are cheaper at every run size. shahidirfan (56 users, shahidirfan/hacker-news-data-scraper) is $0.0009 per result plus a $0.00005 start fee, 4.5x our Free rate, with only 3 inputs. Verified live 2026-10-02.
The most feature-rich rival in this niche, never named here before, is ryanclinton/hackernews-search (172 users, re-verified live 2026-11-06 — was 131 when this paragraph was first written, found by bin/check-claim-coverage: the bold markdown right after the handle hid this claim from check-competitor-claims' USERS regex, now fixed — 35 of them in the last 30 days, still more lifetime users than automation-lab, constructive_calm and shahidirfan combined (33 + 28 + 58 = 119, all three re-fetched live 2026-11-06 rather than carried over), and still the fastest-growing rival here after gentle_cloud's 63). Its 31-field input schema goes past ours in several places: a 0-100 author-influence score computed from karma/age/submission count — we closed this gap as of 2026-10-22: usernames lookups now return authorInfluenceScore, our own transparent, documented formula off the same three public signals (see "User profile lookups" below); a GitHub-link "correlation" mode that classifies a linked repo's freshness and maturity — we closed this gap as of 2026-10-22: enrichGithubLinks now returns githubFreshness/githubMaturity, our own transparent bucketing of the push-date/archived/created-at fields the same GitHub API call already returns (see "GitHub enrichment" below), sentiment/theme/risk heuristics on results, a two-window period-compare mode, and — the one gap worth calling out against our own FAQ above — it auto-splits a query past Algolia's 1,000-hit ceiling into date-bucketed sub-runs for you (autoSplitLargeQueries/maxSplitRuns). That gap is closed as of 2026-10-21: our own autoSplitLargeQueries halves the date range adaptively until each window clears the ceiling (see above), verified live returning 3,692 of 3,692 declared matches on a query that stops at 1,000 without it. Theirs still costs 9x-25x ours per row. It is also the dearest rival priced in this README: $0.005 per item on Free down to $0.0009 on Platinum/Diamond plus a $0.00005 start fee — 9x to 25x our own $0.0002-$0.0001 range, so none of the above comes free. Verified live 2026-10-03 (input schema read off build 1.1.6).
Never named here before, sian.agency/hacker-news-scraper (11 users, 3 of them in the last 30 days, 31 runs in the last 30 days) covers the same general surface we do — front-page stories, full comment threads, Ask/Show HN, jobs, polls, search and user profiles — split into two charge events: a cheap OverviewItemExtracted row ($0.006 Free down to $0.0013 Diamond) and a dearer DetailItemExtracted row ($0.011 Free down to $0.0025 Diamond), plus a $0.05 Actor-start fee (down to $0.005 from Bronze) that we don't charge. Even its cheapest event at its cheapest tier is 6.5x our own $0.0002 Free rate, and 13x our $0.0001 Gold+ floor — dearer than us at every tier and every event, before the start fee. This sweep also ruled out eight more small listings with >=3 users. Five were the same shape as us and uniformly dearer; two of those — crawlerbros/hacker-news-scraper, which had been 4 users, and crawlerbros/hackernews-scraper, which had been 5 users, both adding a $0.005 Actor-start fee on top of $0.001-$0.005/result — are gone from the Store as of 2026-10-09 (both confirmed 404, a removal not a rename). The other three are still live and still dearer: scrapeworks/hacker-news-search (6 users, count re-verified live 2026-11-06) is a flat $0.005/result plus a $0.00005 start fee; cloud9_ai/hackernews-scraper (5 users) is a flat $0.002/result; fresh_cliff/hacker-news-api-scraper (4 users) tiers $0.0021-$0.003/result plus a $0.007-$0.01 start fee — all 10x-50x our range. Three are a different product shape, not a head-on rival: nexgendata/news-mcp-server (18 users) and nexgendata/job-market-mcp-server (13 users) are per-tool-call MCP servers at $0.02-$0.05/call bundling general news or job-market data, not per-row HN scrapers; scrapemint/hn-lead-monitor (15 users, 116 runs in the last 30 days, the most active small listing found this sweep) charges $0.015 per matched keyword lead — a filtered alert product, not raw story/comment/user rows. Verified live 2026-10-04; user and run counts re-verified 2026-10-25.
For the Who's Hiring use case specifically, logiover/hacker-news-who-is-hiring-scraper (67 users, user count re-verified 2026-10-18) is dedicated to that one thread type (jobs, salary, tech stack extraction) at $0.0035/result on Free down to $0.00199 Gold+ — pricier than our flat rate at every tier and narrower in scope (job threads only, no keyword/story/comment search), not a general-purpose rival. Two more Who's Hiring specialists, never named here before: parseforge/hn-whoishiring-scraper (18 users) parses postings into role/company/salary/remote-flag/tech-stack rows at $0.02/result on Free down to $0.015 Gold+ — dearer than us at every tier, and dearer than logiover too. bikram07/hn-who-is-hiring is on Apify's FREE pricing model — $0 per result, cheaper than every paid rival in this niche including us (the 2026-10-07 sweep below found fourteen more $0 listings that it ties rather than beats) — it filters by keyword/remote-only, pulls any past month, and has an "alert mode for new listings only" (its own version of watch mode, scoped to this one thread type); neither specialist exposes our broader story/comment/user-profile/GitHub-enrichment surface. A fourth Who's Hiring specialist, never named here before, getascraper/hn-hiring-scraper (5 users, 3 of them in the last 30 days — actively growing), charges $0.00089/result on Free down to $0.00067 Gold+ plus a $0.00005 start fee — cheaper than logiover and parseforge but still 6.7x-8.9x our own flat rate at every tier, and still dearer than bikram07's FREE model. Verified live 2026-10-04. Three more listings this sweep surfaced are checked and not a threat: mrbridge/latest-news-mcp-server (108 users) and miccho27/trends-aggregator (63 users) both bundle Hacker News as one of many sources alongside global news, crypto, weather, Reddit and Google Trends at $0.005-$0.008/item — a different product shape, not a head-on HN search tool — and nexgendata/hacker-news-scraper (51 users) outputs aggregate trend analytics (topic/domain breakdowns, top authors) at $0.05/record rather than per-item rows, also a different shape and far dearer. Never named here before, benthepythondev/hacker-news-intelligence (28 users, tied with automation-lab for 9th by users in this niche) pulls Top/New/Best/Ask/Show/Job feeds with a minScore/minComments/timeRange/keywords filter set and a top-5-comments option, plus a marketed "AI engagement score" that is actually a documented, transparent formula (upvotes + comment count + recency + story-type weighting) rather than a model call — no comment-only search, no user lookups, no GitHub enrichment, no watch mode, no webhook. It is also the dearest head-on HN scraper found in this niche: $0.018 per result on Free down to $0.0126 on Diamond, 90x to 126x our own $0.0002-$0.0001 range. Verified live 2026-10-03. (The 2026-10-07 sweep below priced dearer listings still — scrapemint's multi-source radar pipelines run $0.14-$0.18 per row — but those bundle GitHub, Reddit, Stack Overflow or 8-K filings with HN and are not per-row HN scrapers.)
Never named here before, onescales/hacker-news-data (6 users as of 2026-11-06, 4 of them in the last 30 days, 3 bookmarks and 2 five-star reviews — the only reviews or bookmarks found on any rival in this niche sweep, all three figures re-verified live 2026-11-06) is on Apify's FREE pricing model, $0 per run at any volume, a genuine undercut of our entire tier ladder. Its scope is narrower than ours: 7 inputs (a category feed — Top/New/Best/Ask/Show/Jobs — or a date-range scan, plus a nested-comment depth/replies-per-comment setting and a pagination token), no keyword search, no tag-only comment search, no point/comment thresholds, no excludeKeywords, no GitHub enrichment, no user-profile lookups, no watch mode, no webhook. spiky_pepperoni/hacker-news-scraper (3 users, 1 new in the last 30 days) is also on the FREE model — $0 per run — with only 4 inputs (queries, tags, sortByDate, maxItems): keyword search across stories/comments only, no thresholds, no date window, no user lookups, no GitHub enrichment, no watch mode. Both are real $0 substitutes for a plain keyword-or-feed pull, joining bikram07's FREE Who's Hiring model as rivals we cannot undercut on price by definition — the case for us against any of these three is feature surface, not cost. Verified live 2026-10-05.
A fifth Who's Hiring specialist, never named here before: nomad-agent/hackernews-scraper charges a flat $0.0001 per job result at every tier, Free through Diamond, plus a one-time $0.00005 Actor-start fee per run. Against our own flat per-item rate (which tapers $0.0002 Free -> $0.0001 Gold+), its flat $0.0001 undercuts our Free/Bronze/Silver tiers on any run size, ties our Gold+ floor, and its $0.00005 start fee then makes it marginally dearer than us at Gold and above — by a constant $0.00005/run regardless of volume, the mirror image of the constructive_calm crossover above, just a much smaller gap. It filters by keyword, location, remote-only, job type and posting age, with repeat-posting suppression; scope is Who's Hiring threads only, the same limitation as logiover/parseforge/getascraper above, and it is still dearer than bikram07's $0 FREE model. Verified live 2026-10-05.
This sweep also checked 12 more small listings at >=3 users, all confirmed dearer at every tier: two more Who's Hiring specialists — seemuapps/hn-who-is-hiring-scraper ($0.002/job + $0.00005 start) and nexgendata/hn-whos-hiring-scraper ($0.05/row + $0.01 start, the dearest Who's Hiring listing found in this niche) — plus chimerical_quicklime/hackernews-whos-hiring-scraper (flat $0.00625/row + $0.005 start, matching its own title's "$6.25/1k" claim); and eight general-shape clones, all 5x-160x our range: cryptosignals/hackernews-scraper (flat $0.005), viralanalyzer/hackernews-intelligence ($0.03->$0.00567 tiered), gio21/hackernews-scraper (flat $0.005), coder_zoro/hacker-news-scraper ($0.00499->$0.00299 + start), hipersoft/hacker-news-scraper ($0.004->$0.002/item plus a separate $0.001 api-request event plus start), seemuapps/hn-search-scraper (flat $0.001 + start), andok/hackernews-scraper ($0.001->$0.00014 per row but a tiered $0.01->$0.0014 Actor-start fee, the dearest start fee found in this niche), and parseforge/hacker-news-scraper ($0.01599->$0.01199 + a $0.005 init fee). One more, agency-shift/hackernews-jobs-ask-scraper (3u), bills only a flat $0.05 Actor-start fee with no per-row event at all — cheaper than us only past roughly 250-500 rows/run depending on tier, and its row-count-per-run is undisclosed so that crossover is unverified either way. Ten further >=3-user matches were ruled out of scope and not priced — a multi-board 8-source job aggregator, a general news-finder monitor, a resume/candidate-sourcing tool, a GitHub+HN launch-radar pipeline, a trend aggregator, two top-stories-only feeds, two cross-site buyer-intent/brand-monitor pipelines, and a broader tech-news aggregator — none a substitute for our story/comment/job/user-profile surface at any price. Verified live 2026-10-05.
Tenth sweep (2026-10-07), the one that moved our pricing position the most: this niche's cheap tail has grown, and we are no longer the cheapest per-row listing in it. Re-swept to 308 listings seen / 275 matching the niche / 237 of them not yet named above, and live-priced all 237 (0 unresolvable), reading each one's tiered block tier-by-tier rather than collapsing it to a single headline rate. Results, in order of how much they matter:
- Two paid rivals now undercut us at every tier with no start fee to claw it back.
myagizm/hackernews-scrapercharges a flat $0.00008 per dataset item — 2.5x under our $0.0002 Free rate and still 20% under our $0.0001 Gold+ floor — for keyword search over stories, comments and feed sections returning authors, links, points, comment counts and timestamps, i.e. the same shape we sell.quodlibetical_buffalo/hacker-news-mcpcharges a flat $0.00007 per search result, cheaper still; it is packaged as an MCP server for AI agents but bills per row off the same Algolia search, so it is a real substitute, not a different shape. Neither was named here before. - One more beats every tier of ours past the first row or two:
prince_gabriel/fresh-hn-feedbills $0.00005/row plus a $0.00005 Actor-start fee, so it crosses under us at roughly the first row at Free and the second at Gold+; its scope is one freshness-ranked feed, not search, comments, jobs or user profiles. - One has a non-monotonic ladder that beats us everywhere except Silver:
maximedupre/hacker-news-scraperis $0.00015 Free and Bronze, $0.00035 Silver, $0.00005 Gold/Platinum/Diamond — cheaper than us at four of our six tiers and 2.7x dearer at Silver. Feeds, topic, user, date and listing pulls with optional top-level comments; no comment-only search, no GitHub enrichment, no watch mode. - Partial undercutters.
scrapesignal_labs/hacker-news-search-scraperis a flat $0.0001/row + $0.00005 start (its title advertises "$0.10/1K", which its live record matches exactly): under us at Free, Bronze and Silver, then it ties our Gold+ floor and the start fee makes it marginally dearer — the same mirror shape asnomad-agentabove.luminar/hackernews-scraper-monitor(delisted -- the Apify API returned 404 for this Actor on 2026-11-07) tiered $0.000175 Free → $0.00008 Gold+ but added a $0.002 per-collection-checked fee, so it only beat us past roughly 80 rows at Free and 100 at Gold+.hridayrungta/hackernews-scraperis a flat $0.00019 story/comment rate, 5% under our Free tier and dearer than us from Bronze down — note its own description advertises "$0.40 per 1,000 rows" while its live record bills $0.19 per 1,000, so its marketing overstates its own price.bovi/hacker-news-scraper(delisted -- the Apify API returned 404 for this Actor on 2026-11-07) undercut only our Free tier and was dearer than us at Bronze and below ($0.00019 → $0.0001805 + $0.00005 start).delectable_incubator/my-actor-1(branded "Low-cost", $0.00019 Free → $0.00015 Gold+ + $0.00005 start) also undercuts only our Free tier and is dearer than us at Bronze and below — the same "low-cost"-branded owner whose headline prices this fleet has caught overstating cheapness before. - One genuine volume crossover — forecast landed, re-verified live 2026-10-16:
fetch_cat/hacker-news-search-scraper's scheduled 2026-10-09 pricing change is now in effect. It switched from a flat $0.0000277/result to a tiered rate — $0.0000318 (Free), $0.0000277 (Bronze), $0.0000216 (Silver), $0.0000166 (Gold and up) — behind the same $0.005 Actor-start fee. Still dearer than us on small runs and cheaper per-row at every tier (6-12x under ours), but the crossover is no longer one number: roughly 30 rows at Free, 35 at Bronze, 46 at Silver, 60 at Gold and up. - Fourteen listings currently bill $0 (Apify FREE model, or no pricing record at all — which bills nothing either way), against the three this README named before. Six are head-on general-purpose HN search/scrape products:
convertfleetdotonline/hacker-news-scraper,flyingnimbus/hackernews-intelligence-scraper,ahampton83/hackernews-scraper,your_scraper_guy/hacker-news-scraper-stories-search-jobs,orinity/hackernews-scoutandsociable_scissors/hn-discussion-monitor. Two are narrower:scrupulous_waterbird_m4w/hn-launches(Show/Launch HN only) andm_ctim/hn-whos-hiring-parser(Who's Hiring only — and it has a PAY_PER_EVENT entry scheduled for 2026-10-11, so its $0 is about to end). Six are a different product shape, not substitutes:toronto_777/ai-product-launch-monitor,gabrielaxy/brand-mention-aggregator,abidchaudhry/brand-mention-monitor,darknezz/sentiment-analyzer(delisted -- the Apify API returned 404 for this Actor on 2026-11-07),joeslade/hn-developer-sentiment-mcpandapiagent/ai-agent-news-aggregator. A $0 listing beats every tier we have by definition; what we sell against it is reliability, the filter surface, watch mode and GitHub enrichment, not price. - Checked and NOT claimed as cheaper, because we cannot verify it from outside. Three listings' cheapest per-row event is Apify's platform-default
apify-default-dataset-itemat $0.00001 while they also carry a dearer named row event —reverberant_equality/hn-top-stories($0.00001 vs its ownstory-result$0.002 and a $0.005 search-start),thebee/TechContentScraper($0.00001 vscontent-row$0.001) andlistless_adzuki/tech-news-monitor($0.00001 vshn-result$0.003). Which event actually fires per row is not visible in the public record, so they are logged as unresolved rather than counted either way. Same treatment forzentrafoundry/public-data-product-launch-sprintandzentrafoundry/ugly-website-ai-agent-connector, whose records carry several $0.0001 deliverable events alongside a $0.01result-delivered— the first one's own description states "$0.01 / story", so its headline $0.0001 events are not what a buyer pays.chuckling_yarn/dollar-hunter-multi-sourceis unambiguous by contrast (start fee + dataset item are its only events, so it really is $0.00001/row), but it fetches GitHub, HN and Reddit as one bundled API rather than HN rows with our filter surface. - Two previously unpriced "out of scope" listings, now priced per our own rule that a scope exclusion has to come from the live record and not the title:
variable_nose_u5u/technews-aggregatoris $0.0001/row + $0.00005 start — under our Free, Bronze and Silver tiers — though it aggregates TechCrunch/The Verge/Ars Technica alongside HN rather than selling HN depth;nomad-agent/ml-ai-dev-bundleis a flat $0.0002/row at every tier + $0.00005 start, which ties our Free rate and is dearer than us at every tier below it. New head-on listingfeedforge/hacker-news-scraper($0.0002/row + $0.00005 start) likewise ties our Free tier and is dearer below it.
Everything above is read off each rival's live pricingInfos — verified live 2026-10-07; our own ladder was re-verified live against the platform the same day ($0.0002 Free / $0.00017 Bronze / $0.00013 Silver / $0.0001 Gold+, no start fee, unchanged since 2026-09-26). Rival user counts are deliberately not quoted here: every listing in this sweep sits in the low single digits, where a one-user change swings the number by more than it is worth.
Eleventh sweep (2026-10-08): widened the Store search past the bare "hacker news" phrase to catch the no-space ("hackernews") and abbreviated ("hn") forms that a contiguous two-word phrase match cannot see by construction — the same blind spot already fixed for the Steam and Google Play niches. 304 listings now match (vs 281 on the old 11-term sweep), 21 of them genuinely new. Three are real $0 substitutes, never named here before: thenomadinorbit/hn-scraper (no pricing record at all — top/new/best/ask/show feeds, a head-on general-purpose clone) and two more Who's Hiring specialists on Apify's FREE model, toronto_777/hn-who-is-hiring-leads and vitado_shortcake/hn-remote-jobs-premium (the "premium" in its own name notwithstanding) — bringing the running $0-listing count to twenty. The other 18 were dearer and not a threat: the one that had cleared our informal 3-user bar, carmine_tennis/hn-who-is-hiring-scraper (had been 3u, flat $0.002/job, 10x the Who's Hiring specialists already named above), is now gone from the Store as of 2026-11-09 — confirmed 404 on the direct lookup and on a search=carmine_tennis account-wide query (0 results), a whole-account removal, not a rename; the remaining 17 all sit at 1-2 users and are either flat-rate clones well above our range (solidcode/hackernews-scraper $0.00215→$0.0018 + $0.005 start, tqm/hackernews-job-scraper $0.0015/row, wiggly_book/hn-jobs-finder $0.01/job, xtracto/hackernews-search $0.01/row, yadroo/hackernews-search $0.0005/row, superslowsloth/hacker-news-scraper $0.0004/row + $0.002 start) or too small/dormant (0 users in the last 30 days) to be worth a full write-up. Verified live 2026-10-08.
Twelfth sweep (2026-10-09): dropped the >=3-user floor and scanned every charge event x every tier on the full unnamed tail, not just the headline-selected event. Own price re-verified live first: unchanged, $0.0002 Free / $0.00017 Bronze / $0.00013 Silver / $0.0001 Gold+, no start fee. Re-swept to 403 listings seen / 306 matching the niche / 226 of them not yet named above (197 with no owner mention, 29 whose owner appears elsewhere in this README's prose), and live-priced all 226 (0 unresolvable). A raw per-event, per-tier scan against our own ladder returned 8 hits across 8 handles, but 6 were a single false-positive shape and not real: each has a tiny apify-actor-start/actor-start event (a one-time per-run fee, same as every other listing's start fee in this niche) that happens to lack the platform's isOneTimeEvent flag in its public record, so a scan that only trusts that flag misreads the run-start charge as a cheap per-row rate — their real named row event (mention-found, item, mention-observed, company-signal, or the tiered apify-actor-start on an MCP server whose only other event is the same start fee) is 2.5x-75x our rate, confirming headline_price already read these 6 correctly. The 2 real hits: supermiojo/hacker-news-scraper (1u, Firebase-API-based, no tiers) bills a flat $0.0001/row with no other per-row event — under our Free/Bronze/Silver tiers, ties our Gold+ floor exactly, never named here before. reverberant_equality/mcp-hacker-news (1u, an MCP server, same owner as the already-logged reverberant_equality/hn-top-stories above) carries the same unresolvable shape already ruled out in the "Checked and NOT claimed" paragraph below — a $0.00001 apify-default-dataset-item row charge alongside a dearer, not-obviously-alternative tool-call event ($0.005) — so it is left unresolved rather than counted either way, same treatment as its sibling listing. Verified live 2026-10-09.
Thirteenth sweep (2026-10-16): full-tail resweep, no >=3-user floor. Own price re-verified live first: unchanged since 2026-09-26, $0.0002 Free / $0.00017 Bronze / $0.00013 Silver / $0.0001 Gold+, no start fee. Re-swept to 523 listings seen / 344 matching the niche / 264 not yet named above (235 with no owner mention, 29 whose owner appears elsewhere in this README's prose) — up from the twelfth sweep's 403/306/226, a jump consistent with the pre-cycle-1780 limit=100 Store-search truncation bug surfacing rows that were always there rather than real niche growth (same pattern confirmed on remote-jobs-scraper and uk-find-a-tender-scraper), not re-derived here in full. Live-priced all 264 (0 unresolvable), every charge event x every tier, not just the headline-selected event. Three more genuine $0 substitutes, never named here before — bringing the running $0-listing count from twenty to 23: naimurhasanrwd/hackernews-stories-scraper (2u, Firebase-API top/new/best/ask/show/job lists, no keyword search), scraper_guru/ycombinitor (2u, front-page title/score/author/age/url only, despite its name this is a plain HN front-page clone, not a YC-companies product), and emperor1978/hn-top-stories-scraper (2u, top/new/best/ask/show/job feeds with points/comments filtering and pagination). All three are narrower than the 20 already logged — no keyword search across stories+comments, no user-profile lookups, no GitHub enrichment, no watch mode — so the case against them is still feature surface, not price. tidyharvest/hacker-news-search-scraper (2u) bills a flat $0.00019/item (matches its own "$0.19 per 1,000 items" claim) — under our Free tier only, dearer than Bronze down, the same shape already logged for hridayrungta above. Three more ruled out as false matches or a different shape, not substitutes: dami_studio/research-mcp-server (2u) is a 14-source MCP bundle (arXiv, GitHub, Stack Overflow, HN, Wikipedia, SEC and more) whose only declared event is a $0.0000125 Actor-start fee — its own description says "each call bills the scraper it runs", so there is no per-row HN rate to compare at all; lsso/ai-tool-momentum-index (2u) is a GitHub/Hugging Face trending tool that folds HN buzz in as one minor signal among several, not an HN scraper; wiggly_book/security-news-digest (2u as of 2026-11-06, 1u at the time of that sweep) scrapes "The Hacker News" (thehackernews.com, a cybersecurity news site) rather than Hacker News (news.ycombinator.com) — a name collision from the term sweep, not a real competitor. The remaining 257 of 264 were confirmed dearer or a different product shape (job-board aggregators, brand-monitor/buyer-intent pipelines, trend-radar bundles), none undercutting any tier. Verified live 2026-10-16.
Fourteenth sweep (2026-10-23): full-tail resweep, no >=3-user floor — and this one found the first paid rival since 2026-10-07 that undercuts our whole ladder with no start fee, plus four flat-per-run listings that beat us on any run past roughly 100 rows. Own price re-verified live first: unchanged since 2026-09-26, $0.0002 Free / $0.00017 Bronze / $0.00013 Silver / $0.0001 Gold+, no start fee (bin/check-own-price-freshness, 25 Actors, 0 flags). Re-swept to 577 listings seen / 384 matching the niche / 296 not yet named above (264 with no owner mention, 32 whose owner appears elsewhere in this README's prose) — real growth of 40 matching listings over the thirteenth sweep's 523/344/264. Live-priced all 264 unnamed-owner listings (0 unresolvable), every charge event x every tier, compared tier-by-matched-tier against our own ladder rather than against our Free rate alone. Verified live 2026-10-23.
- One paid rival now undercuts every tier of ours with no start fee:
v0iddo/hn-algolia-search(4u) bills $0.0001 Free → $0.00009 Bronze → $0.00008 Silver → $0.00007 Gold+ per story-or-comment extracted, i.e. half our Free rate and still 30% under our $0.0001 Gold+ floor, for keyword search over stories and comments off the same Algolia API we use. Never named here before; it joinsmyagizmandquodlibetical_buffaloin the every-tier-cheaper group. - Two more genuine $0 substitutes, never named here before — bringing the running $0-listing count from 23 to 25:
fit_melon/hacker-news-search-scraper(2u, on Apify's FREE model, keyword search returning stories and comments — now gone from the Store as of 2026-10-30, confirmed 404 on its Actor record, a removal not a rename, so it is no longer a substitute and the running count below drops it) andrbbscrap/fast-hacker-news-scraper(renamed, see correction immediately below). Both are narrower than our surface (no user-profile lookups, no GitHub enrichment, no watch mode, no point/comment thresholds), so the case against them is feature surface, not price — a $0 listing beats every tier we have by definition. Correction, re-verified live 2026-11-08: therbbscraphalf of that is no longer true on either count. The handlerbbscrap/fast-hacker-news-scraperno longer resolves (the Apify API returned 404 for it on 2026-11-08) — it was renamed, not removed: the same listing is live asrbbscrap/hacker-news-scraper(still 4 users, samecreatedAtof 2026-10-13, title "Hacker News Scraper — Stories, Comments & News"), and it is no longer a $0 listing — a pricing record took effect on 2026-11-08 charging $0.0008 per story scraped, which is 4x our $0.0002 Free rate and 8x our $0.0001 Gold+ floor, i.e. dearer than us at every tier. So the case against it is now price and feature surface. Note this handle is distinct from the long-delistedcrawlerbros/hacker-news-scrapernamed earlier in this section. - One is cheaper than us on comment pulls at every tier:
manageable_gully/hacker-news-scraper(2u) splits its billing into a$0.0002flatitemevent (ties our Free rate, dearer than us from Bronze down) and a$0.00005flatcommentevent, behind a $0.0003 Actor-start fee. On comment-heavy pulls that comment rate crosses under us at about 2 comments on Free and 6 on Gold+, so for any real comment pull it is genuinely cheaper than us at every tier; on story rows it is not. It covers the official HN and HN Search APIs with an only-new alert mode, but no user-profile lookups, no GitHub enrichment and noexcludeKeywords. - Four flat-per-run listings beat us past roughly 100 rows.
second_coming/hn-intelligence(1u),second_coming/tech-launch-tracker(2u) andsecond_coming/brand-mention-monitor(4u as of 2026-11-06) each bill a single one-time $0.02scanevent and no per-row event at all, so a whole run costs $0.02 however many rows it returns — crossing under our per-row bill at about 100 rows on Free and 200 on Gold+.brilliant_gum/hackernews-insights-scraper(3u) is the same shape with a caveat worth stating: all four of its events (apify-actor-start$0.01,story-scraped$0.005,comment-scraped$0.001,user-scraped$0.005) carryisOneTimeEvent: truein its live record, so as published a full run is capped at $0.021 — a ~105-row Free / ~210-row Gold+ crossover — though per-event prices that high on a per-row-sounding event name suggest the flags may be mis-set on its end. Either reading is disclosed here: capped, it is cheaper than us on large runs; uncapped, it is 25x our rate. Onlyhn-intelligenceis a head-on HN product (Algolia keyword search with velocity/digest modes);tech-launch-trackercovers Product Hunt + Show HN launches and both brand monitors are multi-source OSINT pipelines, so their scope is narrower or wider than ours rather than equal — but the crossover is real at any scope overlap. - Five partial undercutters, all cheaper than us at our Free tier and most of them dearer from Silver down.
bepolabs/hacker-news(2u) is a flat $0.00016/item with no start fee: 20% under our Free rate, 6% under Bronze, dearer than us at Silver and below.rowcraft/hacker-news-scraper(2u) tiers $0.00018 → $0.00012 behind a $0.00005 start fee, so it is cheaper than us past ~3 rows on Free and ~5 on Bronze, and dearer at Silver and below.best_apis/hacker-news-scraper(2u) is a flat $0.00015/item behind a comparatively large $0.0005 start fee — cheaper than us past ~10 rows on Free and ~25 on Bronze, dearer below that.sophisticated_occasion/hacker-news-search-monitor(2u) is a flat $0.00019/story, 5% under our Free tier only.themineworks/hacker-news-scraper(2u) is a flat $0.00018/item behind a $0.003 start fee, the narrowest crossover found this sweep: cheaper than us only on Free-tier runs past about 150 rows, and never at Bronze or below. - Three sibling listings tie our ladder almost exactly and beat it at one tier:
deriverge/hacker-news-scraper,deriverge/hacker-news-searchandderiverge/hacker-news-comments-scraper(2u each) all price theircommentevent at $0.0002 / $0.00016 / $0.00013 / $0.0001 — identical to our ladder at five of six tiers and 6% cheaper at Bronze. Theirstoryevent ($0.0005 → $0.00025) is 2.5x dearer than ours at every tier, so on story pulls we win and on comment pulls it is a wash outside Bronze. - Checked and ruled out, with the reason from the live record rather than the title. Dearer at every matched tier:
charlesmarkette/hacker-news-scraper-stories-comments-search($0.0002 → $0.00014 — it ties our Free rate and is dearer at every paid tier, the shape a Free-rate-only comparison misreads as an undercut),arman-bd/hacker-news-scraper($0.001 → $0.00018),recordsdata/hackernews-search-scraper($0.01-$0.012/row, 50x),grit-77/hacker-news-scraper($0.00025/item),datalantern/hacker-news-search($0.0003/story + $0.001 start),enezli/hacker-news-scraper($0.002/result, 10x),transparent_meteorite/developer-signals($0.001/hn-item) andintelscrape/brand-mentions-monitor($0.001/mention). Not the same site:arman-bd/lobsters-stories-scraperscrapes lobste.rs, a term-sweep name adjacency rather than a rival. And the twelfth sweep's false-positive shape recurred on eight listings this time (pinkish_armada,reaperoak,leadgenguy,joey001,jedii_123,quarry,angaba92/hiring-intent-signals-scraper,khadinakbar): each carries a $0.00005apify-actor-startevent whose public record lacks the platform'sisOneTimeEventflag, so a scan that trusts only that flag reads a once-per-run fee as a cheap per-row rate, while their real named row events are 5x-75x ours. The remaining ~235 of 264 were confirmed dearer or a different product shape (job-board aggregators, brand-monitor and buyer-intent pipelines, trend-radar bundles), none undercutting any tier. Verified live 2026-10-23.
Fifteenth sweep (2026-10-30): full-tail resweep, no >=3-user floor — the first sweep in this niche whose tier comparison is done rung-for-rung in code rather than by hand, and the quietest result yet: no new paid rival undercuts our whole ladder. Own price re-verified live first: unchanged since 2026-09-26, $0.0002 Free / $0.00017 Bronze / $0.00013 Silver / $0.0001 Gold+, no start fee, single result event. Re-swept to 646 listings seen / 427 matching the niche / 317 not yet named above (274 with no owner mention, 43 whose owner appears elsewhere in this README's prose) — real growth of 43 matching listings over the fourteenth sweep's 577/384/296. Live-priced all 274 unnamed-owner listings (0 unresolvable, 0 that failed to resolve a per-row ladder), every charge event x every tier. Verified live 2026-10-30.
- Method change, and it removed two thirds of the raw hits. The fourteenth sweep's script compared every rival rung against our Free rate alone and the resulting false positives were caught by hand ("compared tier-by-matched-tier" above). This sweep does that comparison in code (
bin/_batch_price_hn3.py, rung-for-rung throughcheck-price-superiority'stier_price_at), keeping the old scalar read alongside purely to measure it: the scalar read flags 6 every-tier undercutters, the rung-for-rung read confirms 2 — a 3x over-flag. All four it dropped are flat-rate listings that sit just under our Free rung and are dearer than us at every paid rung, so they are disclosed below as Free-tier-only undercutters rather than as whole-ladder ones. An Apify pricing tier is the buyer's plan, not the seller's discount, so the rung a buyer actually pays is the only honest comparison. - Two more genuine $0 substitutes, never named here before — bringing the running $0-listing count from 25 to 27:
tomstudio/hacker-news-search(2u, "stories, comments, Ask HN, Show HN and jobs by keyword, author, points and date, or export full comment threads", same Algolia API we use) andauthoritative_glen/hackernews-trends-stories-leads(2u, front page / Show HN / Ask HN / Who is Hiring / keyword search). Neither has a pricing record at all, so both bill nothing; we checked each for a scheduled futurePAY_PER_EVENTentry as well (the trap astartedAt <= nowfilter hides) and neither has one. Both are head-on general-purpose HN scrapers by coverage, and at $0 nothing we charge can beat them on price — the honest comparison is filter surface, watch mode, GitHub enrichment and run reliability. - Four Free-tier-only undercutters, every one of them dearer than us from Bronze down. Each bills a single flat rate with no tier ladder, which is why they cross our ladder exactly once:
fecklesspleb/hacker-news-scraper(2u) at a flat $0.00015/item with no start fee is 25% under our Free rate and 12% under Bronze, then dearer at Silver and below (its own description says "$0.15 per 1,000 items");examsens.ai/hacker-news-scraper(2u) at a flat $0.00018/item-returned plus a $0.00005 start fee is under our Free rung only;digivibetech/hacker-news-search-monitor(2u) at a flat $0.00019/story with no start fee, likewise Free only; andmicro-tool-factory/hacker-news-comments(2u) at a flat $0.00019/result-item plus a $0.00005 start fee, Free only and comment-search-scoped. A buyer on Apify's Free plan pays less at all four; a buyer on any paid plan pays more. - Two listings tie our Free rate exactly and are dearer at every paid rung, recorded because this is the precise shape the old scalar comparison misread:
groundtruth/hacker-news-search(3u, flat $0.0002/row plus a $0.00005 start fee — the widest-coverage new entrant this sweep, with search, front page, the top/best/new/ask/show lists and full comment trees) andpappy-dev/hackernews-discussions(2u, flat $0.0002/item-scraped, plus four $0.05 one-time analysis events on top). - Checked and ruled out, with the reason from the live record rather than the title. No listing in the 274 bills a flat run fee with no per-row event this time, so the fourteenth sweep's
second_coming-shaped crossover class produced no new members. The remaining ~262 resolved a per-row ladder that is dearer than ours at every matched rung — the cheap end of that remainder isrod_analytics/hackernews-scraper($0.0005 → $0.00035/item, 3.5x our Gold+ floor) andleftwinglautus,lafuan,hata1234,muzafferkadir(all $0.0005/row) — or is a different product shape (job-board aggregators such ascancap/remote-jobs-actor, brand-monitor and mention pipelines, trend-radar bundles, AI-summary products likescrapers_latat $0.008/result + $0.012/summary). Worth one note on the dearest end: four listings bill $0.01-$0.05 per row (samwise.agency,lulzasaur,moving_beacon-owner1, bothtrisertlistings at $0.05), 50x-250x our Free rate, which is a reminder that this niche's price spread is three orders of magnitude wide and a title tells you nothing about where a listing sits in it. - One rival named on 2026-10-23 has since been delisted, and this sweep could not have found that out.
fit_melon/hacker-news-search-scraper, one of the two new $0 substitutes the fourteenth sweep named, now returns 404 on its Actor record (re-checked 2026-10-30). A niche sweep enumerates what is in the Store, so it is structurally blind to a rival leaving it; the delisting was caught by the separate pass that re-reads every handle this README already names. The $0 count above is therefore 27 found and 26 live, and the fourteenth sweep's entry has been annotated rather than rewritten. Verified live 2026-10-30.
Sixteenth sweep (2026-11-06): full-tail resweep, no >=3-user floor — the quietest price result in this niche's history (one every-rung undercutter out of 297), and a churn verdict that reversed itself under checking. Own price re-verified live first: unchanged since 2026-09-26, $0.0002 Free / $0.00017 Bronze / $0.00013 Silver / $0.0001 Gold+, no start fee, single result event. Re-swept to 708 listings seen / 469 matching the niche / 351 not yet named above (297 with no owner mention, 54 whose owner appears elsewhere in this README's prose) — real growth of 42 matching listings over the fifteenth sweep's 646/427/317. Live-priced all 297 unnamed-owner listings rung-for-rung (0 unresolvable, 0 that failed to resolve a per-row ladder, and — unusually — all 297 on PAY_PER_EVENT, so no listing had to be hand-read out of the ambiguous bucket this time).
- One every-rung undercutter, and it is not a clean undercut.
plotpointtech/hacker-news-scraper(2u) tiers $0.00015/item on Free down to $0.000093 on Gold+ — under our ladder at every matched rung — but adds a $0.00005 Actor-start fee we do not charge, so on small runs it is dearer in total. The scalar-vs-our-Free read would have reported 5 every-tier undercutters here; rung-for-rung confirms 1, a 5x over-flag, wider than the fifteenth sweep's 3x. - Four Free-tier-only (or Free-through-Silver) undercutters, all at 2 users, all carrying a start fee.
itayozer9/hacker-news-searchandfluyeporlaweb/hacker-news-scraperbill a flat $0.00015/row (under our Free and Bronze rungs, dearer from Silver down) plus a $0.00005 start fee;my_agent_c/hacker-news-stories-scraperandactor-forge/hacker-news-scraperbill a flat $0.0001/row (under Free, Bronze and Silver, tying our Gold+ floor) with start fees of $0.005 and $0.00005 respectively —my_agent_c's start fee alone equals 50 rows at our Gold+ rate, so it only undercuts us on large runs. - The churn story this sweep's cohort diff told was false, and nine seconds of re-fetching killed it. 22 of the fifteenth sweep's 274 handles are absent from this sweep's cohort, including all six of its undercutters — a set difference that reads as "the undercutting field turned over completely in 336 cycles." Re-fetching all 22 one at a time: 21 are live,
isPublictrue,isDeprecatedfalse, and 20 of those 21 are priced byte-identically to 336 cycles ago. Onlywearefluent/hn-searchis confirmed 404 — delisted, and re-verified still 404 on 2026-11-08, when a check of the wholewearefluentaccount found every one of its handles named anywhere in our docs dead andGET /v2/store?search=wearefluentreturningtotal: 0, so this is a whole-account removal rather than one dropped listing. They left the cohort because Apify Store search recall drifts between runs and because this README now names some of them — not because they left the Store. The honest total undercutter count is therefore this sweep's 5 plus 6 verified-live absentees, not 5:digivibetech/hacker-news-search-monitor($0.00019, Free only),micro-tool-factory/hacker-news-comments($0.00019, Free only),examsens.ai/hacker-news-scraper($0.00018, Free only),fecklesspleb/hacker-news-scraper($0.00015, Free and Bronze), and two with no pricing record at all —tomstudio/hacker-news-searchandauthoritative_glen/hackernews-trends-stories-leads(both 2u,pricingInfos: null, i.e. $0 at any volume, confirmed directly on the Actor record rather than inferred from an empty ladder) — which beat every tier we have by definition and keep the running $0-listing count at 27 found / 26 live. - One real price change among the 22, and it shadows our ladder without undercutting it.
groundtruth/hacker-news-search(3u) moved from a flat $0.0002/row + $0.00005 start fee to a tier ladder — $0.0002 Free / $0.00018 Bronze / $0.00016 Silver / $0.00014 Gold+ — which ties us exactly at Free and sits above us at every paid rung (our $0.00017/$0.00013/$0.0001). This is the second listing in this niche to adopt a ladder of our shape at a higher level; it is a pricing-position signal, not a threat. - What the tail actually looks like, which matters more than any single rival. Across the 297: the largest has 10 lifetime users, the median has 2, and 227 of 297 sit at exactly 2 — and all five of this sweep's undercutters are 2-user listings. The niche's cheap end is made almost entirely of near-zero-traction listings, so the competitive pressure here is on discovery and feature surface rather than on price. Six listings now advertise a per-1,000 rate in their Store title ($0.15/1K to $4.00/1K), a visible-pricing tactic worth watching. Verified live 2026-11-06.
Seventeenth sweep (2026-11-12): full-tail resweep, no >=3-user floor. Own price re-verified live first: unchanged since 2026-09-26, $0.0002 Free / $0.00017 Bronze / $0.00013 Silver / $0.0001 Gold+, no start fee, single result event. Re-swept to 762 listings seen / 498 matching the niche / 377 not yet named above (322 with no owner mention, 55 whose owner appears elsewhere in this README's prose) — real growth of 29 matching listings over the sixteenth sweep's 708/469/351. Live-priced all 322 unnamed-owner listings rung-for-rung (2 ambiguous multi-event listings hand-read below rather than dropped). Verified live 2026-11-12.
- Three more genuine $0 substitutes, never named here before — bringing the running $0-listing count from 27 found/26 live to 30 found/29 live:
extas_corp/hackernews-scraper(2u, FREE model, a head-on general-purpose clone — stories, comments, keyword search, jobs, user profiles and Who is hiring threads via the official Firebase/Algolia APIs — the broadest-scoped $0 listing found in this niche to date, though it still lacks GitHub enrichment, watch-milestone alerts and our depth/domain filters) and two narrower Show HN specialists,exuberantal/hackernews-showhn-launches-scraper(2u, FREE model, capped at 25 rows/run) andadaptive_labs/hacker-news-launches-monitor(2u, FREE model, Show HN plus front-page launches with keyword filter and only-new mode). All three resolved live (not delisted). - One new partial undercutter with broad scope, cheaper at three of our six tiers.
tidelane/hacker-news(2u) ladders itshn-itemevent $0.00015 Free → $0.000135 Bronze → $0.00012 Silver → $0.000105 Gold+ (a separate, 10x-dearerjob-postevent for Who's Hiring rows), behind a $0.00005 start fee — under our ladder at Free, Bronze and Silver, dearer from Gold down. Covers keyword/domain/date/points search, front page, New/Best/Ask/Show HN, brand-mention only-new mode and Who's Hiring parsing into structured job rows (company, role, location, remote, salary) — close to our own scope, no GitHub enrichment or user-profile lookups. - Four Free-and-Bronze-only undercutters, all flat-rate, no ladder.
rowfeed/hacker-news-api(2u) bills a flat $0.00015/row behind an unusually large $0.001 start fee (under our Free/Bronze, dearer Silver down);lanternlab318/hacker-news(1u) bills $0.000149/story behind a $0.0005 start fee, feed-only (no search);topapi/hacker-news-scraper(1u) bills $0.00014/item with no start fee on that event, broad scope (search, feeds, comment trees, user profiles, Who is hiring/wants-to-be-hired, keyword trends);informed_almandine_vzb/hacker-news-search(2u) bills $0.00001 at Free but jumps to a flat $0.001 at every paid tier (Apify's unconfigured-paid-tier default, not a real ladder) — so it undercuts us by 95% on Free-tier runs only and is 5-10x dearer everywhere else. - Two ambiguous multi-event listings, resolved without needing the primary-event flag.
flets_sv/hacker-news-monitor(2u) carriesitem($0.00025) andhiring-job($0.001), both already above our $0.0002 Free ceiling, so it is dearer than us at every tier regardless of which one is primary;datasetworks/news-trend-monitor(1u) carriessearch-term($0.01) andarticle($0.001), same shape, same conclusion. Both ruled out cleanly rather than dropped from the count. - The rest of the 322 were confirmed dearer or a different product shape (job-board aggregators, brand-monitor/buyer-intent pipelines, trend-radar and OSINT bundles, MCP wrappers with no standalone per-row HN rate) — none undercutting any tier. Verified live 2026-11-12.
Eighteenth sweep (2026-11-13): an incremental sweep, deliberately not a full resweep — and the first one in this niche's history to find nothing new at all. Own price re-verified live first: unchanged since 2026-09-26, $0.0002 Free / $0.00017 Bronze / $0.00013 Silver / $0.0001 Gold+, no start fee, single result event. Re-swept the Store to 772 listings seen / 502 matching the niche / 372 not yet named above (316 with no owner mention, 56 whose owner appears elsewhere in this README's prose). Against the seventeenth sweep's 762/498/377 one day earlier that is +4 matching listings, and the unnamed count fell by 5 — because that sweep named three new $0 substitutes and five new undercutters here, moving them out of the unnamed tail. Because the full 322-listing rung-for-rung sweep was only a day old, this one priced the difference rather than repeating it: exactly 6 of today's 316 no-owner-mention listings were absent from the seventeenth sweep's priced set, and all 6 were live-priced rung-for-rung (0 unresolvable, 0 ambiguous). All 6 are dearer than our ladder at every one of our six rungs, and every one of them also charges an Actor-start fee we do not — lx112/hacker-news-scraper (2u, flat $0.0003/item + $0.00005 start), consequential_programmer/hacker-news-scraper (2u, flat $0.0005/story + $0.00005 start), lowcountrytech/hacker-news-scraper (2u, flat $0.0005/story + $0.00005 start), felixgoldwin/hacker-news-scraper (1u, flat $0.001/row + $0.005 start), dabooti/hacker-news-scraper (1u, flat $0.0003/item + $0.00005 start) and tenfoldfleet/brand-mentions-monitor (1u, flat $0.0025/mention + $0.00005 start, a Bluesky/HN/Mastodon brand monitor rather than an HN scraper). So the undercutter list above is unchanged by this sweep: nothing added, nothing retracted. The whole unnamed tail now tops out at 10 users (lokki/news-finder-monitor, a multi-source news bundle at $0.005/item on Free down to $0.0035 Gold+, behind a $0.02 start fee — dearer than us at every rung) — every one of this niche's ten largest listings by user count is already named and priced somewhere above. Verified live 2026-11-13.
What we do not claim. Our input surface is still not a strict superset of every rival's in this niche. We closed one of the two gaps named here at cycle 1116 (domainFilter, added 2026-10-02): it restricts stories, and comments on those stories, to a given set of link domains the same way constructive_calm's does. We've now closed half of the other: constructive_calm ships maxCommentDepth and flattenComments; we added maxCommentDepth at cycle 1127 (resolved by walking each comment's parent chain, see Output above), but not flattenComments — our comments come from a keyword/tag search across the whole site, not a full per-story crawl, so we never hold a complete thread to flatten in the first place, only the scattered subset that matched. constructive_calm also exposes user profiles like we do, so profile lookups are no longer unique to us here. ryanclinton goes further still: sentiment/trend/compare heuristics remain a gap in our own input surface, not theirs, and genuinely heuristic-shaped — we have not found an honest non-LLM version of it, and CLAUDE.md rule 2 blocks any inference-shaped shortcut, so it stays open rather than forced with a weak proxy. Its other three advantages are now closed: the author-influence score (authorInfluenceScore, 2026-10-22, see "User profile lookups" below), automatic date-bucketed splitting past the 1,000-hit ceiling (autoSplitLargeQueries, 2026-10-21, see above), and GitHub-link freshness/maturity classification (githubFreshness/githubMaturity, 2026-10-22, see "GitHub enrichment" above). We are not the cheapest per-row listing in this niche, at any tier. As of the 2026-10-30 sweep above: myagizm ($0.00008/item), quodlibetical_buffalo ($0.00007/search result) and v0iddo ($0.0001 Free → $0.00007 Gold+) undercut every tier of our ladder outright with no start fee, manageable_gully undercuts every tier of ours on comment rows ($0.00005/comment), maximedupre undercuts four of our six, prince_gabriel undercuts all six past the first row or two, four listings (second_coming/hn-intelligence, second_coming/tech-launch-tracker, second_coming/brand-mention-monitor, brilliant_gum/hackernews-insights-scraper) bill a flat ~$0.02 per run with no per-row event and so beat us past roughly 100-210 rows, and 29 listings in total bill $0 (30 found, less one delisted) — the three this README named before (bikram07 for Who's Hiring, onescales and spiky_pepperoni for general story/comment pulls) plus fourteen more found on 2026-10-07 (six of them head-on general-purpose HN scrapers) plus three more found on 2026-10-08's wider term sweep (thenomadinorbit, toronto_777, vitado_shortcake) plus three found on 2026-10-16 (naimurhasanrwd, scraper_guru, emperor1978) plus two found on 2026-10-23 (fit_melon — now delisted, 404 on 2026-10-30, and dropped from the count — and rbbscrap, re-verified live) plus two found on 2026-10-30 (tomstudio, authoritative_glen — see above) plus three found on 2026-11-12 (extas_corp/hackernews-scraper, exuberantal/hackernews-showhn-launches-scraper, adaptive_labs/hacker-news-launches-monitor — see the seventeenth sweep above). One caveat on that newest trio, visible on the live records rather than in the sweep's own tally: extas_corp is the broadest-scoped $0 listing in this niche to date but it is a recent switch, not a long-standing free listing — its pricing record was PAY_PER_EVENT at $0.0003/result from 2026-11-04 and only became FREE on 2026-11-12, hours before we first priced it, so treat it as the least settled entry on this list. Also see the Gold-tier match with gentle_cloud, the ~200-comment crossover with constructive_calm, the small Gold+ gap with nomad-agent on Who's Hiring pulls, the ~29-69-row crossover with fetch_cat, the Bronze-tier tie-and-6%-undercut by the three deriverge listings, and the partial undercuts by scrapesignal_labs, luminar, hridayrungta, bovi, delectable_incubator, variable_nose_u5u, tidyharvest, bepolabs, rowcraft, best_apis, sophisticated_occasion, themineworks, fecklesspleb, examsens.ai, digivibetech and micro-tool-factory, all above. Two later sweeps have added to that undercutter list, and nothing on it is retracted here. From the sixteenth sweep (2026-11-06): plotpointtech/hacker-news-scraper sits under our ladder at all six rungs ($0.00015 Free → $0.000093 Gold+) but adds a $0.00005 Actor-start fee we do not charge, so it is dearer than us in total on small runs. From the seventeenth (2026-11-12): tidelane/hacker-news undercuts three of our six rungs on its hn-item event ($0.00015 Free → $0.000105 Gold+, behind a $0.00005 start fee, and its job-post event is 10x dearer); rowfeed/hacker-news-api ($0.00015/row), topapi/hacker-news-scraper ($0.00014/item) and lanternlab318/hacker-news ($0.000149/story) each undercut our Free and Bronze rungs only, on their per-row event, and are dearer from Silver down; and informed_almandine_vzb/hacker-news-search undercuts our Free rung by 95% ($0.00001/result) while charging a flat $0.001 at every paid rung, 5-10x dearer than us there. What we compete on against the cheap tail is the filter surface, watch mode, GitHub enrichment and reliability — not being the lowest price on the page. The 2026-10-30 figures in this paragraph were verified live that day; every handle it adds from the sixteenth and seventeenth sweeps, and all three of the newest $0 listings, were re-verified live rung-for-rung on 2026-11-12.
Tips
- Want the current front page? Set
queriesto[]andtagsto["front_page"]— no keyword needed, returns the stories on HN's front page right now (verified againsthacker-news.firebaseio.com/v0/topstories.json, refreshes on the same cadence as the live site). - For the current "Who is hiring?" thread: set
queriesto["Ask HN: Who is hiring"],tags: ["story"],sortBy: "date",maxItemsPerQuery: 1to find the thread, or usetags: ["comment"]withpostedAfterset to the 1st of the month to pull all replies. - Use
sortBy: "date"for a live monitoring feed of new mentions of your keyword. - Use
authorto pull everything a specific user has posted (e.g. track a founder's HN activity), orminCommentsto surface only high-engagement discussions. - Use
excludeKeywordsto keep a broad query narrow, e.g.queries: ["rust"]+excludeKeywords: ["cryptocurrency"]keeps Rust-the-language discussions and drops mentions of an unrelated Rust-named crypto project. - Leaving both
queriesandtagsempty is not a "browse everything" mode — it's rejected (with a warning, no charge) rather than matching HN's entire 46M+ item history by relevance. Always set at least one tag (e.g.["story"],["front_page"]) or a search query.
FAQ
What happens if postedAfter/postedBefore isn't a real date? As of v0.1.91 the run fails immediately — "postedAfter" is not a parseable date (got "...") . Use ISO 8601, e.g. "2026-01-01". — before any search runs. Before this fix, an unparseable value became NaN inside the Algolia query string Algolia answers that with its own HTTP 400, but this Actor didn't surface it: the run read back an empty response and reported the existing "nothing matched, and that is the whole truth about this query" empty/complete:true status — a plausible-looking zero-row result that actually meant your date was malformed, not that nothing matched. 0 rows means 0 charge either way, but you now get told why. An inverted window (postedAfter on or after postedBefore) also now fails fast with a "swap them" message.
I asked for a common word and got exactly 100 (or 1000) rows — is that everything? No, and the run now tells you so instead of leaving you to guess. Two separate ceilings apply. Yours: maxItemsPerQuery (default 100). HN's: Algolia's HN index never returns more than 1000 hits for a single query, whatever you set — a search for ai declares nearly 2 million matches (measured: nbHits: 1,978,072) and will still only ever hand over 1000 of them; page 1000 returns Algolia's own pagination-limit message, not an error. When either ceiling truncates a query, the run's status message names the query with both numbers ("20 scanned of 60108 declared"), and the per-query detail lands in a machine-readable RUN_SUMMARY record. To actually get more than 1000, set autoSplitLargeQueries: true and the run halves the date range for you until every window fits under the ceiling (see Going past the 1,000-hit ceiling above — measured live at 3,692 of 3,692 matches on a query that otherwise stops at 1,000). To do it by hand instead, split one broad query into consecutive postedAfter/postedBefore windows — verified live to reconstruct the true total exactly with no gap or double-count (twelve monthly slices of python summed to the same 3,995 as one over-the-ceiling yearly query) — but the window width has to match how hot the query is: monthly stays under the ceiling for python (256–447/month) but not for ai (3,368 in one month alone, needing ~weekly slices instead). Read hitPaginationCeiling back and halve the window whenever it's true, rather than guessing a fixed width. See the 1,000-hit ceiling guide for the full live measurement.
How do I check completeness from code, without reading the log? Every run writes a RUN_SUMMARY key-value record — GET https://api.apify.com/v2/actor-runs/<runId>/key-value-store/records/RUN_SUMMARY — with a top-level complete boolean and one entry per query: declaredMatches (what HN says exists), scanned, delivered, filteredOut, status, complete, incompleteReason and hitPaginationCeiling. status and complete are deliberately separate: a query can be "ok" and truncated, which is the case worth catching. The top-level timeBudgetExceeded boolean tells you whether the run stopped itself short of the platform run timeout rather than hitting maxResults/the charge limit — when it's true, any query still marked notReached was never searched because the clock ran out, not because nothing was left to check; narrow the input (fewer queries, lower maxItemsPerQuery, or enrichGithubLinks:false) or run them separately. When watchLabel is set, baselineTruncated/baselineTruncatedTotal report whether this run's baseline record hit its 20,000-id size cap (see "Baseline size cap" above) — non-null and non-zero means some already-delivered ids will be re-charged on a future run.
{ "complete": false, "pushed": 25, "queriesIncomplete": 2, "queriesNotReached": 1,"queries": [{ "query": "rust", "declaredMatches": 60108, "scanned": 20, "delivered": 20, "filteredOut": 0,"status": "ok", "complete": false, "incompleteReason": "max-items-per-query", "hitPaginationCeiling": false, "scanCap": 20 },{ "query": "kubernetes", "declaredMatches": null, "scanned": 0, "delivered": 0, "filteredOut": 0,"status": "notReached", "complete": false, "incompleteReason": null, "hitPaginationCeiling": false, "scanCap": null }] }
incompleteReason is one of algolia-pagination-ceiling, max-items-per-query, seed-cap, max-results, charge-limit, scan-short-of-declared, request-failed or time-budget (the run stopped itself short of the platform run timeout, not a real data ceiling). The same fields are posted to webhookUrl if you set one.
I passed 5 queries but rows only came back for the first two. Three possible causes, and the status message and RUN_SUMMARY tell you which: the run hit maxResults (or your pay-per-event charge limit) partway through, or — if timeBudgetExceeded is true in RUN_SUMMARY — the run was approaching the platform run timeout and stopped itself before the clock ran out, which is a different problem with a different fix (narrow the input rather than raising maxResults). Either way the remaining queries are never searched — they appear in RUN_SUMMARY with status: "notReached" and are named in the status message, so "we never looked there" can't be mistaken for "nothing matched there".
Why did my run return 0 items with status SUCCEEDED? The status message distinguishes "no matches for this query/tags/date/points filter" from "the Algolia request failed" — check it before assuming the query is wrong.
What happens if Algolia's API has a transient blip mid-run? Each page request is retried up to 3 times on a connection-level failure (measured on remote-jobs-scraper's upstream boards at roughly 1 fresh request in 4 for HTTP/2 faults, 2026-09-21) before that query is given up on and named in the status message — a single blip no longer silently truncates a query to whatever page it reached.
Can I ask for two content types at once, e.g. stories and comments? Yes — tags: ["story","comment"] returns both. The tags you list are OR-ed with each other, and author is AND-ed on top of that group, so author: "pg" + tags: ["story","comment"] returns pg's stories and pg's comments. (Underneath, HN's Algolia index treats a bare comma as AND, so story,comment would match nothing at all — the Actor wraps your tags in the OR form for you. See the guide linked below.)
Can I combine author and minComments? Yes, filters are ANDed together, e.g. author: "pg" + minComments: 50 returns only that user's high-engagement posts.
I set minPoints or minComments with tags: ["job"] or tags: ["comment"] and got 0 rows — why? Verified live: job and comment hits carry points: null and no comment count at all in HN's own Algolia data, so those filters can never match a job or comment item, however low the threshold. The status message says so explicitly and names which of the two filters is set. Drop the filter, or search tags: ["story"] instead if you want a points/comment-count floor.
I have watchChanges on with minPoints: 100 and the default milestone ladder, but never see a 25, 50, or 100 alert — is that a bug? No, and it's not something a longer wait fixes: minPoints/minComments guarantee every item in the baseline already starts at or above that bound, so any milestone rung at or below it can mathematically never be crossed (the run warns and names the dead rungs explicitly; see Watch mode above). Drop those rungs from the ladder or lower the threshold.
What if queries has an accidental duplicate? Deduped automatically — the same story/comment matched by two queries is only pushed (and charged) once per run.
Does this scrape the HN website? No — it uses Algolia's official HN Search API, the same one that powers hn.algolia.com, so there's no scraping fragility to break.
Do I get charged for empty queries? No — only items actually returned to the dataset are charged.
Is there a dedicated "top stories" or "by user" mode? No separate mode needed — tags: ["front_page"] with no query returns the live front page, and author returns everything a given user posted, combinable with any other filter (points, comments, date range) that a fixed mode wouldn't let you apply.
Can I look up a user's karma or bio, not just their posts? Yes — put their username(s) in usernames. It's a separate lookup (via HN's official Firebase API) that returns one row per user with karma, about, accountCreatedAt, submissionCount and a computed authorInfluenceScore, independent of author/queries. To fetch profiles only, with no story search mixed in, set queries and tags to [] too.
What if a username in usernames doesn't exist? It's skipped with no charge; the run's status message names which username(s) weren't found.
How do I get a scheduled alert for new mentions of a keyword, not the same matches every time? Set watchLabel (see above). The first run is a free baseline (0 results); schedule the same input to run again later and it returns only hits it has never delivered under that label before.
I set watchLabel and usernames together — why did the profile row still show up on the baseline run? By design. watchLabel only tracks story/comment/job search hits (they have a stable id to dedupe on); a usernames profile is a snapshot, not a discrete new item, so it is charged every run regardless of watch mode.
My watch says "nothing new" every single run — is it broken? Check whether you set postedBefore. A watch combined with a fixed upper date bound is a dead end: every brand-new story/comment/job is posted "now", so once the calendar passes your cutoff no new item can ever satisfy it again, and the run honestly reports zero new rows forever. Such a run now logs two warnings and sets watchDeadDateBound: true in the RUN_SUMMARY key-value record and the webhookUrl payload so a scheduled watch can alert on it instead of trusting the quiet. Fix it by removing postedBefore from the watched query — keep postedAfter if you want a floor — or drop watchLabel if you genuinely wanted a one-off historical window rather than an ongoing watch. (Changing the filter starts a fresh free baseline, since postedBefore is part of the watch fingerprint.)
How is webhookUrl different from Apify's own platform webhooks? Apify's platform webhooks are configured separately per Task/Actor via the Console or the Webhooks API — useful if you already live in the Apify Console, but extra setup if you're calling this Actor's API directly and just want a completion ping. webhookUrl is a plain input field: set it on the run itself and it POSTs a JSON body (actorRunId, defaultDatasetId, finishedAt, pushed, scanned, and — if watchLabel is set — watchSeeding/watchNewCount/watchSkippedCount) once the run finishes and every item is already pushed and charged. Especially useful with watchLabel on a scheduled keyword alert: your endpoint is told how many new hits landed without polling the dataset, and watchSeeding: true distinguishes "this was the free baseline run" from "your watch is live and quiet" — both report zero new rows otherwise. It's best-effort — a slow or failing webhook only logs a warning, it never fails the run, changes the result set, or affects billing. Note: a run that ends in an unhandled error exits before this block runs, so webhookUrl reports what the run found, not that it happened — pair it with Apify's own run-status webhook if you need failure alerts too.
Why are githubStars/githubLanguage null even though githubRepo is set? Either GitHub's public API had nothing at that path (repo renamed, deleted or private — rare) or the run's 200-lookup budget or GitHub's own unauthenticated rate limit was hit; githubRepo itself is a free regex match against the item's own URL/text, not an API call, so it's always populated when a link is found. githubCreatedAt/githubArchived/githubFreshness/githubMaturity go null in exactly the same cases, since all come from that same API response.
What do githubFreshness/githubMaturity mean? githubFreshness is how recently the repo was pushed to (active/maintained/stale/dormant, or archived if GitHub's own archive flag is set) and githubMaturity is how old the repo is (new/young/established/mature, off its creation date) — see "GitHub enrichment" above for the exact day thresholds.
For a comment, is githubRepo a repo the comment itself links to? Not necessarily — see "Comment rows check the parent story's link first" above. It's the first GitHub repo found checking the story's link, then the comment's own text, then its title, in that order — so a comment discussing repos the story itself doesn't link to only surfaces them when the story has no GitHub link at all.
Why isn't there a "NOT" or negative-term operator in queries? HN's underlying Algolia search doesn't support one — excludeKeywords gets you the same result a different way: it fetches your normal queries/tags match set, then drops (uncharged) any item whose title or text contains one of your excluded words, so you only pay for what's left.
I listed many queries/usernames/excludeKeywords/domainFilter and the status message looks cut off. As of v0.1.23 the status message is capped at 500 chars (Apify's own platform limit) with a trailing ... marker if it had to cut — the full, untruncated detail is always in the run log and RUN_SUMMARY. A long watchLabel is similarly shortened to 60 chars in the status message only; the full label is still used for the baseline record itself.
My watch-mode run FAILED instead of succeeding with zero rows — what happened to my baseline? As of v0.1.24, if every Algolia request for a seeding run fails (upstream outage, not a single blip — those retry automatically), the run ends FAILED rather than silently saving a truncated baseline, which would make every item past the failure point look "new" (and billable) on your next run. The failure message always keeps the "no baseline was saved, nothing was charged" disclosure and the re-run instruction intact — even with several long failing queries and a long watchLabel, that part of the message is reserved out of the 500-char budget and only the upstream error detail gets shortened.
Related guides
Engineering write-ups behind this Actor:
- Hacker News has two public JSON APIs, and neither one needs a key — Firebase (items/users/live lists by id) vs. Algolia (full-text search), measured live, and when to use which
- HN search caps every query at 1,000 hits, no matter what nbHits claims — live proof of the ceiling, and why the date-slicing workaround's window width has to match how hot the query is
- HN's search API: commas mean AND, not OR
- A Hacker News comment has no URL of its own — so GitHub links get attributed to the wrong repo — why
githubRepoon a comment row reports the thread's repo, measured on 500 live comment hits, plus the two extractor traps (gists and legacy*.github.compages) that inflate any count you take yourself - Eight ways an "only new since last run" watch mode silently stops working — how
watchLabelis built and the three-run test that proves a baseline is complete - We nearly charged our own buyers twice for rows they'd already paid for — a capped watch-mode baseline can silently evict old-but-current ids on a high-volume run, re-delivering (and re-billing) rows already paid for.
- Substack, Apple Podcasts, Google News and Hacker News — four free APIs where the first response isn't the finished product — how this Actor's second-step trap compares to the other three content platforms we scrape.
Only publicly available data is collected via HN's official search API. Questions or feature requests: support@fetchsmith.com. Also available as a hosted API at https://fetchsmith.com/tools/hacker-news-scraper
Source code: https://github.com/Fetchsmith/fetchsmith/tree/main/actors/hacker-news-scraper