Jobs Feed API - Deduplicated LinkedIn, Indeed & ATS Jobs
Pricing
from $1.00 / 1,000 unique jobs
Deduplicated job postings from Greenhouse, Lever, Ashby, LinkedIn (public) and Indeed in one normalized schema, with a delta mode that returns only new, changed and removed jobs between runs - guest pages only, no login.
GUEST_NEEDS_MEMORY stops a run before scraping if it has less than 2,048 MB and Indeed or Glassdoor is in scope. Rollback env JOBS_FEED_GUEST_TRANSPORT=http restores the old HTTP path per-run if the browser path regresses. Live Indeed smoke through the browser transport: 45 listed, 14 matched, no challenge, 51 s run, 803 MB peak.datePosted window, up to 30 jobs, snippet descriptions, contributes no removal signal.costTelemetry.perSource now carries browser request/byte accounting (requests, HTML characters, wire bytes, challenges, failures) alongside the existing HTTP accounting, for Indeed and Glassdoor.simhash64 (bit-identical output, reference test), shingle-set hash switched to FNV-1a (membership-only, containment verified numerically equivalent on the calibration corpus). Dedup thresholds untouched.src/http/proxySelect.js, behaviour unchanged), 10K-row memory ceiling, CSV/XLSX export smoke tool (tools/deploy/csv-export-smoke.js).start=10) is a login wall on Indeed (Camoufox spike 2026-08-20), so it is never requested (D18). Was "up to 10 pages" in the 0.6.0 note. Store title shortened to the platform's 63-character cap.Sources research + a compliance fix found by it (docs/specs/2026-08-20-sources-glassdoor-trueup-design.md).
Perf: pipeline throughput ~12x on large runs (5K postings 64.6 s -> 5.2 s) - per-shingle sha1 memoised in simhash64 (bit-identical output) and the shingle-set hash switched to FNV-1a (membership-only; containment verified numerically equivalent on the calibration corpus). Dedup thresholds untouched.
Glassdoor: attempted, paused, hidden. (Owner call 2026-08-20, after the smokes below.) The source is removed from the input schema enum, the store title/description and the README until a working access path exists; the adapter, its tests and the operator kill switch stay in the build so nothing has to be rebuilt when one does. Original note: The adapter is complete and tested, but the 0.6.0 smokes showed Glassdoor serving a Cloudflare managed challenge to every exit reachable from the platform (built-in residential in any country, the auto group, the container egress) on the first request, while a consumer IP is served normally. The operator kill switch JOBS_FEED_DISABLE_GLASSDOOR is set in Console, so selecting glassdoor yields sourceHealth.status: disabled and costs nothing. It is re-enabled when a working path exists (own proxy via proxyConfiguration.proxyUrls, or a browser fallback after its own cost/class decision). Details: docs/deploy-checklist.md.
Indeed search is now robots-compliant (behaviour change). Our search URLs put start last, so Indeed's Allow: /*&start=0& … /*&start=90& never matched and the blanket Disallow: /*&start= applied - every search request, page 1 included, was disallowed. searchUrl() now emits ?q=…&start=N&sort=date (+ l=/fromage=) and pagination stops at the real allowance of 10 pages, down from 30. Consequence for users: up to 100 listings per search term instead of 300, ordered by date rather than relevance. For a delta feed (what changed since the last run) this is close to neutral; for a first backfill it is a genuine reduction. /viewjob is disallowed and is emitted as a link only - we never fetch it.
Robots watch in the daily canary. tools/robots/check.js re-asserts, for glassdoor.com and indeed.com, every access fact our adapters were designed against; it runs in the source-health workflow and fails the run when a rule we depend on moves. Snapshots in docs/robots/; tests/unit/robots.adapters.test.js checks every URL the adapters can construct against them, so CI fails too. Assertion-based, so cosmetic edits stay quiet.
Research outcome: TrueUp / Lenny's Jobs is not built - the only lawful access door is their $1,000/month data licence; there is no unsigned public job surface, and both robots.txt and the Terms exclude automated agents by name.
Glassdoor (opt-in guest source, spec 2026-08-20 S1). First search page per query × location × datePosted window (30-day ceiling when unset), up to 30 jobs each; never paginates and never opens detail pages (both robots-disallowed - asserted in tests against the stored snapshot); snippet descriptions; p10-p90 salary with estimated; easyApplyFlag. Contributes no removal evidence. Kill switch JOBS_FEED_DISABLE_GLASSDOOR. Locations resolve through a small hand-maintained token table (data/glassdoor-locations.json); unknown locations fall back to a United States search filtered by the card location (GLASSDOOR_LOCATION_UNMAPPED). New codes GLASSDOOR_CHALLENGED, GLASSDOOR_SLICE_TRUNCATED, GLASSDOOR_LOCATION_UNMAPPED.
Sources enum order now encodes the basis hierarchy: ATS trio, then Indeed → Glassdoor → LinkedIn.
Feature-gap P0 (frozen spec §3, owner review): location + datePosted - table stakes a jobs product cannot ship without.
location (top-level) and datePosted (any/24h/3d/7d/14d/30d or integer days 1-365) scope the whole run; per-query objects {"q", "location", "datePosted"} override them (spec §3 object queries; console keeps flat fields, the API accepts both).location= + f_TPR=r{s}, Indeed l= + fromage=) and trust the platform's geo resolution; ATS boards filter client-side (location substring over the posting's location text; posted/updated date cutoff, dateless postings kept). datePosted doubles as the cost control - no more paying to re-scan history.scopeFingerprint: filters enter the canonical scope only when set - existing delta states (including the soak store) keep their fingerprint byte-for-byte; scan keys gain |loc:/|age: suffixes only for filtered searches.location + datePosted in "What to search"; proxyConfiguration prefill dropped (the built-in residential default needs no toggle); first-click math updated to the new price grid ($0.28 at 200 rows).docs/specs/2026-08-18-pricing-update-spec-s1-grid.md; tests/ci/pricing-parity.js fails CI on any divergence across pay_per_event.json / events.js / README / input schema. Money table and first-click math updated ($0.70 day 1, ≈ $1.7/month scenario, $0.28 first click).Spec amendments batch 2026-08-18 (docs/specs/2026-08-18-spec-amendments-batch.md): Part A proxy & detail economics + Part B WS6 jobseeker axis.
proxyConfiguration moved to Advanced as the power-user override ("bring your own proxy") - never removed, it is the margin escape hatch.detailFetch input: off (default) / all / selective (+ detailFetchMinScore); the spec object form { minProvisionalScore } is accepted in the API.job-detail-fetch billable event (PLACEHOLDER $0.0020 FREE → $0.0015 GOLD_PLUS): one per full posting page fetched from a guest source, charged once per run ({runId}:job-detail-fetch), never in the default list mode. Price finalized only after costTelemetry measures real detail-page weight through residential (decision-18 margin gate applies independently). Register on the platform BEFORE pushing this build. A charge failure never kills a delivered run: DETAIL_FETCH_CHARGE_FAILED warning, our loss. Report: costTelemetry.detailFetchesCharged.matchProfile: whole-word, case-insensitive keyword scoring with aliases and weights (matchScore = matched weights / total weights × 100, integer); phrase semantics for multi-word keywords. Row fields matchScore, matchedKeywords, unmatchedKeywords, matchBasis (title+snippet list mode vs title+description - the honesty marker). Rows below minMatchScore are not emitted and not billed (report matchFiltered); minMatchScore: null annotates only.emitted: false (insert-if-absent, no snap); a known-but-never-emitted occurrence that later passes emits as unique-job with the same occurrence id and preserved firstSeenAt; a filtered observation of an emitted occurrence suppresses the row without rebaselining (the pending change still emits when it passes). Removal tracking and snaps run only for emitted: true occurrences.detailFetchMinScore); minMatchScore applies to the FINAL score on full text. A fetched page is billed even when the final score then filters the row - you paid to find out (documented in README).applicantsHint (strictly from "Be among the first N applicants" on LinkedIn detail pages), easyApplyFlag (Indeed), earlyApplicantsOnly input filter (filter-before-billing, same as B1).templates/n8n/jobseeker-digest.json (schedule → delta run with matchProfile → route matchScore ≥ 80 to a digest); README "For job seekers" section with the $2-4/mo math.EVENT_ORDER stays row-only.companies[].greenhouseBoard is the console name for the Greenhouse board slug (legacy greenhouseToken still accepted in the API); descriptions no longer say key-value / keywords / tokens. No behaviour change.ats-catalog-builder actor (plan Phase F; public discovery from the yc-oss companies list + explicit seeds, verified against the board APIs): 704 boards (197 Greenhouse, 427 Ashby, 80 Lever; 23.3K postings), 188 with a verified board ↔ company domain link; unverified boards carry no domain and are named from the board itself (label collisions like a YC "Pulse" vs "Pulse Healthcare" stay apart). Catalog schema: domain nullable, domainVerified; the resolver answers only from verified entries; ATS adapters take the catalog company name when the board has none (Lever/Ashby). tools/catalog/pull.js syncs catalog-latest into the build; builder scheduled weekly (Sunday 03:00 UTC).tools/health/publish.js aggregates the owner canary tasks (source-health-canary 2×/day GH + LinkedIn, removals-soak-canary daily) into docs/health/source-health.{md,json} and a public gist (anonymous read verified); README links it; .github/workflows/source-health.yml refreshes daily once APIFY_TOKEN / GIST_TOKEN secrets are set.seedFromDataset migration kit (УТП spec WS4, plan Phase D; spec §3/D10 amendments in §14): resourcePicker(dataset, READ) + seedFormat auto / jobsFeed / linkedinScraperCompat / indeedScraperCompat; seeds an EMPTY delta state for free (revision 1, no rows, no events) so the first live run emits only genuinely new jobs. Preconditions SEED_NEEDS_DELTA, SEED_STATE_NOT_EMPTY, SEED_TOO_LARGE (50,000), SEED_FORMAT_MISMATCH (≥ 5 % unparseable) - all pre-scrape, $0. Seeded occurrences: seeded: true, firstSeenAt = their postedAt, lastSeenAt = seed time (T18), aliases as a live observation would derive them, null hashes for absent fields (baselined silently on first sight - null-hash rule in changedFields), seenVia empty (never removable until observed live), snap from the seed. Report seed, deltaStats.seeded / seedSkipped.templates/n8n/daily-jobs-feed.json (schedule → run task → route new / updated / removed). Report meta.origin (WEB / API / SCHEDULER) and OUTPUT origin; OUTPUT/log nextStep nudge on non-scheduled or non-delta runs.job-removed (УТП spec WS1, plan Phase B; spec amendments D1/D9/D10 in §14): opt-in trackRemovals + removalConfirmRuns (delta mode); conclusive-scan rule over per-occurrence seenVia scan keys (≤ 8 per source) and per-run scan completeness from every adapter; removalConfidence high (ATS boards) / best-effort (LinkedIn/Indeed) with a first-run REMOVALS_BEST_EFFORT_SOURCE warning; removal rows carry the last known summary from separate snap shards (own 60 MB budget, LRU, fallback row when evicted; snapshotAvailable), removedAt, daysOpen; deltaStatus: removed in both dataset views; emission priority new > updated > removed; a removed identity seen again reopens as a new occurrence with repostOf (billed unique-job).job-removed ($0.0004 FREE / 0.00035 / 0.0003 / 0.00025 GOLD+), registered with this build; CI asserts the three-event set.LIMITS.snapMaxBytes 60 MB for removal summaries.deltaStats.removed / removalsExamined / removalsInconclusive / reopened, sourceHealth[].scans.companies, ATS sources enumerate the bundled catalog of verified boards (data/ats-catalog.json, catalogVersion 2026.08.1, 44 boards) largest-first within maxScannedJobs; new input catalogMode auto/off; report field catalog; optional operator KVS live override (CATALOG_KVS_ID, warning CATALOG_OVERRIDE_UNAVAILABLE), CATALOG_UNAVAILABLE only for a corrupt bundle. The resolver answers from the catalog before probing (companies.resolvedFromCatalog).maxItems title matches are emitted before description matches (deterministic). Guest sources stay title-only at list time.maxItems: 200 (first click ≈ $0.18); API default unchanged (1000).tools/catalog/build.js + seed-boards.txt: verify boards live and write the catalog (Phase F replaces the seed with public discovery).Store discount tiers registered (Bronze/Silver/Gold+; Free unchanged). Example task prepared for publishing.
Dedup: aggregator fan-out is same-country only (a LinkedIn card in another country with a region suffix in the title stays a separate, annotated row). Cross-source holdout (141 pairs, human-confirmed + rule A): pairwise precision 0.957 (4 "fp" = D7-resolved twin cases), recall 1.0.
location.all = union of member cities; per-source LinkedIn hashes cover the city set. Dense smoke: 357 of 403 unmerged LinkedIn rows were this class.US-/CA- prefixes, city shorthand (SF, NYC, SEA, CHI, ATL, DC…), comma lists of places as multi-locations, LinkedIn "New York, United States" as the city. Blocking adds a company|title-token block so cross-source listings with unparsed locations still meet (dense smoke: 293 Stripe pairs never met).tools/corpus/xsource-pairs.js (stratified sample) and tests/corpus/labels-xsource.csv (150 pairs, awaiting labels).locationConflict). Holdout confirmed by Sergey: fp=0/57, tp=2/2 (docs/dedup-calibration.md).#LI-Remote / #LI-Hybrid in the description may override the work mode; phrase heuristics apply only when no city is named (a "four weeks of fully remote work" perk no longer flips a hybrid role to remote).sergeyfaraday/jobs-feed-api)unique-job / job-update.publicFetch) for every user-influenced URL.proxyConfiguration input for the guest sources.