hiring.cafe Jobs Scraper

Scrapes job listings from hiring.cafe — a job aggregator that pulls postings from hundreds of upstream ATS platforms (ADP, Workday, iCIMS, Dayforce, SmartRecruiters...) and runs every one through an internal "AI search" pipeline that extracts ~80 structured fields per job: compensation range, seniority level, remote/onsite/hybrid policy, degree requirements, shift/travel requirements, visa sponsorship, plus ~18 fields of enriched company data (funding stage, employee count, industry). Never covered by this portfolio before.

Public data only. No login, no cookies, no browser.

The one thing you need to know before using this

No WAF was found — 8 TLS profiles, all clean, cold. The catch is robots.txt: it explicitly disallows the site's own search-UI query strings (?searchState=, ?page=), but explicitly Allows the plain /jobs/{slug} path form. This actor speaks ONLY that allowed path form — one request per query, one page of results (hiring.cafe's own ceiling, normally 20), no query string ever touched. See CRAWLING_METHOD.md for the full trail, including why a working deeper-pagination mechanism was found and deliberately not used (policy, not a technical wall).

What you get

Three record types share one dataset, told apart by recordType.

JOB — one row per listing

Convenience fields (title, company, compensation, seniority, workplace type, apply URL...) are pulled to the top level for the dataset table; the full upstream object is also nested verbatim under job — nothing is lost to the projection, including the raw v5_processed_job_data and enriched_company_data blocks if you want every one of the ~80+18 fields.

SEARCH_SUMMARY — one row per query

Upstream's own totalCount / companyCount, plus its own human-readable headingText (e.g. "6,118 software engineer jobs at 1,678 companies in New York") as an independent sanity check on the structured numbers.

ERROR — one row per input that failed

So every entry in Search queries maps to at least one output row.

Input

Field	What it does
Search queries	free text — job title, US state/city, work type, or a combined phrase ('data scientist austin tx', 'remote nurse') — one request per entry
Max concurrent requests / Min seconds between requests	tuned generously — no bot-mitigation gate observed

Example

{
  "queries": ["software engineer", "Austin, TX", "remote data analyst", "part time"]
}

There is deliberately no separate location filter: combining two canonical slugs as extra path segments does not work upstream (verified live), but folding both into one query string does — put the whole title+location phrase in one entry.

Notes on reliability

• Bogus query → genuinely honest zero results. Verified live (notarealslugxyz999 → totalCount: 0) — not the silent-widen-to- national-baseline trap that dominates elsewhere in this portfolio, so no defensive gate was needed.
• Hard ceiling: one page per query, normally 20 rows. There is no robots-compliant way to go deeper — ?page= is disallowed by policy, and a second path segment (/jobs/akron-oh/2) is parsed as a second, unrelated slug and returns zero results, not page 2. Add more/narrower queries to get more rows, not a page count.
• Every result was is_expired: false and estimated_publish_date same-day in recon — hiring.cafe appears to already filter to live postings server-side.
• Compensation fields are null when the original posting did not disclose pay (compensation.isTransparent: false), not a parsing gap.

Output envelope

Every record carries _input, _source and _scrapedAt. Upstream field names pass through verbatim, nested under job — no renaming.

See CRAWLING_METHOD.md for the full reverse-engineering trail.