Fixed — Apify flagged this listing UNDER_MAINTENANCE (QA run fERfQaFN1JSIDffhT, build 0.0.4, 2026-09-29)
The QA log showed two distinct failures, 18.6s apart:
Zero backoff on Cloudflare-challenge session rotation.IndeedScraper.scrape()
rotated sessions on a detected challenge with no delay between attempts — all 4
rotations in the flagged run landed inside ~3 seconds, burning the entire
MAX_SESSION_ROTATIONS budget against the same hot anti-bot decision before
enough wall-clock time passed for a retry to plausibly look different. Added
_rotation_backoff_s() — exponential-with-full-jitter delay (2s base, 30s cap,
matching the org's standard backoff envelope) between rotations.
A clean HTTP 401 on the second query was lumped into the generic
HTTP {code} block path, indistinguishable in the log from any other
unexpected status. Gave it a dedicated IndeedAuthRequiredError (still an
IndeedBlockedError subclass — REQ-12's "treated as a block, no silent
retry" contract is unchanged) and logs a body snippet, so the condition is
named distinctly and the next occurrence is diagnosable from the log alone.
Verified locally only — pytest/ruff/pyright all green, plus a local
curl-cffi probe confirming direct egress still clears (200) and WebShare
datacenter egress still gets the known Cloudflare 403 challenge. The exact
RESIDENTIAL-proxy 401 from the flagged run could not be reproduced: Apify
Proxy is account-wide down as of this fix (REST + proxy.apify.com both
refusing), so what Indeed's 401 represents is recorded as an open hypothesis,
not a confirmed root cause. See docs/specs/indeed-jobs-scraper/notes.md and
src/scraper.py's module docstring. Ships via apify push once the platform
block lifts; not yet republished.
