Avito deployed a new firewall layer (QRATOR proof-of-work challenge, HTTP 439) that the previous
engine could not pass over HTTP, pushing every page to the slow browser fallback — the root cause of
the recent FAILED / TIMED-OUT runs.
- Proof-of-work solver (
src/pow.js): the challenge is solved in-process on the same
session/IP and the page is re-fetched — no browser needed.
- Redirect over HTTP: Avito's keyword→category redirect is now read from the embedded page state
(
loaderData.data.redirected / url) and followed by the HTTP pass. The browser is only a
last-resort fallback.
- Exact pagination: fan-out is bounded by Avito's reported
totalCount; a page that returns a
catalog with 0 items is treated as end-of-results, a bare shell is not.
- Robust retries: generous retry budget with fast timeouts for throttled/dead residential IPs;
pages that fail every retry get one more HTTP round and are reported, not silently dropped.
- Clear zero-result semantics: an empty query finishes cleanly; a run that could not load any results
page fails loudly with proxy diagnostics.
Initial release — Avito search-results scraper.
- Hybrid engine: impit HTTP pass (real Chrome TLS + HTTP/2 fingerprint, no browser) plus a
Playwright fallback that resolves Avito's client-side category redirect once, then hands pagination
back to the fast HTTP pass.
- JSON-first extraction from Avito's embedded catalog state — enriched fields (exact price,
posted date, GPS coordinates, category, photo count, seller rating); DOM selectors as a fallback.
- Input: structured
keyword / region / category / priceMin / priceMax / sort, or
ready-made startUrls. Apify Residential (country RU) proxy by default.
- Output: dataset with an "Overview" table view; public listing fields only (no personal data).
- Browser pass blocks images/media and third-party trackers to cut load time and rate-limit pressure.
- Unit tests for the parser and URL builder.