Scrape used-car listings from coches.net, Spain’s #1 car marketplace — price, make, model, year, mileage, fuel, power, transmission, environmental label, seller, location, and photos. Export to JSON or CSV; enrich each listing with extra detail-page fields.
Fix detail pages rejecting "cold" requests with a persistent HTTP 403
(30-day public success rate still 17% — 21 FAILED/4 TIMED-OUT/5 SUCCEEDED
of 30 — after v0.7's ProxyError fix). Reproduced directly against the
live Actor: cloud run ezxrbObUgW57S5zXg (build 0.7.1) shows the search
page fetching real content cleanly, then every subsequent detail-page
fetch getting HTTP 403 — even after the existing retry loop rotated proxy
IP and browser-impersonation profile on each attempt. Root cause: the
module's own docstring always documented the intended design ("GET the
search listing first to warm the session's cookies, then issue detail
requests on the same session with a Referer") but the implementation never
did this — _get_with_retry opened a brand-new, cookie-less
AsyncSession for every single HTTP request since the Actor's very
first commit. coches.net's anti-bot layer reliably rejects a detail-page
"click" from a session that never visited the search results first,
regardless of which residential IP it comes from.
Replaced the per-request session with _RotatingSession: one curl_cffi
session (and its cookies) now persists across a search-page fetch and all
of that page's detail-page fetches — like a real visitor browsing from
results into ads — and is only replaced (and re-warmed with a fresh
search-page revisit before the next request) when the target actually
blocks it. See tests/test_session_reuse.py.
Pin the proxy exit country to Spain by default (apifyProxyCountry
defaults to "ES" unless the customer overrides it) — coches.net is
Spain-only content; an unpinned residential exit can land geo-random (see
memory feedback-pin-proxy-country). See tests/test_proxy_guard.py.
Add a geo-splash/off-site-redirect guard: a 200 response whose final
URL left www.coches.net (a geo-mismatched exit redirected to a
country-selector/splash host) is now treated as a block — same
retry-and-rotate path as a 403 or the existing bot-detection soft block —
instead of being handed to the parser as real content. See
tests/test_geo_block_detection.py.
Never a silent empty success: zero listings parsed off search page 1
now raises EmptySearchResultError carrying the raw HTML, which
main.py dumps to the Actor's key-value store (EMPTY_SEARCH_PAGE_HTML)
before failing the run loud — so a wrong-locale/challenge page that slips
past every rotation is inspectable after the fact instead of just
producing a bare "no listings found" error. See
tests/test_empty_search_result.py and
tests/test_empty_search_debug_dump.py.
0.7.0 — 2026-08-05
Fix the ProxyError-misclassified-as-permanent bug (30-day public success
rate 13%, 21 FAILED + 5 TIMED-OUT of 30 — still failing after v0.6's
soft-block fix). Root cause: v0.5 (see below) correctly identified that
a genuine proxy-authorization rejection is a permanent, whole-run
condition and should fail loud instead of retrying — but its fix caught
everycurl_cffiProxyError, not just that specific 407-class
failure. In production, the dominant ProxyError signature is
curl: (56) CONNECT tunnel failed, response 590 — a per-session
tunnel failure against this specific target/exit IP (libcurl error 56,
a generic network-receive error), not an account/plan authorization
problem. _try_get raised PermanentProxyError on the first occurrence
of anyProxyError, killing the whole run instantly — including mid-run,
after the retry loop had already successfully rotated past two real HTTP
403s on the same detail page.
Reproduced directly against the live Actor: 3/3 cloud runs with the
default input (maxResults: 50, enrichDetails: true) FAILED in ~18-20s
with this exact signature; one additional run recovered from two 403s via
the existing retry/rotate path and then died on the third attempt's
ProxyError. Confirmed curl_cffi.curl.CurlECode 56 = RECV_ERROR, a
generic network condition, not a distinct auth-failure code — genuine
proxy-auth rejections surface with 407 in the message instead.
_try_get now only raises PermanentProxyError when the ProxyError
message carries the 407 auth-failure marker (PROXY_AUTH_FAILURE_MARKER).
Every other ProxyError maps to HTTP_UNAVAILABLE and flows through the
existing retry-and-rotate path — a fresh session on the next attempt gets
a different residential IP, which the direct repro confirms frequently
succeeds (search-page fetches with real residential IPs returned genuine
__INITIAL_PROPS__ content with zero retries in isolated small runs).
Added a run-wide wall-clock budget (MAX_RUN_SECONDS = 600) via a new
_FetchContext dataclass threaded through _iter_listings/_emit_page.
Now that more requests retry instead of instantly killing the run, this
bounds worst-case duration so a heavily-blocked run fails loud (zero
results) or returns a partial dataset well before an external
platform/customer timeout would silently kill it as TIMED-OUT — directly
addressing the 5/30 TIMED-OUT runs alongside the 21 FAILED.
Added regression tests: test_proxy_error_fail_fast.py now covers the
exact "response 590" production signature (must retry, not raise) and a
407-marked variant (must still raise PermanentProxyError).
test_run_budget.py covers the new deadline behavior in both
_iter_listings (fail loud on zero results) and _emit_page (stop
cleanly with partial results).
No changes to the v0.6 soft-block detection — this is additive.
0.6.0 — 2026-07-30
Fix the HTTP-200 bot-detection soft block (30-day public success rate
10%, 27/30 FAILED/TIMED-OUT — unchanged from the day before at 28/30).
Root cause: coches.net's bot-detection layer sometimes answers a blocked
request with HTTP 200 and a small "Ups! Parece que algo no va bien..."
interruption page instead of a 403/429. _try_get only branched on
response.status_code, so a 200 was always treated as a successful
fetch — the challenge page's body was handed to parse_initial_props,
which found no __INITIAL_PROPS__ and returned None. On page 1 that
raised a misleading "no listings found" ValueError (a FAILED run); on
later pages it silently ended iteration early (a truncated but
"successful" run, or a TIMED-OUT run once retries piled up across many
detail-page enrichments). Neither path ever rotated the session or
retried, even though the existing 403/429 retry-and-rotate machinery
would have handled it identically if only it had been told about it.
Reproduced directly against the live site (repeated requests from the
same network eventually get served this exact interruption page).
Added parser.is_bot_block_page() — detects the interruption page via
three stable markers in its markup (showBlockPage,
onProtectionInitialized, interruption-message). _try_get now
checks this on every 200 response and returns the new HTTP_SOFT_BLOCK
sentinel instead of the body when detected, routing it through the same
retry/rotate path as a real 403/429.
Added tests/test_bot_block_detection.py (captured interruption-page
fixture + _try_get unit coverage) pinning the regression.
No proxy-guard or proxy-error-handling changes in this release — this
is additive to the v0.4 RESIDENTIAL guard and v0.5 PermanentProxyError
fail-fast, both already live.
0.5.0 — 2026-07-10
Fix TIMED-OUT customer runs (30-day public success rate 15%, 28/33
TIMED-OUT). Root cause: when a run's Apify Proxy configuration
requests a group the customer's plan can't authorize (RESIDENTIAL with
0 GB quota is common on FREE-tier accounts), Apify Proxy answers with a
permanent proxy-auth rejection for the entire run. curl-cffi
surfaces this as ProxyError (libcurl CURLE_PROXY — HTTP 407-class),
which the old code caught with a broad except (OSError, RuntimeError)
and mapped to HTTP 503 — a transient, retryable status. Every single
search/detail request then burned the full 5-attempt exponential
backoff (up to ~60s of sleeps, compounding across up to 100 pages and
1000 detail-enrichment requests), blowing past the Actor's default run
timeout without ever raising — hence TIMED-OUT, not FAILED, and zero
visibility in our own run history (the failures all belonged to
customers).
Added PermanentProxyError in scraper.py. _try_get now catches
curl_cffi.requests.exceptions.ProxyError specifically and raises it
immediately instead of mapping it to a retryable status — the Actor now
fails loud on the first proxy-auth rejection instead of silently
exhausting the retry budget on a condition that cannot self-heal within
the run.
Added tests/test_proxy_error_fail_fast.py — pins the exact regression
(ProxyError -> fail fast) and confirms genuine transient network errors
keep their existing retry semantics.
Added tests/fixtures/input.qa.json (small, enrichDetails: false) for
cloud QA.
0.4.0 — 2026-06-10
Force-residential proxy guard: _resolve_proxy_configuration now always
injects RESIDENTIAL into apifyProxyGroups when the group is missing,
empty, or set to a datacenter-only group — preventing the HTTP 403 that
customers hit when running without residential proxy.
Fail-fast on proxy unavailable: if Actor.create_proxy_configuration raises
or returns None the Actor exits immediately with a clear RuntimeError
describing the required plan/config, rather than silently returning empty
results from datacenter exits.
country_code (apifyProxyCountry) preserved through the guard.
Updated models.py proxy field description and input_schema.json proxy
description to document the RESIDENTIAL requirement.
Added tests/test_proxy_guard.py with unit coverage for all guard branches.
0.3.0 — 2026-06-05
Config-refresh re-push: RESIDENTIAL proxy group explicitly set in both
input_schema.jsondefault and prefill so Apify auto-QA receives
residential exits rather than datacenter IPs, clearing the 403 / "Under
maintenance" trap triggered by DataDome-class anti-bot on coches.net.
No code changes — build is functionally identical to 0.2; only the
published proxy config is updated.
0.2.0 — 2026-06-03
Per-attempt proxy session rotation: each retry now opens a fresh AsyncSession
with a new session_id (new residential exit IP) and a randomly-picked
browser impersonation profile, defeating persistent IP/fingerprint blocks.
Removed the single static proxy URL resolved at startup; the ProxyConfiguration
object is now threaded through to the scraper for per-request fresh URLs.
Fixed DockerfileCMD to use python -m src (via __main__.py).
![Coches.net [Just 💰$3] — Used Cars in Spain avatar](https://images.apifyusercontent.com/6iAomz4uQYbjfXtT0JQ1TQiMDeSeqjY0iaRkukWAJiU/rs:fill:76:76/cb:1/aHR0cHM6Ly9hcGlmeS1pbWFnZS11cGxvYWRzLXByb2QuczMudXMtZWFzdC0xLmFtYXpvbmF3cy5jb20vclB1SjNjc1NCUjVsaTVmd1QtYWN0b3ItZEZheFpBcGI3Nm1yemRxTXEtOGdhSEVoTUU1Ry1jb2NoZXMubmV0X2xvZ28ucG5n.webp){kind=avatar}
