# Changelog of leboncoin France Car Scraper (`devilscrapes/leboncoin-france-cars`) Actor

- **URL**: https://apify.com/devilscrapes/leboncoin-france-cars/changelog.md
- **Full Actor documentation**: https://apify.com/devilscrapes/leboncoin-france-cars.md

## leboncoin France Car Scraper — Changelog

### 0.7.0 — 2026-09-17

- **Fix: run-wide wall-clock budget stops the run before the platform kills
  it TIMED-OUT.** All 8 of this Actor's TIMED-OUT runs in the last 30 days
  (30-day public stats: 293 runs, 21 failed) landed inside one ~23h window —
  a spell of heavier DataDome blocking on detail-page fetches let
  retry+backoff time pile up across many enrichment calls until the platform
  force-killed the run past its declared `timeoutSecs` (3600s default),
  discarding the honest partial-success reporting this fleet otherwise
  guarantees. The 2026-09-02 fix (0.6.0) bounded each individual Apify SDK
  round-trip but not the SUM of scraper-side HTTP retry time — a single
  blocked detail page can already cost up to ~235s
  (`RETRY_MAX_ATTEMPTS` x `HTTP_TIMEOUT_S` + backoff), and nothing capped
  how many of those a run could accumulate.
  - **Reproduced live**: a 100-result cloud recon run (`enrichDetails=true`,
    the default) showed real, frequent HTTP 403 blocks and curl
    connection-timeouts on `detail <id>` fetches — several listings each
    burning 20-40s across 2-4 retries before succeeding or giving up.
    Extrapolated to this Actor's max `maxResults=1000`, that pace comfortably
    exceeds the platform's 3600s run timeout.
  - `src/run_budget.py` (new): a pure `RunBudget` dataclass — `deadline` +
    latching `exceeded` flag — mirrors the same pattern already proven on
    `craigslist-listings-scraper`/`offerup-listings-scraper`.
  - `src/main.py`: `_resolve_run_budget()` derives the deadline from the
    REAL platform-assigned kill clock (`Actor.configuration.timeout_at`),
    minus a 300s safety margin — not a guessed constant, so it tracks the
    run's actual `timeoutSecs` (default or customer-overridden). Falls back
    to 3300s when unavailable (local `apify run`). `_build_status_message`
    now reports "Stopped early — approaching the run time limit after N car
    listing(s) scraped." when the budget (not a push stall) ended the run.
  - `src/scraper.py`: `_iter_listings` checks `budget.check()` before every
    new search-page fetch (stops pagination, never raises — even on page 1,
    this is self-imposed time management, not a block); `_emit_page` checks
    it before every detail-page enrichment fetch (skips just the enrichment,
    still yields the base row — it already carries price/brand/model/year
    from the free search payload). Once budget is exhausted the run finishes
    SUCCEEDED with whatever rows are already collected instead of being
    force-killed TIMED-OUT.
  - 15 new tests (was 39, now 54): `tests/test_run_budget.py` (RunBudget
    unit tests), `tests/test_run_budget_integration.py` (`_iter_listings`/
    `_emit_page` stop issuing requests once budget is exceeded, base rows
    still yielded), `tests/test_main.py` (+6: `_resolve_run_budget` reads
    `timeout_at` correctly / falls back / clamps near-zero remaining time;
    `_build_status_message` covers all three finish reasons). ruff clean,
    pyright 0 errors, `apify run` against `tests/fixtures/input.qa.json`
    exits 0.

### 0.6.0 — 2026-09-02

- **Fix: unbounded Apify SDK round-trips could turn a complete run into
  TIMED-OUT.** 30-day public stats: 220 customer runs, 20 failed, 9 of those
  TIMED-OUT (not errored) — this Actor's build was 12 days old at diagnosis
  time, so most of that window postdates the last fix and the failures are
  current, not residue. Every `Actor.*` / `ProxyConfiguration.new_url()` call
  after input validation — `Actor.charge`, `Actor.create_proxy_configuration`,
  the residential-proxy probe `new_url()`, `Actor.push_data`,
  `Actor.set_status_message`, and the mid-retry IP-rotation `new_url()` inside
  `_get_text` — had no timeout of its own. `apify_client`'s own retry policy
  can stall a single round-trip for tens of minutes, well past this Actor's
  declared run timeout, so any one stall could convert a functionally-complete
  run into a platform-forced TIMED-OUT with nothing in our own logs to
  explain it. Exact same defect class root-caused and fixed the same way
  2026-09-01 on `workday-jobs-scraper`, `zoopla-uk-property-scraper`, and
  `vrbo-vacation-rentals-scraper`.
  - `src/main.py`: added `PUSH_TIMEOUT_S = 60.0`. `push_data` now raises a new
    `PushStalledError` on stall (never swallowed — a silent swallow would
    report a false "Done" while the batch never landed); the scrape loop
    (`_scrape_and_push`) stops draining `scraper_run()` as soon as this fires,
    keeping every already-pushed row and reporting an honest partial status.
    `charge`, `create_proxy_configuration`, the proxy probe `new_url()`, and
    `set_status_message` are all now bounded — the first three fail loud
    (`RuntimeError`, consistent with the existing "never fall back to direct
    routing" contract), the last two swallow-and-log (non-fatal telemetry).
  - `src/scraper.py`: added `PROXY_ROTATE_TIMEOUT_S = 60.0` around
    `ProxyConfiguration.new_url()`. The initial call in `run()` (before any
    request is made) still fails the whole run loud on stall. The mid-retry
    rotation call inside `_get_text` now swallows a stalled rotation and
    keeps the previous exit IP instead of hanging or crashing the whole run —
    the retry loop already tolerates one failed attempt (fault-isolation
    pattern, `reference-fleet-fault-isolation-pattern`).
  - Own-account runs never reproduced the customer timeouts: PPE customer
    runs execute under the customer's account, and `/v2/acts/{id}/runs`
    returns only this Actor's own 2 runs (both SUCCEEDED, tiny QA fixture,
    finished in ~15s — nowhere near the wall-clock/SDK-retry conditions
    needed to trigger a stall). This is an evidence-based code audit against
    the proven fleet-wide pattern, not a reproduced failing log, and it is
    unproven until cloud QA plus a live-traffic window confirm the timeouts
    stop.
  - 39 tests (was 24; +15: `tests/test_main.py` new, `PushStalledError`
    swallow/raise pairs; `tests/test_ip_rotation.py` +4 covering bounded/
    stalled rotation), ruff clean, pyright 0 errors, and a real local
    `apify run` against `tests/fixtures/input.qa.json` via live residential
    proxy exited 0 with 5 real rows — not a zero-row false green.

### 0.5.0 — 2026-08-19

- **A search that matched nothing no longer fails the run.** `_iter_listings`
  collapsed two different outcomes with `if not ads:` and raised on page 1,
  throwing away the distinction its own return type already carried
  (`list[...] | None`): `None` means we could not READ the page (no
  `__NEXT_DATA__`, or a DataDome block `_get_text` already retried), while `[]`
  means `__NEXT_DATA__` parsed cleanly and `searchData.ads` was empty — the
  search RAN and genuinely matched nothing, which is easy to do with a narrow
  brand + price + region filter combination. `actor-start` is charged before the
  scrape, so the old behaviour billed the customer and then handed them a FAILED
  run. 22 of the last 227 customer runs (30d) were FAILED.
  **A block on page 1 still fails loud** — reporting a wall as "no results"
  would give the customer a wrong answer, which is worse than an error.
- **14 of this Actor's 24 tests never ran.** `tests/test_ip_rotation.py` and
  `tests/test_proxy_guard.py` drove coroutines through
  `asyncio.get_event_loop().run_until_complete(...)`, which raises "There is no
  current event loop in thread 'MainThread'" on Python 3.11+. So the entire
  IP-rotation and residential-proxy-guard suite — the regression cover for the
  0.4.0 root-cause fix — was silently failing. Switched to `asyncio.run`:
  28 tests pass, was 14 failing.
- Added `tests/test_empty_search_succeeds.py` pinning all four branches
  (empty page 1, unreadable page 1, unreadable later page, exhausted result set).

### 0.4.0 — 2026-06-13

- Root-cause fix for 70% failure rate (2026-06-10 to 2026-06-12): DataDome was
  blocking specific residential exit IPs with HTTP 403. The old retry loop rotated
  browser profiles but kept the same proxy URL (same session\_id = same exit IP),
  making all 5 retries futile against a blocked IP.
- Fix: on each HTTP block (403/429/401/405), `_get_text` now calls
  `proxy_configuration.new_url(session_id=fresh)` to obtain a different
  residential exit IP before the next attempt. Proxy URL is propagated back
  through the call chain so subsequent requests use the fresh address.
- `_resolve_proxy_url` renamed to `_resolve_proxy_configuration` and now returns
  the `ProxyConfiguration` object (not a plain URL string) so the scraper can
  rotate IPs at will.
- Proxy URL passed per-request via `session.get(proxy=url)` instead of baking
  it into the `AsyncSession` constructor, enabling per-retry IP rotation without
  recreating the session.
- 6 new unit tests covering IP rotation: session\_id uniqueness, rotation on block,
  no rotation on network error, returned proxy\_url reflects rotated address.

### 0.3.0 — 2026-06-10

- Force-residential proxy guard: `_resolve_proxy_url` now always injects
  `groups=["RESIDENTIAL"]` regardless of customer input, preventing the
  datacenter-exit footgun that causes DataDome HTTP 403 on leboncoin.fr.
- Fail-loud on unavailable proxy: raises `RuntimeError` with a clear message
  instead of silently falling back to direct routing and returning empty results.
- Added unit tests asserting RESIDENTIAL is forced and raises when proxy is
  unavailable.

### 0.2.0 — 2026-06-05

- Config-refresh re-push: ship RESIDENTIAL proxy prefill to live build so Apify auto-QA routes through residential exits (DataDome-class anti-bot requires residential IPs).
- Bumped version to 0.2 — strictly greater than published 0.1.

### 0.1.3 — 2026-06-03

- First live cloud publish. Fixed network-error retry (proxy 595 → retryable like HTTP blocks).
- Added per-request Referer + Sec-Fetch-Site headers so DataDome sees a proper
  same-origin navigation chain (landing → search → detail). Resolves HTTP 403
  on the search endpoint from cloud residential IPs.
- QA PASS: 5 rows, actor-start=1, result-row=5, runId=lEv4fYGUbnsiGEnVF.

### 0.1.0 — 2026-05-15

- Initial scaffolded release.