# Changelog of Bilbasen Scraper — Denmark's Car Marketplace (`blackfalcondata/bilbasen-scraper`) Actor

- **URL**: https://apify.com/blackfalcondata/bilbasen-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/blackfalcondata/bilbasen-scraper.md

## Changelog

### 0.4.28 (2026-09-13)

- Fixed: two runs started at the same moment on the same `stateKey` could both pass the ownership check and deliver — and bill — the same listings. Ownership is now settled atomically before the soft lock, so exactly one run proceeds and the other exits immediately with no results.
- Fixed: a run that had already lost its lock could delete the lock belonging to the run that held it. A lock is now only released by the run that still owns it.
- Improved: updated the crawler library to a version that no longer wedges a run when an internal storage call fails (such a run used to sit idle until its timeout and return nothing).
- Fixed: this actor's run-completion record carried the whole internal schema report, making it roughly 40x larger than any comparable actor's and occasionally too slow to deliver — which left some finished runs showing no final status even though they had succeeded. The report now travels separately and only when something actually changed, so the completion record arrives reliably. No change to results, input or billing.

### 0.4.27 (2026-08-18)

- Fixed: two runs started at the same time on the same `stateKey` but for *different* searches (e.g. VW ID.3 and VW ID.4) shared one change-tracking lock, so the second run quietly exited with zero results while reporting success. Change tracking is now kept per search (make + model + start URLs), so unrelated searches on one `stateKey` never block each other. Existing tracking is carried over automatically; the first run of each search after upgrading reports no `EXPIRED` rows (it re-establishes coverage), then resumes normally.

### 0.4.26 (2026-08-06)

- Fixed: searching by `model` returned **zero results** for any model whose name is not a single lowercase word — Tesla Model S / Model 3 / Model Y, VW ID.3 / ID.4 / ID.5, Golf VIII, and many more. The model was sent to the site in a dashed, punctuation-stripped form (`model-s`, `id3`) that the site's own model taxonomy does not recognise; it answered with an empty, successful result set, so the run finished green with nothing in it. Model names are now sent in the form the site actually uses. Single-word models (Enyaq, Born, G6, A4, Passat) were never affected. Present since 2026-06-20.
- Fixed: `strictModelFilter` dropped every listing for models containing a dot (VW ID.3, ID.4) because it compared the listing URL against `id.3` while listing URLs use `id3`. It now uses the same slug form the listing URLs carry.

### 0.4.24 (2026-08-02)

- Changed: `emitExpired` now defaults to **false** (was `true`). EXPIRED rows are billed as standard results, so sold-detection is now something you opt into rather than something you pay for by default. If you want EXPIRED rows, set `emitExpired: true` — behaviour is otherwise unchanged. Existing runs that already pass `emitExpired: true` explicitly are unaffected.
- Fixed: moving search pages could repeat the same vehicle at a page boundary, exporting and charging it twice. Listings are now deduplicated across the full run before enrichment, result-budget reservation, export, and billing.
- Fixed: removed a legacy low-memory result cap that could silently reduce a user's requested coverage instead of relying on the active streaming pipeline.
- Improved: user-visible progress and warning logs no longer expose raw URLs, pagination limits, internal state coordination, pacing values, or raw failure causes.

### 0.3.22 (2026-06-10)

- Added: sold-detection via `changeType: "EXPIRED"` rows. In incremental mode, a tracked listing that has been missing from 2 consecutive fully-covered runs is exported once as a minimal EXPIRED row (`url`, `portalUrl`, `externalId`, `contentHash`, `scrapedAt`; other fields null), billed as a standard result. Controlled by the new `emitExpired` input (default `true`).
- Added: coverage and scope safety guards around expiry. EXPIRED only fires when a run provably covered the full search scope — clean make/model searches or browse-all. Filtered searches (year, price, mileage, etc.), multi-search runs, and partial or interrupted crawls never expire listings, so a listing absent because a filter excluded it is never falsely reported as sold.
- Deprecated: `pushUnchangedSkeletons`. EXPIRED-based sold-detection replaces it without re-exporting the full inventory every run. The flag still works for existing configurations.
- Corrected: the 0.3.20 claim that unchanged skeleton rows are "no longer billed as Results" was wrong — live verification shows the platform bills every exported dataset row, including skeleton rows. The actor cannot exempt individual rows from per-result billing. Input-schema and README copy now state this honestly; this correction is part of why skeleton mode is deprecated in favour of EXPIRED rows, which only export when something actually changed.
- Changed: incremental state upgraded to v2, adding per-listing lastSeen/missed tracking to support expiry. Existing v1 state migrates transparently on first run — no input changes and no re-export of already-tracked listings.

### 0.3.21 (2026-06-10)

- Fixed: a run resumed after a platform migration could re-export and re-bill listings that were already delivered before the migration. Delivered listings are now tracked in persisted run state (recorded only after a successful export), and every export path — detail, recovery, and list-only — checks it before pushing, with `maxResults` enforced against the migration-surviving counter.
- Fixed: `incrementalMode: true` had no effect on the primary search-results-only path (`includeDetailPages: false`) — every run re-exported and re-billed the full result set. The path now performs the same change detection as the rest of the actor: unchanged listings are skipped at no charge, and exported items carry `changeType`/`contentHash`.
- Fixed: in incremental runs, listings the actor deliberately skipped as unchanged could be resurrected by the end-of-run recovery pass as billed search-level rows. Recovery now distinguishes "skipped as unchanged" from "fetch failed", re-checks the stored hash before exporting, and stamps recovered rows with `changeType`/`contentHash` so `notifyOnlyChanges` handles them correctly.
- Fixed: unchanged listings no longer consume the `maxResults` budget in incremental runs (previously a run could deliver far fewer items than requested while reporting an inflated export count). Change detection now happens before budget slots are taken, both at export and at detail-page scheduling.
- Fixed: incremental state is recorded only after an item is successfully exported (previously a crash in a narrow window could permanently suppress a never-delivered listing from future runs).
- Added: the run now fails with a clear error instead of succeeding with 0 items when the website's listing-link format changes (the failure mode of the v6 site update). Genuinely empty search results still exit SUCCESS.
- Fixed: `descriptionFormat` and `excludeEmptyFields` inputs were accepted but silently ignored; both now apply to every exported item (skeleton rows keep their fixed shape).
- Fixed: search-result extractors no longer turn a legitimate `0 km` mileage into `null`. Note: this changes the content hash for affected listings, so incremental users may see a one-time `UPDATED` re-export of the 0-km cohort on the first run after upgrading.
- Improved: concurrent-run protection hardened — the state lock is renewed during long runs (previously a second scheduled run could take over after 30 minutes), released on abort, and periodic state saves are skipped after a lost lock.
- Improved: clearer, leaner progress logging during retries and recovery.
- Fixed: after a platform migration, listings whose detail fetch had failed before the migration were silently dropped (the recovery pass only knew about the current incarnation's work). The scheduled detail work is now tracked in persisted run state, so recovery covers pre-migration listings too — and unchanged skeleton rows are no longer duplicated after a resume.
- Fixed: a rare platform billing hiccup on a single item no longer causes that item to appear twice in the dataset (the item is kept, the billing failure is recorded internally, and the run completes normally).
- Improved: the website-change tripwire now also detects format changes via the structured search data itself, while ad/tracking links can no longer be mistaken for evidence — genuinely empty searches always exit SUCCESS.
- Fixed: a price of `0` from any source now consistently normalizes to `null` (it previously slipped through as `0` on one extraction path).
- Improved: runs that receive a non-actionable `browseAll` request (e.g. no usable make/model/URL inputs) now exit immediately with a clean SUCCESS status instead of spending time in the crawl loop. No results are pushed; the run status accurately reflects that there was nothing to scrape.
- Fixed: run footer no longer labels results as "jobs" — the counter now correctly reads "listings".
- Improved: short-timeout runs (e.g. Apify's automated quality check) now stop cleanly at the startup boundary and report the correct deadline-exit status, rather than timing out mid-crawl. `maxResults`-bounded runs also stop early once the target count is reached instead of waiting for the full crawl to drain.

### 0.3.20 (2026-05-19)

- Changed: unchanged-skeleton items (changeType:"UNCHANGED", pushed by `pushUnchangedSkeletons` mode) are no longer billed as Results. They cost the actor essentially nothing to produce (no detail fetch, no compute beyond a hash compare) and previously made `pushUnchangedSkeletons:true` runs cost roughly the same as full-detail runs — at odds with the flag's "large cost saving on stable inventories" description. Only new and updated listings (plus serp-only items returned when detail-fetch fails) count as billable Results. Input-schema description updated to reflect this.

### 0.3.19 (2026-05-16)

- Fixed: detail-page data collapsed to 0% on runs where the site rate-limited the primary detail path. The 0.3.18 rescue pass pre-pushed missed listings as `serp-only` and incremented the push counter, which permanently disabled the later detail recovery pass. All affected runs returned items with `dataCompleteness: "serp-only"` and every detail-only field (description, EV specs, equipment, ratings, dealer contact, inspection dates, etc.) null. Recovery now tries the stronger detail path first for missed URLs and only falls through to `serp-only` for listings that still cannot be enriched. Honest-exit no longer fails the run when recovery yielded items.
- Improved: the circuit-breaker tripping no longer emits a user-visible `ERROR [BIL-2030]` log line. The breaker handing off to the alternate path is the expected recovery flow, not a fatal error — the existing WARN line ("Stopping fast-path detail retrieval... will attempt browser-based fallback") already signals the transition, and the run's recovery is logged at INFO. The operator still receives the structured event (type/code/detail/payload) on the diag sink for monitoring.
- Added: wall-clock budget on the alternate detail-retrieval path. On a fully-blocked session each navigation can hit its 15-second timeout, so a 300-listing recovery loop could outlast Apify's default 60-minute run timeout and end TIMED-OUT before the safety-net push could run. The loop now bails at 25 minutes (or after 20 consecutive misses, as before) and hands the remainder to the serp-only safety net, so the run still exits cleanly with a mix of detail + serp-only items rather than timing out with neither.
- Added: hard per-listing timeout on the alternate detail-retrieval path. The underlying page navigation timeout was observed to hang past its deadline on a fully-blocked session — a 300-listing run sat for 40+ minutes in the recovery loop because individual navigations never resolved, so the loop-level budget check at the top of each iteration never got control back. Each retrieval now races against a setTimeout so the loop ALWAYS gets the iteration back within ~20 seconds, regardless of lower-level navigation behaviour.

### 0.3.18 (2026-05-16)

- Fixed: when skeleton-mode (incremental + pushUnchangedSkeletons) pushed unchanged listings before any detail-fetch had been attempted, the `pushCount > 0` precondition disabled the browser-detail fallback. Any listing whose detail-fetch then failed (WAF block, timeout, etc.) was silently absent from the dataset — breaking sold-detection in downstream consumers because a listing missing for N consecutive runs gets marked sold even though it's still live and just WAF-rejected. After `crawler.run`, a rescue pass now pushes each missed listing as `dataCompleteness: "serp-only"` with the SERP hash recorded so the next run can short-circuit to skeleton-push.
- Fixed: detail outputs no longer include the raw Next.js page state. `bodyType` now populates from the current `modelInformation.Type` payload, while the stale `engineSize` output field was removed because current detail payloads do not expose an engine-size label.

### 0.3.17 (2026-05-16)

- Fixed: the serp-only fallback (used when both detail-retrieval paths are unavailable) was pushing items to the dataset but not recording the SERP hash in incremental state. URLs that consistently hit the WAF on detail fetch sat in a permanent re-enqueue loop: every run re-attempted detail-fetch, failed, fell through to serp-only push, state stayed empty for that URL — and the cost stayed at first-ingest level forever. The fallback now records the hash, so subsequent runs can short-circuit to skeleton-push.
- Fixed: end-of-run incremental-state save now writes to the same `stateKey`-scoped store key as the read (`hashes-<stateKey>`). Previously the final save wrote to the unscoped `hashes` key, which could leave the scoped key stale for short runs that didn't trigger the periodic save handler.

### 0.3.16 (2026-05-13)

- Fixed: the secondary detail-retrieval path returned empty data on searches where the primary path is unavailable, so items on those searches were tagged `dataCompleteness: "serp-only"` instead of the expected `"detail"`. Detail items now come back populated on those searches too.

### 0.3.15 (2026-05-13)

- Improved: detail retrieval throughput on multi-page result sets. Output content, ordering, `dataCompleteness` tagging, billing, and `maxResults` semantics are unchanged.

### 0.3.14 (2026-05-13)

- Fixed: on searches where the primary detail-retrieval path is unavailable, the secondary path (introduced in 0.3.13) was returning empty payloads and items fell back to `dataCompleteness: "serp-only"`. The secondary path now reliably retrieves full per-listing detail data, and items on those runs are tagged `dataCompleteness: "detail"` with detail-only fields populated (description, features, ratings, EV specs, contact, inspection dates).
- Items that the secondary path also cannot retrieve continue to be returned with search-result-level fields tagged `dataCompleteness: "serp-only"`, so consumers can filter by coverage.

### 0.3.13 (2026-05-12)

- Improved: when detail pages cannot be retrieved through the primary path, the actor now automatically falls back to a secondary retrieval path. Previously affected runs returned zero items; they now return per-listing details, just slower.
- Added: `dataCompleteness` output field on every item. Set to `"detail"` when a per-listing detail fetch succeeded, or `"serp-only"` when only search-result-level fields were available (last-resort partial coverage). Lets consumers filter or weight items by coverage.
- Last-resort fallback: if neither retrieval path can produce detail data for a run, items are returned with search-result-level fields (price, year, mileage, make/model, location, images, dealer) tagged `dataCompleteness: "serp-only"`, instead of failing the run with zero output.

### 0.3.12 (2026-05-12)

- Internal: all output paths now go through a single billing wrapper, and the run fails loudly at exit if the per-result charge count does not match the pushed item count. Architectural follow-up to 0.3.11 to prevent the missing-charge class of bug from recurring. No user-facing behaviour change.

### 0.3.11 (2026-05-12)

- Fixed: per-result billing was inconsistent on one fast list-only code path; result charges are now applied uniformly across all output paths. Pricing itself is unchanged (Actor Start + per Result, as published).

### 0.3.10 (2026-05-12)

- Improved: when a search cannot retrieve any listings, the run now aborts within seconds instead of waiting for the full retry budget to drain. Builds on 0.3.9's honest-exit behaviour.

### 0.3.9 (2026-05-12)

- Improved: runs that cannot retrieve any listings now stop early and report a clear failure instead of spending the full timeout returning an empty dataset. Previously some runs against transiently unavailable searches finished with 0 results but a "succeeded" status; they now end faster and with a status that reflects the outcome.

### 0.3.8 (2026-05-12)

- Fixed: startup failure that caused new runs to exit immediately with an incompatible browser runtime in the build environment. Existing configurations continue to work unchanged; no input changes required.

### 0.3.6 (2026-05-11)

- Added: `pushUnchangedSkeletons` input flag (default `false`). When combined with `incrementalMode: true`, the actor computes the SERP content hash up front and — for listings whose hash matches the previously stored value — pushes a minimal skeleton item (`url`, `portalUrl`, `externalId`, `contentHash`, `changeType: "UNCHANGED"`, `scrapedAt`) instead of enqueueing a detail fetch. Lets consumers do list-based sold-detection (listings absent from N consecutive runs = sold) without re-fetching every detail page each run. Detail-only fields stay null on skeletons; consumers should keep prior values via `COALESCE`-style merging.

### 0.3.5 (2026-05-11)

- Fixed: detail-page fetching regression introduced 2026-04-22 — under certain configurations the actor pushed search-result-only data directly to the dataset and skipped the per-listing detail fetch entirely. Detail-only fields (firstRegistrationDate, listPriceNewDkk, drivingRangeKm, batteryCapacityKwh, accelerationSec, ratingAverage, dealer, full description, equipment list) collapsed from ~99% fill rate to ~6%. Detail fetches now run on every applicable listing again.
- Added: `horsepowerHp` (number) and `torqueNm` (number) — parsed split of the existing `horsepower` string ("175/300 Nm" → 175 + 300; Tesla-style "-/750 nm" → null + 750). Raw `horsepower` string preserved for backwards compatibility.
- Added: `ratingLabel` (string) — Bilbasen's own merit label ("populær", "anbefalet").
- Improved: `features` array now populated from structured equipment data (20-90 entries per car) when available, instead of regex-mining the free-text description. Falls back to description bullets when equipment data is absent.

### 0.3.4 (2026-05-08)

- Added: full coverage of all 110 makes Bilbasen indexes (was: ~20 via guesswork). Embeds Bilbasen's canonical make list and resolves user input case-insensitively across hyphens, underscores, and spaces.
- Added: `?make=` query-param fallback for makes that have no working URL path slug (e.g. `Lynk & Co`). These makes are now filterable for the first time.
- Fixed: `make: "Alfa Romeo" / "Land Rover" / "Aston Martin"` previously resolved to hyphenated slugs (`alfa-romeo`, …) which Bilbasen treats as invalid and silently returns the unfiltered ~38k pool. Now correctly resolved to underscore form (`alfa_romeo`, `land_rover`, `aston_martin`).
- Fixed: `make: "Volkswagen"` resolved to `volkswagen` (Bilbasen returns fallback) instead of `vw` (the canonical slug).
- Improved: setting `model` for a make without path-form support (e.g. `Lynk & Co`) now throws a clear validation error at startup instead of silently returning all results for that make.

### 0.3.3 (2026-05-08)

- Fixed: `priceType: "Leasing"` (and any other filter) was dropped during paginated search-result retrieval. Filters are now preserved on every page.
- Fixed: `make: "Mercedes"` / `"Mercedes-Benz"` resolved to the wrong slug `mercedes-benz`; Bilbasen's actual path is `/brugt/bil/mercedes`. Alias table corrected.

### 0.3.2 (2026-04-16)

- Fixed: search and detail requests were intermittently rejected by the target. Runs now reliably retrieve data on first attempt.

### 0.3.1 (2026-04-16)

- Fixed: initial search and detail requests were being rejected. Runs now retrieve search results and detail data reliably from the start.

### 0.3.0 (2026-04-16)

- Added: 29 new output fields
  - EV data: drivingRangeKm, batteryCapacityKwh, energyConsumption, chargingAc/Dc/TimeDc
  - Performance: accelerationSec, topSpeedKmh, towWeight
  - Inspection: lastInspectionDate, nextInspectionDate
  - Model specs: listPriceNewDkk, vehicleCategory, vehicleTypeClass, cargoLiters, weightKg, widthCm, lengthCm, heightCm, payloadKg, maxTowWeightBrakedKg, drivetrain
  - Safety: hasAbs, hasEsp, airbagsNote
  - Ratings: ratingAverage, ratingCount, subRatings
  - Identifiers: syiId, tenant
- Added: 9 new filter inputs — mileageFrom, mileageTo, fuelType, gearType, bodyType, sellerType, zipCode, radiusKm, sortBy, includeEngrosCvr
- Added: direct listing-detail URL support (scrape a single listing without searching)
- Added: multiple startUrls per run (removed single-URL restriction)
- Added: optional `enrichFromDescription` flag — Danish NLP extracts serviceStatus, damageFlag/severity, warrantyStatus/expiry, registrationNumber, coarseRegion
- Improved: compact-mode now includes priceAmount
- Internal: pre-push gate validates dataset schema matches OutputItem; deploy wrapper syncs PPE pricing

### 0.1.x — 2026-04-14

- Added: `descriptionHtml`, `descriptionMarkdown` output fields (triple-format descriptions for RAG/LLM pipelines)
- Added: `contentHash` output field (SHA-256 signature of content-identifying fields)

### 0.1.x — 2026-04-14

- Added: cross-run repost detection (`isRepost`, `repostOfId`, `repostDetectedAt`)
- Added: `skipReposts` input to exclude detected reposts from output

### 0.2.20 (2026-04-13)

- Fixed: detail-page fetching was intermittently rejected. Detail pages now return reliably.

### 0.2.0 (2026-02-22)

- Filter-based search: make, model, year range, price range
- Smart input resolution (VW → Volkswagen, Mercedes → Mercedes-Benz, etc.)
- 40 output fields: pricing, vehicle specs, seller info, geo coordinates
- Compact output mode for AI and MCP workflows
- Incremental mode with change detection
- Detail enrichment: description, features, extended specs, seller contact
- Geo-ready output: lat/lng, city, zip code, region

### 0.1.0 (2026-02-01)

- Initial release
- URL-based search with detail page support
- Basic car listing extraction