# Changelog of Blocket Scraper: Car Monitor (`redomic/blocket-se-scraper`) Actor - **URL**: https://apify.com/redomic/blocket-se-scraper/changelog.md - **Full Actor documentation**: https://apify.com/redomic/blocket-se-scraper.md ## Changelog ### Unreleased - 0.1 (2026-11-15): write per-source status and run summary to the run key-value store (`STATUS`, `SUMMARY`) instead of dataset rows, so the dataset contains only listings. Market comparison now uses same make, model and model year within ±1 (and a mileage band when enough comparables exist) and leaves market fields empty when a narrow comparison is not possible. - 0.1 (2026-11-15): rewrite README as a buyer-focused sales page (clear who it is for, what you get, real output, plain reliability and pricing) and remove internal test/log terminology. - 0.1 (2026-11-15): add `includeRemoved` (default `true`) for monitor mode to suppress `removed` rows and report only `new`/`changed` alerts while keeping removals tracked in state. Input schema, monitor output/summary, README and tests updated. - 0.1 (2026-11-14): add `priceType` input filter (`fixed`, `auction`, `request`) for search/detail/monitor to keep only matching price types (e.g. fixed-price only), applied during collection and separated in monitor state. Input schema, tests and README updated. - 0.1 (2026-11-14): split dataset views so the first `Listings` view carries listing fields only (no monitor/status columns that are blank on default search), and move change/monitor and run-status columns to a second `Changes` view. `Full details` keeps all columns. Dataset schema and tests updated. - 0.1 (2026-11-14): add `listingAgeDays` (integer days since `postedAt`, `null` when unknown) to search/detail/monitor rows, computed from `postedAt` vs run time and preserved across monitor re-runs (including removed rows). Dataset schema and README updated. Tests in `tests/test_parser.py` and `tests/test_monitor.py`. - 0.1 (2026-11-14): fix `shippingOnly` for car searches by treating unknown shipping as `null` (not `false`) and keeping unknown rows when `shippingOnly` is set, so Blocket Bil queries like `volvo` return rows instead of empty; Torget explicit `shipping: false` rows are still excluded. Tests in `tests/test_filters.py` and `tests/test_parser.py`. - 0.1 (2026-11-14): apply `publishedWithinDays`, `onlyWithImages` and `shippingOnly` while collecting search/monitor pages (skip non-matching rows and keep paging, page cap unchanged) so `maxItems` still returns up to the requested number of passing rows. Tests in `tests/test_filters.py`. - 0.1 (2026-11-13): populate `shipping` from search JSON so default search rows are never blank; Torget keeps `true`/`false` from `shipping_exists`/`fiks_ferdig`, and car search rows default to `false` when no shipping signal is present. `shippingOnly` keeps only `shipping: true`. Tests in `tests/test_parser.py` and `tests/test_filters.py`. - 0.1 (2026-11-13): added `shippingOnly` (Blocket Frakt) filter and `shipping` (boolean/null) on listings (search/monitor/detail). Torget rows expose `shipping` from `shipping_exists`/`fiks_ferdig` and detail `eligibleForShipping`; car rows are `null` when unknown. `shippingOnly: true` keeps only `shipping: true` and monitor state is keyed by the filter. Tests in `tests/test_filters.py`. - 0.1 (2026-11-13): added `onlyWithImages` input to keep only listings with at least one photo (`imagesCount > 0`) in search/detail/monitor, filtered before market comparison, push and charge. Monitor state is keyed by the filter. Tests in `tests/test_filters.py`. - 0.1 (2026-11-12): monitor `changed` rows carry `priceDelta` (new minus previous price, same currency only) and every monitor row carries `firstSeenAt`, kept in the stored state across runs (listings stored before this change show null until next seen as new). - 0.1 (2026-11-12): added `publishedWithinDays` input to keep only listings published within the last N days (applied to `postedAt`) in search/detail/monitor, filtered before market comparison, push and charge. Monitor state is keyed by the recency filter. Tests in `tests/test_filters.py`. - 0.1 (2026-11-11): dealer inventory: a dealer page URL in `startUrls` or an ID in the new `dealerOrgIds` input returns that dealer's whole car stock (car search API filtered by `orgId`; live 11 Nov: dealer 7903992, 343 cars, every row `sellerId` 7903992). Monitor mode on it tracks a competitor's new, repriced and sold cars. Tests in `tests/test_dealer.py`. - 0.1 (2026-11-10): honor the `proxy` input by building requests through Apify proxy configuration with a fresh session per retry (direct only when `useApifyProxy` is false). Client tracks proxy request usage and rotates sessions on retries. Tests in `tests/test_proxy.py`. - 0.1 (2026-11-10): monitor mode ends SUCCEEDED on a quiet day (no new/changed/removed listings), so two unchanged runs report 0 new/0 changed/0 removed at $0 and exit successfully. Legal sign-off recorded (council 2026-11-10-0645, decision d480b24e4): business sellers only, no contact extraction, data use disclaimer in README. Showcase Store task `.actor/store_task.json` verified (search Blocket Bil cars by make and model, example `volvo`, 50 rows). Tests in `tests/test_monitor_quiet.py`, `tests/test_store_task.py`, `tests/test_legal.py`. - 0.1 (2026-11-10): showcase Store task `.actor/store_task.json` for the most-searched buyer use case (search Blocket Bil cars by make and model, example input `volvo`, 50 real rows): its own Store landing page with SEO title/description and the `overview` dataset view, published by `pipeline.store_task` on the next publish. Tests in `tests/test_store_task.py`. - 0.1 (2026-11-10): default search and monitor rows omit keys that are blank on every row (detail-only specs, monitor-only `changeType`/`previousPrice*` in a search run, market columns when no row in the run has a comparison), so a CSV export opens with no blank columns; detail mode and `scrapeDetails: true` still carry the detail specs, and the dataset schema's details view keeps them. Zeros stay (`marketComparables: 0`, `imagesCount: 0`). Tests in `tests/test_parser.py` and `tests/test_reliability.py`. - 0.1 (2026-11-10): runtime auto-partitioning past Blocket's 2,500-per-query ceiling. A capped search with `partition` on (default) and `maxItems` above 2,500 probes the real price range (`price_asc`/`price_desc`), splits the query into price bands (`src/urls.py:split_price_band`), recursively halves any band with more than 2,000 matches, collects and dedupes by ad ID, and the status row adds `partitioned`/`partitions`/`delta` plus an expected-vs-collected reason. `partition: false` keeps the old stop-at-2,500 behaviour; `maxItems` maximum rises to 10,000 so a run can ask for the extra rows. Real fixtures `car-volvo-page1` (21,076 matches) and `car-price-filter` (100,000–200,000 SEK band, 4,454 matches, ad `28097649`) back the README section; tests in `tests/test_partition.py`. - 0.1 (2026-11-09): server-side `mileageMin`/`mileageMax` (mil), `bodyType` (Swedish name→code map, e.g. Kombi→4) and `location` (region name→Blocket code, e.g. Stockholm→0.300001, or a region/city code) filters end to end, preserved from pasted search URLs, with monitor state keyed by the new filters; live fixtures `car-volvo-mileage-max2000`, `car-volvo-stockholm` and `car-volvo-stockholm-mileage` prove 0 rows over 2,000 mil and Stockholm-region-only rows; README moves Output before Input and adds a real filtered example with image. Tests in `tests/test_filters.py` and `tests/test_input.py`. - 0.1 (2026-11-09): legal sign-off recorded (council 2026-11-09-1845, decision d0a7fc6af): seller identity is business-only — `sellerName`/`sellerId` are emitted only for `Företag` sellers (`organisation_name`/`org_id`), `Privat` rows return `null` for both in search, Torget detail and car detail (detail fallback no longer matches breadcrumb `"name":"Bil"` and only runs for `Företag`, so dealer detail now returns e.g. `Brandt Bil - Vänersborg`), and no contact extraction anywhere (no phone/email/contact fields in any parsed row; login/reveal APIs never used). RED was build 0.1.4 only with 0 failures on 0.1.5-0.1.17, so the stale-prefill/retry fixes were not reworked. Re-battery from captured fixtures: all mapped battery search/detail fixtures parse to real rows (8/8 search fixtures 50 rows each, 3/3 detail fixtures 1 row each; fixture battery >=95%, live success clean on 0.1.5-0.1.17). Listing copy, pricing and `version` unchanged. Tests in `tests/test_legal.py`. - 0.1 (2026-11-09): drop stale `volvo` prefill in Blocket search-URL form too (`{"url": ".../search/car?q=volvo"}`, plain URL strings, or URL in `query`) when an explicit non-volvo `query` is present, so it cannot run as a second volvo search with the real query's filters (runs 7XqSK5cxAAj8oYrwE, YC7TffvReMLGzxyrU). Regression test reuses the bmw fixture. - 0.1 (2026-11-09): end-to-end regression tests run the exact RED inputs (7XqSK5cxAAj8oYrwE audi, YC7TffvReMLGzxyrU bmw) through the search path against live-captured fixtures and assert whole-SEK prices, no stale volvo prefill, and 10 rows. - 0.1 (2026-11-09): retry transient HTTP 403/429 with backoff before failing (parallel battery runs 7XqSK5cxAAj8oYrwE and YC7TffvReMLGzxyrU failed 0 rows simultaneously); live re-captures of both inputs now parse 50 rows. - 0.1 (2026-11-09): stale-prefill hardening for runs 7XqSK5cxAAj8oYrwE (audi/priceMin 100000.0 -> 400) and YC7TffvReMLGzxyrU (bmw/priceMax 150000.0 -> 400): the injected `volvo` default in `queries` ran with the real query's filters and failed. Drop `volvo/car` in dict and string form when an explicit non-volvo `query` is present, keep whole-SEK prices, and cover both failed inputs with fixtures. - In-run below-market valuation (no extra HTTP): new `src/market.py` adds `marketValue` (median of ≥5 same make+model priced rows in the run), `priceVsMarketSek`, `priceVsMarketPct`, `marketComparables`, `marketConfidence` and `redFlags` (motorfel/växellådsfel/körförbud/startar ej/krockskadad/reservdelar or under 35% of market); new inputs `compareToMarket` (default on), `onlyBelowMarketPct`, `onlyBelowMarketSek`, `includeRedFlagged`; stable keys in parser base row + dataset schema; real fixture example in README. Tests in `tests/test_market.py`. - Fix 0-row failures when Apify UI injects schema-prefill `volvo` into `queries` alongside an explicit `query`: drop the stale prefill so the real keyword/filters are used (runs 7XqSK5cxAAj8oYrwE, YC7TffvReMLGzxyrU). Tests cover both failed battery inputs. - Honest pricing and mileage: `priceAmount: 0` now returns `null` with `priceType` (`fixed`/`auction`/`request`, e.g. Klaravik auction `28107876` vs Torget free `28092346`), `mileage` is always mil with new `mileageKm` (`×10`, km sources divided by 10 with a year-plausibility guard), `salesForm` stays the Blocket code with new `salesFormLabel` (`Begagnad`/`Ny`/`Leasing`), `chassisNumber` is upper-cased, detail pages without JSON-LD recover their price from page targeting, `imagesCount: 0` is documented (seller uploaded no photos, e.g. dealer `28107688`), and README separates default search `null`s from detail/`scrapeDetails` richness with real fixtures for both. Captured volvo-recent + XC60/no-image detail fixtures. - Fix 400 on float prices: build_search_url/with_page emit whole-SEK ints, pasted page/API search URLs are converted/sanitized to API URLs, single `query` merges with `queries` so a stale schema default cannot swallow the keyword, and HTTP 400 fails fast without retry. Captured bmw/audi battery fixtures. ### 0.1 - First build. Search (Blocket Bil + Torget over the public JSON API, 50/page up to the 2,500/query ceiling with honest expected-vs-collected counts), detail mode (full listing pages: vehicle specs, VIN, inspections, equipment, description), monitor mode (`new`/`changed`/`removed` from `blocket-se-scraper-state`, unchanged items free). Pricing: $0.0025 per `result` and per `change` alert, plus $0.01 run start. Never-silent per-input status rows, flat stable schema with `null`s.