-
First build: scrape Otodom.pl search results (sale/rent, flats/houses/etc.) from start URLs or structured filters.
-
Monitor mode with named KV store state: reports new, changed and removed listings (charges only changes).
-
Flat output with price, area, rooms, floor, advertiser, location, and timestamps; status/summary rows per run.
-
Output only real /pl/ad/ listing rows (investment/project and non-ad cards filtered); rows with missing price are still returned when the source card omits it.
-
Normalized dealType to sale/rent (matching input); normalized street (strips common Polish prefixes like ul./al.); floor keeps 0 for ground floor and null when unknown; documented buildYear/market/lat/lng as not available on search cards.
-
README now includes diverse real sample rows (sale/rent, agency/private, non-Warsaw) and monitor change output examples.
-
Monitor changed rows now include previousPriceAmount, previousPriceCurrency, and previousPricePerM2 (when present in stored state) alongside current price fields.
-
0.1.0 (2026-11-08): sale+room queries return 0 on Otodom (no sale-room market). Treat this known no-supply case as an ok/empty result with a clear reason instead of failing; added rent-room and sale-room fixtures/tests.
-
0.1 (2026-11-09): a sale+room query now stays a successful empty result even when Otodom answers 403/404 or with an unparsable shell page (the 2026-11-08 empty: 0 rows failures); real queries still report blocked/failed normally.
-
0.1 (2026-11-09): sale+room no-supply handling now also covers monitor mode (a blocked/404/parse-error page for a sale-room query is an ok empty result, not a failed run), and no-supply detection is decided by each source URL's deal/property path so a flat start URL is never misread as sale-room just because the input carries stale dealType/propertyType. Added fresh 2026-11-09 sale-room/rent-room fixtures and regression tests.
-
0.1 (2026-11-09): optional detail enrichment (fetchDetails, default true). Each /pl/ad/ search card opens its /pl/oferta/ page and merges GPS (lat/lng), buildYear, monthlyCharges (czynsz), heating, buildingType, buildingMaterial, windowsType, ownership, constructionStatus, market, features, photos and a plain-text description; fetchDetails: false keeps the fast search-card path. Seller contact data and e-mails/phones in descriptions are never returned. Added src/detail.py with fixture tests.
-
0.1 (2026-11-09): empty searches and quiet monitor runs now succeed (exit 0) with an empty/ok status row instead of failing; monitor zero-change runs stay exit 0. Added tests for empty search, monitor empty and monitor no-change.
-
0.1 (2026-11-10): default output drops listings without a published price (default sample has 0 null-price rows); set includeNoPrice: true to also return them. Added lat/lng+radiusKm inputs: centre coords add distanceKm to GPS rows, and with radiusKm only within-radius rows are returned (verified: Warsaw Ursus 2km radius returns only distanceKm <= 2 rows, central Warsaw 2km correctly returns none).
-
0.1 (2026-11-10): blocked/unparsable pages are never mistaken for an empty result. A blocked source now emits an explicit failed status row (or partial when earlier pages returned rows) and fails the run instead of an empty SUCCEEDED; in monitor mode a blocked run no longer reaches the diff, so it cannot mark tracked listings as removed or wipe the stored state. Investment junk rows stay out of monitor state/diffs (regression test on the live Warsaw fixture).
-
2026-11-10: Added showcase Store task for Warsaw flats for sale (.actor/store_task.json).
-
0.1 (2026-11-11): fetchDetails now defaults to false for faster, lower-cost search-card runs (set fetchDetails: true for offer-page enrichment). Every fetch uses the configured proxy, with residential PL escalation only for blocked datacenter requests.
-
0.1 (2026-11-11): monitor mode now tracks append-only priceHistory (last 20 observations) in stored state and exposes priceChangePct/priceDirection (up/down/flat/new) on change rows. Added droppedOnly input to emit only down-price changes while still tracking all changes, and summary rows now include dropped.
-
0.1 (2026-11-12): all fetches use the exposed proxy config with per-request datacenter→residential escalation on blocks (detail enrichment stays off residential), fetchDetails remains default off for lower cost, and blocked/unparsable pages are reported as failed/partial (never silent 403-as-empty).
-
0.1 (2026-11-11): added agencyPortfolio mode to group listings by agency/developer with counts, median price/m², listing URLs, and monitor tracking of new/removed stock (unchanged runs report no portfolio rows).
-
0.1 (2026-11-12): added postedWithinDays input (1/3/7/14/30) to filter recent listings in search and monitor runs. Structured searches add daysSinceCreated; startUrls get it when absent (preserved when present), and fixture-backed runs keep only in-window rows while existing inputs remain unchanged.
-
0.1 (2026-11-12): verified economics/reliability path: all requests use exposed proxy config with per-request datacenter-first fetches and residential escalation only after repeated blocks on that URL, fetchDetails defaults false so bulk runs stay search-card-only and detail enrichment never escalates, and 403/503 fetch failures are reported as failed/partial status rows (never silent empty success). Added fixture-backed regression tests.
-
0.1 (2026-11-12): added districts input to search multiple districts in one run (each as a location sub-path) sharing a single maxItems budget with cross-source dedupe and per-source status rows.
-
0.1 (2026-11-12): added advertiserType input filter (all/private/agency/developer, default all) mapped to Otodom ownerTypeSingleSelect on structured searches and startUrls (preserved when already present). Returned rows are filtered to the selected advertiser type when set; default behavior is unchanged.
-
0.1 (2026-11-13): ensured all fetches route through exposed proxy config with per-request datacenter→residential escalation on blocks (detail enrichment never escalates), fetchDetails stays default off, and blocked/unparsable/5xx pages are surfaced as failed or partial status rows (never silent empty rows); monitor state is preserved on blocked runs.
-
0.1 (2026-11-13): added sortBy input (default/newest/oldest/price_asc/price_desc/area_asc/area_desc) to control result order in search and monitor runs; structured/start URLs include Otodom by/direction params when possible and results are ordered before maxItems capping.
-
0.1 (2026-11-13): added pricePerM2Min/pricePerM2Max filters (PLN/m²) mapped to Otodom pricePerMeterMin/pricePerMeterMax for structured and startUrls searches, with post-filter on pricePerM2 (or derived priceAmount/areaM2 when missing) in search and monitor runs.
-
0.1 (2026-11-14): advertiserType, centre radius and pricePerM2Min/Max filters are applied while collecting search/monitor rows so paging continues until maxItems matching listings are gathered (up to page cap), preventing underfilled results when early pages contain non-matching rows.
-
0.1 (2026-11-14): added totalFloors to search rows and offer-page detail enrichment (building/storey count) for richer building context; exposed in dataset schema and README with real example output.
-
0.1 (2026-11-14): fetchDetails defaults to true for richer rows (GPS, build year, total floors, monthly charges and other offer-page details) via datacenter-only detail requests; set fetchDetails: false for fast search-card-only runs.
-
0.1 (2026-11-14): verified default fetchDetails: true remains datacenter-only for detail enrichment with newest-sample fixture coverage >=18/20 rows having non-null lat/lng/buildYear/monthlyCharges and asserted run cost <= $0.01/row; added newest-sample fixtures and regression test.
-
0.1 (2026-11-15): per-source _status rows and the run summary are stored in the run key-value store (STATUS and SUMMARY) instead of the dataset, so exports and the Output table contain only listing/change rows; dataset schema no longer includes _status.
-
0.1 (2026-11-15): added floorMin/floorMax filters (floor number, with -1 cellar/basement and 0 ground floor) mapped to Otodom floorNumberMin/floorNumberMax for structured and startUrls searches; listings are filtered while collecting (non-numeric floors excluded) so maxItems returns matching rows.
-
0.1 (2026-11-15): added buildYearMin/buildYearMax filters (construction year) for structured and startUrls searches. Listings are filtered by build year while collecting and after detail enrichment so capped runs still return matching rows.
-
0.1 (2026-11-16): added detail-only elevator, energyCertificate and availableFrom fields to listing rows (when published on the offer page), with tests, dataset view columns and README/output docs updated.
