# Changelog of Zillow Listings Scraper & Price Intelligence (`zapticx/zillow-market-intelligence`) Actor - **URL**: https://apify.com/zapticx/zillow-market-intelligence/changelog.md - **Full Actor documentation**: https://apify.com/zapticx/zillow-market-intelligence.md ## Changelog Every push auto-increments the **build number** shown as `.N` regardless of change size, so it isn't a reliable "what changed" signal on its own — this file is. Entries below are grouped by feature batch, not exact build number. Version `0.3` is the current line; `0.1` was the beta. ### 0.3 #### Fixed - **Single-property (`targetUrl`) lookups no longer fail on Zillow Showcase listings.** Zillow renders Showcase listings (its paid premium listing format) from a `ViewShowcasePriorityQuery` GraphQL operation instead of `ForSalePriorityQuery`, and the detail parser matched only the latter — so those lookups failed outright with `ParseError`. Measured live, Showcase covers 17% of Miami Beach for-sale listings and 51–71% in Naples, Austin, Charlotte and Scottsdale. Detail-page parsing now selects any cache entry holding a usable `property`, still preferring `ForSalePriorityQuery`, and cross-checks the zpid against the requested URL so a foreign property can never be substituted. Search (`location` / `searchUrl`) runs are unaffected. #### Added - `detailSource` on `target_listing` records (`for_sale_priority`, `showcase`, or `other`). Showcase pages do not carry `propertyTaxRate`, `monthlyHoaFee`, `daysOnZillow`, `yearBuilt`, lot size, `brokerName` or `imageUrl` at all; those stay `null`, and `detailSource` lets consumers tell "Zillow publishes no HOA here" from "this projection never carries one". Nothing is fabricated or defaulted to zero. ### Store-readiness and data-completeness pass (v0.2) **Fixed a whole class of silently-empty output fields.** Zillow's search results carry price history, Zestimate, and rent-Zestimate data inside `hdpData.homeInfo`, *not* at the result item's top level. The normalizer read only the top level, so on every real run: - `rentZestimate` was always `null` — live data present on 14 of 41 listings. - `priceChange` was always `null` — present on 12 of 41. - `datePriceChanged` was always `null` — present on 12 of 41. - `priceReduction` was always `false`, **including for the 10 of 41 listings Zillow was actively advertising a price cut on**. That was the worst of the four: not a missing value but a confidently wrong one. Root cause was a hand-written test fixture built to a guessed payload shape rather than a captured one — it placed all four fields at the top level, so the tests passed while production returned nothing. The fixture now matches a real, live-captured Zillow response, and the normalizer reads top level first with a `homeInfo` fallback, matching the pattern already used for address and geo fields. - `datePriceChanged` is now normalized from Zillow's epoch-millisecond form to an ISO `YYYY-MM-DD` date. The UTC date of those epochs matches the labels Zillow itself renders ("Jun 5", "Aug 27", "Aug 11" — checked against three live listings). - `priceReduction` is now a real boolean derived from Zillow's own reduction marker or a negative `priceChange`, instead of a constant `false`. - `rentZestimate` and `priceReduction` were added to the default "Listing intelligence" dataset view, now that they carry real data. - Confirmed by the same live check that `yearBuilt` is genuinely absent from Zillow's search payload (0 of 41, top level and `homeInfo`). It stays `null` on search rows by design and is now documented as such rather than looking like an extraction bug. **Market summary statistics.** - Added `minimumPrice`, `maximumPrice`, `averagePrice`, and an explicit `listingCount`. - **Fixed a biased median.** `medianPrice` and `medianDaysOnZillow` were computed only over the price-per-sqft-eligible subset, so a listing with a real asking price but no living area (a lot, some new construction) was dropped from a *price* median for a reason that only concerns price *per square foot*. Each statistic family now uses its own eligibility. - Every statistic now publishes its sample size — `priceSampleSize`, `pricePerSqftSampleSize`, `daysOnZillowSampleSize` — so the three different bases can never be silently conflated. - `market` now echoes the filters that shaped the sample (`priceMinFilter`, `priceMaxFilter`, `bedsMinFilter`, `bathsMinFilter`, `maxResultsRequested`), so a `MARKET_SUMMARY` read on its own can no longer be mistaken for whole-market statistics. **Single-property runs.** `address` is now composed from the detail page's published components, so it is populated on `target_listing` rows instead of leaving a permanently-blank column in the "Target listing" dataset view. **Input form.** Regrouped into Search / Filters (optional) / Advanced with section descriptions; `maxResults` moved up beside `location` so the basic path — location, count, Start — is the first thing on screen. Every field has a title and description; price fields gained upper bounds. No field was renamed or removed, so existing inputs and Tasks keep working unchanged. **Schemas.** Every dataset field now has a title, and the ones whose availability differs by run mode or by Zillow's own publishing carry a description saying so. New tests assert the dataset schema matches the emitted model exactly in both directions, that each view projects and displays the same field list, and that no view shows a column its record type never populates. **README.** Added a quick start, a field-availability table stating what Zillow does and does not publish per mode, runnable Python/JavaScript/cURL examples, and a support section. Corrected a pricing claim of a `$0.00005` Actor-start charge that does not exist in the live pricing (the only charge is $0.002 per listing), and removed a stale limitation describing single-property mode as unimplemented when it had already shipped. ### Target-listing mode - New input mode: `targetUrl` — a single `zillow.com/homedetails/...` listing URL. Fetches and normalizes just that one property (address, price, sqft, beds/baths, Zestimate, tax-assessed value, and two fields not available from search results at all: `propertyTaxRate` and `monthlyHoaFee`, both real observed data from the detail page, not estimates) as one `target_listing` record. No comparable-set intelligence and no `market_summary` — there's nothing to aggregate for a single property. Billed through the same `listing` PPE event as a search result. - Existing `location`/`searchUrl` search behavior is fully unchanged — this is strictly additive, a third mutually-exclusive input mode alongside the existing two. - **Real bug found and fixed via live testing across three different properties**: the detail page's top-level `lotSize` + `lotAreaUnits` fields are mismatched — `lotAreaUnits` reported `"Acres"` every time, even when `lotSize`'s raw number was clearly on a square-foot scale (e.g. `32845`, actually a ~0.75-acre lot in sqft, not 32,845 acres). Fixed by parsing the self-consistent formatted string `resoFacts.lotSize` (e.g. `"0.75 Acres"`) instead of trusting that field pair. - New dataset view: "Target listing", alongside the existing "Listing intelligence" view. ### Billing, correctness, and Store-polish pass - **Fixed a real performance bug found via live testing at `maxResults: 820`:** the run took 82 seconds. Root cause was calling `push_data` once per listing after PPE billing went live — each call is a separate awaited network round-trip to charge that one event, so 820 listings meant 820 sequential round-trips. Fixed by batching listings into chunked `push_data` calls (200 at a time), which charges the whole batch in one API call. Also fixed a smaller, real-but-not-dominant O(n²) comparable-tier lookup (was re-filtering the whole eligible population per listing; now a precomputed O(1) dict lookup). Combined: 82s → **21.3s** for the same 820-listing run, verified on the real platform, not estimated. - Wired real pay-per-event billing (`src/billing.py`): one `listing` charge per unique listing, `market_summary` always free. - Fixed the PPE event display title so the Store's auto-pluralized pricing line reads correctly ("Listing" → "$0.002 / listing", not "listing extracteds"). - `market_summary` is now written **first**, before listing rows, so it's on page 1 of the dataset and the "Market summary" view instead of buried after up to 820 listing rows. - Added `minimum`/`maximum` bounds to `priceMin`, `priceMax`, `bedsMin`, `bathsMin` in the input schema, and tightened the mutual-exclusivity wording for `location`/`searchUrl`. - Enriched `dataset_schema.json`'s `intelligence`/`comparisonQuality` fields with real enums (`confidence`, `comparableTier`, `pricePosition`) matching the Python models, and a `recordType` enum — the strongest per-field validation the platform's flat (non-discriminated) dataset schema format supports. - README: fixed "dated" → "current" wording, added an explicit "not affiliated with Zillow" disclaimer, a "what comparable means" clarification near the top, a sample-selection-bias callout, and a Troubleshooting section. - Categories set to Real Estate; memory tuned from the platform default 4096MB down to 256MB (measured peak usage ~52–60MB). ### Initial beta build - Initial working build: search collection, normalization, comparable-tier price/Zestimate/DOM intelligence, market summary aggregation. - Added `output_schema.json` and `dataset_schema.json` with table views. - Fixed a real bug caught via live smoke testing: home-type filters were silently ignored when `maxResults` was small enough to be satisfied by the unfiltered region-resolution request. - Full input-schema section groupings and enum titles for the Console form. ### Scaffold - Actor scaffolded: Dockerfile, input schema, core module structure.