# Changelog of Housing.com Property Search Scraper (`codingfrontend/housing-property-search-scraper`) Actor

- **URL**: https://apify.com/codingfrontend/housing-property-search-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/codingfrontend/housing-property-search-scraper.md

## Changelog

All notable changes to the Housing.com Property Search Scraper are documented here.

### \[2.1.0] - 2026-08-31

#### Changed

- Direct access is now the default; paid proxies are explicit opt-ins and proxy setup failures stop the run.
- Replaced permissive/fallback input handling with strict Housing.com URL, type, proxy, and bounded-limit validation.
- Fixed the browser-context DOM fallback, removed fingerprint spoofing, raw API payloads, seller contacts, and debug page dumps.
- Requests are attempted once per URL, records are compacted/deduplicated and batch-written only after every requested URL succeeds, and zero/blocked results fail closed.
- Reduced production dependencies, added deterministic tests, current schemas/output links, and a lockfile-enforced Node 22 container.
- Fixed standard-proxy URL allocation by removing an invalid custom session identifier.

### \[2.0.13] - 2026-07-08

#### Added

- **Proxy rotation on WAF block**: if the page title is "Security Alert" (Housing.com WAF), the actor now closes the browser, acquires a fresh proxy URL, launches a new browser, and retries. Each URL gets up to 3 attempts with different RESIDENTIAL IPs.

#### Changed

- **Refactored `src/main.js`**: extracted `getFreshProxyUrl()`, `createBrowserSession()`, `isBlockedPage()`, `scrapeUrlWithRetry()` helpers. Each URL now runs in an isolated browser session that is torn down after completion, preventing cross-URL state leaks.

### \[2.0.12] - 2026-07-08

#### Added

- **Sort dropdown trigger**: some housing.com URLs only fire `SEARCH_RESULTS` after the user interacts with the sort bar. The actor now detects the sort bar by visible text ("Relevance", "Newest First", "Price: Low to High", "Price: High to Low") — no CSS class names — and clicks an inactive option first, then the previously-active one. This produces the two GraphQL calls needed to populate the listings.
- **Scroll-driven pagination loop**: after the first batch of properties is captured, the actor scrolls up to 1200px every 2.5s, stops when the property count stops growing for 3 consecutive rounds, and exits when `maxCrawlPerSearch` is reached or after a 90s budget.
- **Text-only selector helper** (`clickByText`): clicks an element by exact text via XPath; never depends on obfuscated CSS class names (e.g. `css-sfwdzm`, `css-c4ctc9`) which housing.com changes on every deployment.

### \[2.0.11] - 2026-07-08

#### Changed

- **Targeted API interception**: now specifically filters for the `SEARCH_RESULTS` GraphQL endpoint (`mightyzeus-mum.housing.com/api/gql/stale?apiName=SEARCH_RESULTS&emittedFrom=client_rent_SRP&isBot=false&platform=desktop&source=web&source_name=AudienceWeb`) and ignores other gql calls.
- **Default URL updated**: prefill is now `https://housing.com/rent/flats-for-rent-in-mumbai-india-Pskwz0ocdh7q42r5` (Mumbai rent SRP).
- **Direct fetch fallback**: if the React app fails to hydrate within the 25s wait, the actor issues a direct `fetch()` to the SEARCH\_RESULTS endpoint from the browser context (reusing accepted cookies and TLS fingerprint) before falling back to DOM extraction.
- **Property node picker** extracted into a `pickPropertiesNode` helper that tolerates wrapper shape changes (`data.searchResults.properties`, `data.properties`, etc.).
- Extended wait window from 20s to 25s to account for slower hydration on residential proxies.

#### Fixed

- API-call log now records the `apiName` query param for every intercepted response, making it easy to see at a glance which gql call was used (or missed) when results are zero.

### \[2.0.10] - 2025-05-21

#### Changed

- Complete architectural rewrite: dropped broken dropdown UI interaction in favour of GraphQL API interception + DOM fallback.
- Intercepts `mightyzeus-mum.housing.com/api/gql*` requests for structured JSON property data.
- DOM fallback: extracts SSR `a[href*="/rent/\d+-"]` listing links when API interception yields no data.
- Updated `INPUT.json` URL from invalid `/searches/RES-RENT-IN-MH-MBI` to valid `/in/rent/mumbai`.
- Fixed `Illegal return statement` syntax error caused by duplicate code in previous refactor.
- Added debug outputs (`debug_html`, `debug_screenshot`, `debug_api_logs`, `debug_api_response_snippet`) saved to KV store on zero-result runs.
- Build name changed to `housing-property-search-scraper` on Apify.

#### Fixed

- Removed old broken CSS dropdown selectors that no longer work on housing.com.

### \[1.0.0] - 2025-05-20

#### Added

- Initial production release of the Housing.com Property Search Scraper.
- `PlaywrightCrawler`-based architecture with residential proxy support.
- Modular project structure:
  - `src/main.js` — entry point with input validation, proxy setup, and crawler init.
  - `src/routes.js` — Crawlee router factory.
  - `src/routes/handlers.js` — search results page handler with pagination.
  - `src/extractors/index.js` — property data extraction from Housing.com SSR JSON.
  - `src/filters/index.js` — URL-based deduplication.
  - `src/schema/input.js` — input validation (searchUrls, maxItems, maxCrawlPerSearch).
  - `src/utils/index.js` — URL utilities and stealth helpers.
  - `src/errors/` — `ScraperError`, `ErrorTypes`, `ErrorHandler`.
- Multi-URL scraping: provide one or more Housing.com search page URLs.
- Property data: title, price, area, bedrooms, bathrooms, location, URL.
- `.actor/actor.json`, `input_schema.json`, `dataset_schema.json` for Apify platform compatibility.
- `validate-datasets.js` for local dataset output validation.
- `Dockerfile` based on `apify/actor-node-playwright-chrome`.
- `README.md` with full Apify usage guide.