Scrape normalized US Redfin listings — for-sale, sold, and pending — with full MLS/geo field coverage and reliable pagination past Redfin's 350-result cap via price-bucket bisection, deduped by property ID.
Real crawler implemented end to end: src/resolver.py (pure RSC gis
query-string extraction tolerating single-/double-escaped /,
ExpectedGeo/ResolvedRegion DTOs, two-signal passes_geo_guard()
for REQ-3's geo-splash defense), src/client.py (the only module
importing curl_cffi — RedfinSession with browser-profile rotation,
408/429/503/network-error retry with 2s→30s capped exponential
backoff, session_id rotation, {}&&-prefix-stripping + resultCode
validation for the gis envelope), src/paginator.py (price-bucket
bisection past Redfin's 350-home server cap with mpt=99, propertyId
dedup via a shared seen_ids set, per-bucket fault isolation),
src/parser.py (pure homes[] dict → ResultRow mapping, never
fabricating a value), and src/main.py (SDK wiring: actor-start/
result-row PPE charging, per-entry geo-guard rotate-retry-once-then-
skip, per-entry fault isolation, REQ-13 fail-loud on zero rows).
Live-confirmed 2026-08-12 (real network fetches against redfin.com):
the RSC regex, the {}&& envelope prefix, the 350-home cap, and the
mpt=99 bisection requirement all match spec.md's wire-format claims
verbatim. Additionally confirmed empirically during implementation
that status="sold" maps to sold_within_days=90 (with sf absent
from the query) — spec.md's Open Questions item 1 for sold is now
resolved. The pending status mapping remains an unconfirmed
best-effort placeholder (STATUS_PARAMS["pending"], see the inline
comment in paginator.py) after several live probes returned either
the for-sale-shaped default or a bot-challenge page — paginator/parser
stay permissive so a wrong guess degrades gracefully, never crashes.
Also discovered live that soldDate/openHouseStart/openHouseEnd
arrive as epoch-ms integers, not the ISO YYYY-MM-DD strings spec.md
assumed — parser._epoch_ms_or_str_to_iso() handles both shapes.
62 unit/integration tests (tests/test_resolver.py,
tests/test_client.py, tests/test_paginator.py,
tests/test_parser.py, tests/test_main.py) — all pure/mocked, no
live network in CI. ruff check clean.
0.0.1 — 2026-08-12
Scaffolded: .actor/ (actor.json, input/output/dataset schemas, PPE,
Dockerfile), src/models.py fully implemented (ActorInput + ResultRow
per spec, search_urls-or-region-triple validator, RESIDENTIAL-proxy
rejection), placeholder src/main.py (boots, validates input, pushes one
{"placeholder": true, ...} row, exits — no real crawler), tests, README
skeleton.
Real implementation (src/resolver.py, src/client.py,
src/paginator.py, src/parser.py) is pending — RSC gis extraction,
price-bucket bisection past the 350-home cap, and the two-signal
geo-splash guard are all live-confirmed wire format per
docs/specs/redfin-property-listings-scraper/spec.md ("Assumptions") but
not yet wired up; see docs/specs/redfin-property-listings-scraper/tasks.md
T02–T12.