AU Real Estate Scraper Pro

A local, publish-ready Apify Actor that collects structured Australian real-estate agency records from public OpenStreetMap data. This Phase 1 build is staged locally only; it has not been pushed, deployed, or published.

What it returns

Each dataset item has a stable recordId, OSM element identity, agency name, address fields, coordinates, optional public contact details, attribution, and a canonical detail URL. The URL is always constructed from the returned element type and ID:

https://www.openstreetmap.org/{osmType}/{osmId}

No listing-page anchors are read or copied. Missing source fields are represented as null, not guessed.

Safe operating envelope

• Only five fixed Australian city bounding boxes are supported; callers cannot supply arbitrary targets.
• The Actor uses public Overpass API endpoints and OpenStreetMap data under ODbL 1.0.
• It sends one request at a time with at least 1,000 ms between requests.
• maxItems, maxRequests, maxRunSeconds, and maxCostCents are hard validated caps.
• Upstream HTTP errors, invalid JSON, exhausted fallbacks, and zero valid records fail explicitly and write RUN_SUMMARY with status: FAILED.
• The Actor accepts no credentials, tokens, cookies, proxy configuration, or arbitrary URLs.
• Output attribution is © OpenStreetMap contributors; downstream users remain responsible for ODbL attribution/share-alike obligations.

Public Overpass instances are suitable for small, bounded use but are community infrastructure. For sustained high-volume use, operate a compliant private Overpass instance or use regional OSM extracts rather than increasing these caps.

Input example

{
  "areas": ["sydney"],
  "maxItems": 100,
  "requestDelayMs": 1500,
  "maxRequests": 6,
  "maxRunSeconds": 180,
  "maxCostCents": 3,
  "requestTimeoutSeconds": 30,
  "includeContactDetails": true
}

Local verification

npm ci
npm test
npm run score
npm run validate
apify actor:run

npm run verify:key independently rechecks the frozen answer key through OSM's XML element API. It is intentionally separate from the Actor's Overpass JSON path and does not rewrite the key.

The run summary is stored at key-value-store key RUN_SUMMARY. A successful run includes request counts, retries, bytes, rejected and emitted records, sleep time, elapsed time, estimated compute cost, and the configured caps.

Source and legal notes

Data © OpenStreetMap contributors, available under the Open Database License 1.0. The Actor uses documented machine interfaces and does not crawl OSM HTML pages. Respect the public instance resource limits and attribution requirements.