Real Estate AU Scraper Pro

A locally staged, publish-valid Apify Actor that collects structured Australian real-estate agency records from public OpenStreetMap data. This Phase 1 build has not been pushed, deployed, or published.

Output

Each dataset item has a stable recordId, OSM node identity, agency name, address fields, coordinates, optional public contact fields, attribution, and a canonical detail URL. The URL is always constructed from source identity:

https://www.openstreetmap.org/{osmType}/{osmId}

The Actor never copies listing anchors. Missing source fields are null; values are not guessed.

Safe operating envelope

• Only five fixed Australian city bounding boxes are supported; arbitrary URLs and coordinates are rejected.
• The sources are public, documented read-only Overpass operators whose robots.txt does not disallow the API route, using OpenStreetMap data under ODbL 1.0.
• Requests are single-threaded, carry an identifying User-Agent, and have at least 1,000 ms between attempts.
• maxItems, maxRequests, maxRunSeconds, and maxCostCents are hard validated caps. Each network timeout is further shortened to the smaller remaining time/cost budget.
• HTTP errors, timeouts, invalid JSON, exhausted fallbacks, and zero valid records fail explicitly and write structured RUN_SUMMARY state.
• No credentials, cookies, proxy configuration, arbitrary targets, accounts, or secrets are accepted.
• Every output identifies © OpenStreetMap contributors / ODbL-1.0; downstream users remain responsible for attribution and share-alike obligations.

Public Overpass instances are community infrastructure with no SLA. This Actor deliberately stays far below the published small-use guidance (at most 12 requests/run and 500 records). Sustained or commercial high-volume operation should use a compliant self-hosted Overpass instance or regional extracts, not increased public-instance caps.

Example input

{
  "areas": ["sydney"],
  "maxItems": 100,
  "requestDelayMs": 1500,
  "maxRequests": 6,
  "maxRunSeconds": 180,
  "maxCostCents": 3,
  "requestTimeoutSeconds": 30,
  "includeContactDetails": true
}

Local verification

npm ci
npm run ship-gate
npm run verify:key
apify actor:run

npm run verify:key makes one bounded XML direct-ID request to the public VK Maps Overpass operator and independently rechecks the frozen five-record key. It does not import the Actor parser and does not rewrite the key. npm run score compares the Actor's separate JSON bounding-box path against that key and reports before/after accuracy per field.

The run summary is stored at key-value-store key RUN_SUMMARY. A successful run includes request counts, retries, bytes, rejected and emitted records, sleep time, elapsed time, estimated compute cost, and configured caps.

Source policy and license

• Overpass is the documented read-only alternative for OSM queries: https://operations.osmfoundation.org/policies/api/
• Public-instance list and operator guidance: https://wiki.openstreetmap.org/wiki/Overpass_API
• Attribution and ODbL terms: https://www.openstreetmap.org/copyright

No map tiles, Nominatim endpoint, HTML pages, forms, authentication, or personal/private sources are accessed.