Commercialrealestate Properties and Details Scraper

This Actor is a modular parser for detailed Australian commercial property listing pages on Commercial Real Estate. It is designed to normalize property information, pricing, images, location, agent details, public property highlights, area metrics, leasing/sale signals, and NABERS/sustainability fields without putting the extraction logic in main.js.

Important source-use boundary

The official Domain Conditions of Use expressly includes the commercialrealestate.com.au website and CRE application. The conditions prohibit directly or indirectly scraping or indexing the Product, deploying robots or similar data-gathering methods, and using the Product to build a property database or derivative product. Because no applicable permission is recorded, this Actor defaults to authorizationConfirmed: false, reports SUCCEEDED / SKIPPED_POLICY, and makes zero source requests. It emits no policy diagnostic or placeholder dataset row. Set the flag to true only when the caller has applicable written authorization or a licensed feed arrangement that explicitly permits the intended automated use. This input is an attestation by the caller, not a bypass of the source terms.

When authorization is confirmed, the Actor still uses a conservative, sequential workflow: it checks robots.txt, follows only HTTPS URLs on the official host, applies a delay between requests, caps pages/listings/response bytes, sends no cookies or credentials, and stops on access failures. It never solves CAPTCHAs, bypasses WAFs, uses stealth/proxies, submits enquiry forms, or calls private APIs. The output is a data-integration aid; verify availability, price, listing status, and legal rights with the source and the relevant agent.

Input

• authorizationConfirmed (required for any request): written permission or a licensed feed must be in place.
• startUrls: authorized search or detail pages; defaults to the public for-sale landing page.
• maxItems and maxPages: hard output/request bounds.
• includeImages, includeDescription, and includeRawData: control optional public media, description, JSON-LD, and labeled source attributes.
• requestDelayMs, timeoutMs, and maxResponseBytes: conservative request controls.

Output

The dataset schema defines more than 65 possible normalized fields, including title/description, listing status/type, property ID, address/suburb/state/postcode/country, latitude/longitude, categories, price text/range/currency/period/unit, auction date/time/location, floor/land/NLA/gross/tenancy/available areas, parking/car spaces, occupancy, yield, rental income, lease term/expiry, zoning/development potential, features/highlights, images/floorplans/videos, agent names/phones/emails, agency details, inspections, last updated, NABERS overall/energy/water/indoor-environment ratings, sustainability text, nearby transport, map URL, raw JSON-LD, and source attributes. These are parser/schema capabilities, not verified live-source coverage: no listing or detail page was opened in the review and no authorized collection was run. Missing values stay null or []; the parser does not invent commercial facts. Dataset output contains property records only; failure codes and aggregate counts remain in the run summary.

Key-value records:

• SOURCE_METADATA: target URL, official Conditions of Use URL, policy text, and permission-gate explanation.
• OUTPUT_SUMMARY: uppercase Actor and run statuses, stored property count, authorization state, request/page counts, rejected URL count, and aggregate failure count.

Local verification

npm install
npm test
npm run lint
apify validate-schema
npm run validate-datasets

The default local run is expected to be policy-gated with no network requests. A live authorized run should only be performed by an authorized operator under the source agreement; this repository verification does not use login, Apify API, Apify push/call, private endpoints, or cloud Actor operations.

Runtime dependency remediation — October 3, 2026

This bounded implementation only uses JSON input, dataset appends, JSON key-value storage and (where needed) dataset counts. Its project-local src/platform.js now uses the official Apify REST storage contract in Cloud and SDK-compatible local JSON files, with Apify’s censoring logger. Removing the unused full SDK/Crawlee runtime removes its transitive crawler, cache, FTP and proxy dependency graph; it does not claim that those vulnerable paths were unreachable. Source permission gates, transport restrictions, output fields and historical storage are preserved. The adapter does not implement queues, migration, automatic retries or non-JSON storage; no such methods are used by this Actor. Tokens remain environment-only bearer headers to the fixed official API host.

The Docker build uses Node 24 and deterministic npm ci --omit=dev --ignore-scripts. npm run test:platform verifies local append/preservation, Cloud storage routing and authenticated write failures. Existing parser/policy tests remain necessary. Technical dependency remediation and a no-request gate run do not qualify live source extraction.