Harcourts Scraper

Cheerio-based scraper for Harcourts Australia property listings (harcourts.net / harcourts.com.au). Parses only the server-rendered HTML — no internal APIs, no __NEXT_DATA__-style JSON blobs, no browser.

Works for buy, sold, rent and leased listings, honours search filters, paginates automatically, and fans out to every property detail page.

How it works

1. Each input URL is classified: search page, individual listing, or office page.
2. Search pages are walked page-by-page (?page=N) by following the <a rel="next"> pagination link — filters are preserved automatically.
3. Listing cards are parsed with Cheerio from .property-listings-results.
4. Each property page is fetched (concurrently) and parsed from visible HTML: address, beds/baths/cars, price, description, agents, office, photos, floorplan.
5. Rows are emitted as JSONL, JSON, or CSV (optionally flattened).

Note: harcourts.com.au is a client-side-rendered shell that serves the same homepage HTML for every path. The scraper therefore normalizes all URLs to the fully server-rendered harcourts.net domain, which backs the same data.

Install

$npm install

Usage

# Scrape a full search (pagination + filters honoured)
node src/cli.js "https://harcourts.net/au/listings/buy"

# Filtered search (suburb, bedrooms, price) → CSV
node src/cli.js "https://harcourts.net/au/listings/buy?keywords=melbourne&bedrooms=2&minPrice=500000" \
  --max-items 50 --format csv --out melbourne.csv

# Rentals
node src/cli.js "https://harcourts.net/au/listings/rent" --max-items 20 --format jsonl --out rents.jsonl

# Single listing
node src/cli.js "https://harcourts.net/au/office/your-place/listing/l42663019-15-tate-street-ropes-crossing-nsw-2760"

# Office page → all its listings
node src/cli.js "https://harcourts.net/au/office/rata-co" --max-items 10

Options

Option	Default	Description
--max-items N	100	Cap on properties collected
--concurrency N	16	Parallel fetches
--delay-ms N	100	Delay between requests (be polite)
--listings-only	off	Search pages only, no per-property visits (fast)
--proxy URL	HARCOURTS_PROXY	HTTP proxy URL
--out PATH	harcourts-results.jsonl	Output file
--format FMT	jsonl	jsonl, json or csv
--no-flatten	off	Keep nested JSON instead of flat underscore keys

Output

One row per property.

Default (includeDetails=false, listings-only) — built from search cards:

portal, listingId, canonicalUrl, listingMode (sale/rent/sold/leased), headline,
address { displayAddress, streetAddress, suburb, state, postcode },
bedrooms, bathrooms, carspaces, priceDisplay, uuid,
photos[1] (thumbnail), photosCount, agents[1] { name }, office { name, slug }

includeDetails=true (full) — same fields plus property page content:

propertyId, propertyType, description, photos[all], photosCount, floorPlanUrl,
studyRooms, office.profileUrl, agents[] { name, office, profileUrl, photo },
harvestedEmails[] — contact/inspection emails found on the listing page (TenApp booking,
mailto, plain) with zero extra requests

With flattening on (default), address_* and office_* become top-level columns and arrays become *_json strings.

Monetization (pay-per-event)

The actor uses usage-based billing with two separate events (defined in main.js). The includeDetails switch changes BOTH the scraping strategy and the pricing:

• includeDetails = false (default, "listings-only") — scrapes search pages only and builds each row from the cards: address, price, beds/baths/carspaces, agent, thumbnail, listing URL. No per-property requests → very fast (seconds to minutes) and cheap. Charged 1 × RESULT per property.
• includeDetails = true (full detail) — visits every property page and adds description, photo gallery, floor plan, agents, property type, uuid — plus harvestedEmails (contact/inspection emails pulled from the listing page HTML with zero extra requests) at no extra charge — a key selling point over comparable actors (competitors bill email enrichment separately). Charged only 1 × RESULT + 1 × PROPERTY_DETAILS per property.

Set up in the Apify Console (Monetization tab → Events) and set the pricing model to pay-per-event:

1. RESULT — charged for every scraped property, regardless of mode. Suggested $0.002 per event.
2. PROPERTY_DETAILS — additional charge per property only when includeDetails = true. Suggested $0.002 per event (full detail then costs $0.004/listing and bundles description, full gallery, floor plan, agent photos, property id/uuid and harvested emails).

Note: the events must exist in the Monetization tab before billing. If they are missing, charging is skipped with a warning and the run still completes.

Tests

Parser tests run against captured HTML fixtures (no network):

$npm test

Notes

• Respect the site and the robots expectations: keep --delay-ms reasonable and only scrape what you need.
• Scraped data remains subject to Harcourts' terms of use and applicable law (including the Australian Privacy Act 1988 for agent contact info).