# Harcourts Scraper \[~$1.5/1K💰] | Sold & Rent | Agent Emails (`ahmed_jasarevic/harcourts-scraper`) Actor

Scrape Harcourts Australia property listings — buy, sold, rent, leased. Address, suburb, price, beds/baths/cars, agent, office, photos, floor plan. Full detail mode harvests agent emails with zero extra requests. Pure HTML parsing, no browser. From ~$1.5 per 1,000 listings (details +$0.50/1K).

- **URL**: https://apify.com/ahmed\_jasarevic/harcourts-scraper.md
- **Developed by:** [Ahmed Jasarevic](https://apify.com/ahmed_jasarevic) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.47 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Harcourts Scraper

Cheerio-based scraper for **Harcourts Australia** property listings
(`harcourts.net` / `harcourts.com.au`). Parses only the **server-rendered HTML** —
no internal APIs, no `__NEXT_DATA__`-style JSON blobs, no browser.

Works for **buy**, **sold**, **rent** and **leased** listings, honours search
filters, paginates automatically, and fans out to every property detail page.

### How it works

1. Each input URL is classified: search page, individual listing, or office page.
2. Search pages are walked page-by-page (`?page=N`) by following the
   `<a rel="next">` pagination link — filters are preserved automatically.
3. Listing cards are parsed with Cheerio from `.property-listings-results`.
4. Each property page is fetched (concurrently) and parsed from visible HTML:
   address, beds/baths/cars, price, description, agents, office, photos, floorplan.
5. Rows are emitted as **JSONL**, **JSON**, or **CSV** (optionally flattened).

> Note: `harcourts.com.au` is a client-side-rendered shell that serves the same
> homepage HTML for every path. The scraper therefore normalizes all URLs to the
> fully server-rendered `harcourts.net` domain, which backs the same data.

### Install

```bash
npm install
```

### Usage

```bash
## Scrape a full search (pagination + filters honoured)
node src/cli.js "https://harcourts.net/au/listings/buy"

## Filtered search (suburb, bedrooms, price) → CSV
node src/cli.js "https://harcourts.net/au/listings/buy?keywords=melbourne&bedrooms=2&minPrice=500000" \
  --max-items 50 --format csv --out melbourne.csv

## Rentals
node src/cli.js "https://harcourts.net/au/listings/rent" --max-items 20 --format jsonl --out rents.jsonl

## Single listing
node src/cli.js "https://harcourts.net/au/office/your-place/listing/l42663019-15-tate-street-ropes-crossing-nsw-2760"

## Office page → all its listings
node src/cli.js "https://harcourts.net/au/office/rata-co" --max-items 10
```

#### Options

| Option | Default | Description |
|---|---|---|
| `--max-items N` | `100` | Cap on properties collected |
| `--concurrency N` | `16` | Parallel fetches |
| `--delay-ms N` | `100` | Delay between requests (be polite) |
| `--listings-only` | off | Search pages only, no per-property visits (fast) |
| `--proxy URL` | `HARCOURTS_PROXY` | HTTP proxy URL |
| `--out PATH` | `harcourts-results.jsonl` | Output file |
| `--format FMT` | `jsonl` | `jsonl`, `json` or `csv` |
| `--no-flatten` | off | Keep nested JSON instead of flat underscore keys |

### Output

One row per property.

**Default (`includeDetails=false`, listings-only)** — built from search cards:

```
portal, listingId, canonicalUrl, listingMode (sale/rent/sold/leased), headline,
address { displayAddress, streetAddress, suburb, state, postcode },
bedrooms, bathrooms, carspaces, priceDisplay, uuid,
photos[1] (thumbnail), photosCount, agents[1] { name }, office { name, slug }
```

**`includeDetails=true` (full)** — same fields plus property page content:

```
propertyId, propertyType, description, photos[all], photosCount, floorPlanUrl,
studyRooms, office.profileUrl, agents[] { name, office, profileUrl, photo },
harvestedEmails[] — contact/inspection emails found on the listing page (TenApp booking,
mailto, plain) with zero extra requests
```

With flattening on (default), `address_*` and `office_*` become top-level columns
and arrays become `*_json` strings.

### Monetization (pay-per-event)

The actor uses usage-based billing with two separate events (defined in `main.js`).
The `includeDetails` switch changes BOTH the scraping strategy and the pricing:

- **`includeDetails = false` (default, "listings-only")** — scrapes search pages only
  and builds each row from the cards: address, price, beds/baths/carspaces, agent,
  thumbnail, listing URL. No per-property requests → very fast (seconds to minutes)
  and cheap. Charged **1 × `RESULT`** per property.
- **`includeDetails = true` (full detail)** — visits every property page and adds
  description, photo gallery, floor plan, agents, property type, uuid — **plus
  `harvestedEmails` (contact/inspection emails pulled from the listing page HTML with
  zero extra requests) at no extra charge — a key selling point over comparable
  actors** (competitors bill email enrichment separately). Charged only
  **1 × `RESULT` + 1 × `PROPERTY_DETAILS`** per property.

Set up in the Apify Console (Monetization tab → Events) and set the pricing model to
**pay-per-event**:

1. **`RESULT`** — charged for every scraped property, regardless of mode.
   Suggested $0.002 per event.
2. **`PROPERTY_DETAILS`** — additional charge per property **only when
   `includeDetails = true`**. Suggested $0.002 per event (full detail then costs
   $0.004/listing and bundles description, full gallery, floor plan, agent photos,
   property id/uuid and harvested emails).

Note: the events must exist in the Monetization tab before billing. If they are
missing, charging is skipped with a warning and the run still completes.

### Tests

Parser tests run against captured HTML fixtures (no network):

```bash
npm test
```

### Notes

- Respect the site and the robots expectations: keep `--delay-ms` reasonable and
  only scrape what you need.
- Scraped data remains subject to Harcourts' terms of use and applicable law
  (including the Australian Privacy Act 1988 for agent contact info).

# Actor input Schema

## `startUrls` (type: `array`):

Harcourts URLs. Supports search pages (auto-paginated, filters honoured), individual listing pages, and office pages (fans out to that office's listings).

## `maxItems` (type: `integer`):

Hard cap on the number of properties collected.

## `maxConcurrency` (type: `integer`):

Maximum number of pages (detail + search) fetched in parallel.

## `minConcurrency` (type: `integer`):

Minimum number of detail pages fetched in parallel.

## `maxRequestRetries` (type: `integer`):

Number of retries before a failed request is given up.

## `delayMs` (type: `integer`):

Delay between requests in milliseconds. Set to 0 for maximum speed (not recommended below 50).

## `includeDetails` (type: `boolean`):

OFF (default): listings-only - scrapes search pages only (address, price, beds/baths/cars, agent). FAST and cheap. ON: visits every property page for full data (description, photos, floor plan, agents) and charges PROPERTY\_DETAILS usage events per property. Requires pay-per-event pricing.

## `proxyConfiguration` (type: `object`):

Select proxies to be used by your crawler.

## `flatten` (type: `boolean`):

Flatten nested fields (address\_*, office\_*) into top-level columns and JSON-stringify arrays (agents\_json, photos\_json) for CSV-friendly output.

## Actor input object example

```json
{
  "startUrls": [
    "https://harcourts.net/au/listings/buy"
  ],
  "maxItems": 1000,
  "maxConcurrency": 16,
  "minConcurrency": 1,
  "maxRequestRetries": 5,
  "delayMs": 100,
  "includeDetails": false,
  "proxyConfiguration": {
    "useApifyProxy": true
  },
  "flatten": true
}
```

# Actor output Schema

## `dataset` (type: `string`):

Dataset containing all scraped property listings

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://harcourts.net/au/listings/buy"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("ahmed_jasarevic/harcourts-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://harcourts.net/au/listings/buy"],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("ahmed_jasarevic/harcourts-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://harcourts.net/au/listings/buy"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call ahmed_jasarevic/harcourts-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,ahmed_jasarevic/harcourts-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/dbQ8Geydl6nUR4zlx/builds/hluM9bd8D7E0jDtWx/openapi.json