OnTheMarket Property Scraper (UK)

Scrapes houses, flats and land for sale or to rent from OnTheMarket — the UK's #3 property portal after Rightmove and Zoopla, and the one with by far the least scraper competition.

Public data only. No login, no cookies, no browser.

What you get

Two record types share one dataset, told apart by recordType.

PROPERTY — one row per listing

Search rows carry price, address, bedrooms, bathrooms, property type, key features, coordinates, agent and all photos. With Fetch full property details on (the default), each row also gets a propertyDetails object:

• the full description text
• EPC (energy performance) rating and certificate
• floorplans and additional documents
• keyInfo — tenure, council tax band, ground rent, service charge
• the complete photo set
• agent details and branch
• areaStats — local area context

SEARCH_SUMMARY — one row per location searched

Upstream's match total, how deep the run paged, which filters you requested next to what OnTheMarket says it actually applied, and whether the query hit the portal's row ceiling.

Input

Example

{
  "searchType": "for-sale",
  "propertyType": "houses",
  "locations": ["bristol", "bath", "cheltenham"],
  "minPrice": 300000,
  "minBedrooms": 3,
  "maxItems": 500,
  "includePropertyDetails": true
}

Two things this actor is honest about

1. OnTheMarket stops at ~1,020 rows per search — whatever it claims. A London search reports 50,000 matches but goes empty after page 34. This is the portal's limit, not the actor's, and no scraper can page past it. When a run hits it you get resultCapReached: true and a log line telling you to split the area:

WARN  for-sale/houses/bristol: upstream reports 1566 matches but OnTheMarket stops
      paginating at 1020 rows per query -- split the area into smaller locations,
      or narrow by price/bedrooms, to reach the rest.

That is exactly why Locations is a list — several narrow searches return far more data than one broad one.

2. A bad sort value returns zero results, not an error. OnTheMarket answers an unrecognised sort-field with HTTP 200 and an empty list, which would look like "no properties match" rather than a typo. This actor validates the sort before the first request and refuses the run instead, so you never get a silently empty dataset.

Every run also reports filtersRequested next to upstreamCriteriaText (OnTheMarket's own description of what it applied, e.g.

"£300k +, 3 beds +, House"

Notes on reliability

• No WAF challenge. Serves data cold on every TLS profile tested.
• Runs unattended. No human-captured session, no expiring token.
• Unknown locations fail cleanly as not_found — never silently-wrong nationwide results.
• Residential proxy by default on cloud runs.
• Failures never vanish. A listing withdrawn mid-crawl degrades that one row (_detailError); every input maps to at least one output row.

Output envelope

Every record carries _input, _source and _scrapedAt. Upstream field names pass through verbatim — no renaming. _source is S1-nextdata-search for listing-only rows and S1-nextdata-search+S2-nextdata-detail once details are attached.

Note location on a property row is upstream's {lat, lon}; the location you searched is locationQueried.

See CRAWLING_METHOD.md for the full reverse-engineering trail, the pagination-ceiling bisection and the sort-trap evidence.