Onthemarket Scraper

Features

This Onthemarket properties Scraper will enable you scrape any sale/rent listing from collection from onthemarket.co.uk.

You can simply take your listing url from browser and enter it into this actor. This actor will crawl through all pages of particular listing and generate dataset for you.

Listing url is something you get when you perform the search on onthemarket site. Example listing urls :

• https://www.onthemarket.com/for-sale/property/e20/
• https://www.onthemarket.com/for-sale/property/london/?radius=0.25
• https://www.onthemarket.com/to-rent/property/london/

This actor is not just scraper but also has monitoring capability. You can turn on monitoring mode and it will give you only newly added properties compared to your previous scrapes.

Input

For simple usecase, you just need to provide browser url of onthemarket search result page & that's all. You can leave other fields as they are to be sensible defaults.

Input example

{
    "listUrls": [
        {
            "url": "https://www.onthemarket.com/for-sale/property/e20/"
        }
    ],
    "propertyUrls": [
        {
            "url": "https://www.onthemarket.com/details/12297424/"
        }
    ],
    "fullScrape": true,
    "monitoringMode": false
}

You can either provide listUrls to search properties from or provide propertyUrls directly to crawl.

Understading monitoring mode :

• fullScrape : This option is by default turned on. When enabled it always force actor to scrape complete listing from all pagination pages regardless of monitoring is enabled or not.

• monitoringMode : This open when turned on will only scrape newly added property listings compared to previously scraped properties by this actor. It's important to turn off fullScrape setting if you are using this mode. If you keep fullScrape on, it will re-scrape complete listing again.

Output

The scraped data is stored in the dataset of each run. The data can be viewed or downloaded in many popular formats, such as JSON, CSV, Excel, XML, RSS, and HTML.

Output example

The result for scraping a single property like this:

{
  "id": "11192001",
  "url": "https://www.onthemarket.com/details/11192001/",
  "title": "3 bedroom semi-detached house to rent",
  "displayAddress": "Morford Street, Bath",
  "locality": "Bath",
  "bathrooms": 3,
  "bedrooms": 3,
  "agent": "Wrights Residential - Trowbridge",
  "agentPhone": "01225 616858",
  "propertyType": "Semi-detached house",
  "price": "£2,220 pcm",
  "secondaryPrice": "£512 pw",
  "furnishing": "Unfurnished",
  "coordinates": {
    "latitude": 51.388222,
    "longitude": -2.36315
  },
  "type": "rent",
  "summary": "PETS CONSIDERED! This three bedroom townhouse is situated within easy reach of Bath city centre. Features include a garage, parking for one vehicle, enclosed rear garden, large open plan kitchen/living room, downstairs shower room and en-suite shower room.",
  "features": [
    "Three bedroom town house",
    "Situated within easy reach of Bath city centre",
    "Garage",
    "Parking for one vehicle",
    "Enclosed rear garden",
    "Large open plan kitchen/living room",
    "Downstairs shower room",
    "En-suite shower room",
    "Part furnished or unfurnished",
    "Available late March"
  ],
  "keyInfo": [
    {
      "title": "Council tax",
      "value": "Unconfirmed"
    }
  ],
  "description": "PETS CONSIDERED! This three bedroom townhouse is situated within easy reach of Bath city centre. Features include a garage, parking for one vehicle, enclosed rear garden, large open plan kitchen/living room, downstairs shower room and en-suite shower room. Available late March, Part furnished or unfurnished.The property comprisesGround FloorEntrance Porch - 3' 10'' x 5' 10'' (1.18m x 1.77m)With composite front door and storage cupboard.Hallway - 17' 9'' x 10' 9'' (5.40m x 3.27m) maxWith double panel radiator, stairs to the first floor with storage cupboard under and PVCu double glazed back door.Shower RoomWith white suite comprising quadrant shower enclosure with electric shower, close coupled w.c and hand basin with vanity unit, fully tiled walls, heated towel rail, extractor fan and window to the rear.First FloorOpen plan kitchen/living room - 27' 8'' x 14' 9'' (8.43m x 4.49m) maxWith a range of eye level and base units, worktops with tiled splashbacks, sink/drainer, integrated electric oven and ceramic hob with extractor fan over, fridge freezer and washing machine, double panel radiator and PVCu double glazed windows to the front and rear.Second FloorLandingWith storage cupboard.Bedroom 1 - 9' 0'' x 11' 8'' (2.75m x 3.56m) maxWith double panel radiator and PVCu double glazed window to the rear. En-suiteWith white suite comprising quadrant shower enclosure with mains shower, close coupled w.c and hand basin with storage under, ceiling spotlights and extractor fan.Bedroom 2 - 11' 9'' x 8' 7'' (3.57m x 2.62m)With double panel radiator and PVCu double glazed window to the front.Bedroom 3 - 10' 0'' x 5' 11'' (3.05m x 1.81m)With double panel radiator and PVCu double glazed window to the front.BathroomWith white suite comprising bath with mains shower over, low level w.c and pedestal hand basin, fully tilled walls, heated towel rail and extractor fan.ExternallyTo the frontWith parking for one small vehicle in front of the garage.GarageWith power, lighting and up and over electric door to the front. To the rearThe pretty enclosed rear garden is mainly laid to gravel with a range of mature shrubs and trees. A gate providing access to the rear.",
  "images": [
    "https://media.onthemarket.com/properties/11192001/1435533190/image-0-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-1-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-2-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-3-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-4-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-5-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-6-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-7-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-8-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-9-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-10-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-11-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-12-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-13-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-14-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-15-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-16-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-17-1024x1024.jpg",
    "https://media.onthemarket.com/properties/11192001/1435533190/image-18-1024x1024.jpg"
  ],
  "schools": [
    {
      "name": "St Andrew's Church School",
      "distance": "0.2mi."
    },
    {
      "name": "Bath Academy",
      "distance": "0.3mi."
    },
    {
      "name": "St Stephens Church School",
      "distance": "0.5mi."
    }
  ]
}

Limitations

Since Onthemarket allows only 1000 properties per listing/search result, you might want to break down your listing urls into smaller area if it has more than 1K results. Good News is that even if multiple list urls contains overlapping results, they will get deduplicated within same run data.

For custom solutions

In case you need some custom solution, you can contact me : dhrumil@techvasu.com

Or learn more about me on github : https://github.com/dhrumil4u360