Contact

If you encounter any issues or need to exchange information, please feel free to contact us through the following link: My profile

Professional Onthemarket.com Real Estate Data Extraction Solution

The UK property market generates millions of listings annually, making manual data collection from property portals an overwhelming task for real estate professionals, investors, and researchers. OnTheMarket.com stands as one of the UK's most user-friendly property portals, featuring hundreds of thousands of properties for sale and rent from leading estate agents. As the UK's fastest-growing property portal since its acquisition by CoStar Group, OnTheMarket provides a rich source of property data that can drive informed decision-making in real estate investments, market analysis, and competitive research.

The challenge lies in efficiently extracting and organizing this valuable property information at scale. Traditional manual browsing and copy-pasting methods are time-consuming and prone to errors, especially when dealing with thousands of property listings across different regions. This is where a specialized OnTheMarket.com property scraper becomes an invaluable tool for automating data collection and transforming unstructured web content into actionable business intelligence.

Comprehensive Property Data Extraction Solution

The OnTheMarket.com Property Search Scraper is a sophisticated automated tool designed to extract comprehensive property information from the UK's rapidly growing property portal. This scraper addresses the critical need for scalable data collection in the real estate sector, enabling users to gather detailed property intelligence without the manual overhead of browsing individual listings.

The scraper's primary strength lies in its ability to process multiple property search result pages simultaneously while maintaining data integrity and completeness. Unlike basic web scraping tools that may miss crucial information or fail under anti-bot measures, this specialized scraper is engineered specifically for OnTheMarket.com's structure and requirements. It handles dynamic content loading, manages request throttling, and ensures consistent data extraction across different property types and locations.

Target users include real estate investors analyzing market trends, property developers researching potential locations, estate agents conducting competitive analysis, and academic researchers studying housing market patterns. The tool's flexibility accommodates both small-scale research projects requiring data from specific areas and large-scale market analysis covering entire regions or property types.

Input Configuration and Requirements

Example url 1: https://www.onthemarket.com/for-sale/property/shropshire/

Example url 2: https://www.onthemarket.com/for-sale/property/burley-leeds/

Example url 3: https://www.onthemarket.com/for-sale/property/roughley/

Example Screenshot of property list by query page:

Input Format

The scraper accepts configuration through a JSON object. You can either scrape by providing property listing page URLs directly, or by using detailed search filters for targeted data collection. Below is a breakdown of all input properties:

---

Scrape with URLs

{
  "max_retries_per_url": 2, // Maximum waiting time when accessing the links you provided.
  "proxy": { // Add a proxy to ensure that during the data collection process, you are not detected as a bot.
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL" 
    ],
    "apifyProxyCountry": "SG" // You should choose an Country that coincides with the Country you want to collect data from
  },
  "max_items_per_url": 20,
  "urls": [ // Links to property list by query pages.
    "https://www.onthemarket.com/for-sale/property/shropshire/",
    "https://www.onthemarket.com/for-sale/property/burley-leeds/",
    "https://www.onthemarket.com/for-sale/property/roughley/"
  ],
  "ignore_url_failures": true // Continue scraping even if some URLs fail
}

• urls (array)
  Description: List of property listing page URLs you wish to scrape. You can add the URLs one by one or in bulk.
  Note: If this property is set, the scraper will use only these URLs for extraction, and all filter-based options will be ignored.

• ignore_url_failures (boolean)
  Description: If true, scraping continues even if some URLs fail after all retries. Useful to prevent one failed link from stopping the whole process.
  Default: true

---

Scrape with Search Filters

{
  "max_retries_per_url": 2, // Maximum number of retry attempts for each filter set if the scrape fails or is detected as a bot
  "proxy": { // Proxy configuration for anonymity and to avoid detection
    "useApifyProxy": true, // Use Apify's residential proxies (recommended for large/bulk scraping)
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ], // Proxy group to use; "RESIDENTIAL" is designed to avoid IP bans
    "apifyProxyCountry": "AU" // Country code matching your target website’s region (e.g., AU for Australia)
  },
  "max_items_per_url": 20, // Limit the number of property listings to extract per search result or filter combination
  "location": "swansea", // The location to search for properties; copy directly from the site URL for accuracy (e.g., 'swansea')
  "operation_type": "for-sale", // Type of property operation: for-sale, to-rent, or new-homes
  "property_type": "flats-apartments", // Category of property: property, detached, semi-detached, terraced, bungalow, flats-apartments, farms-land, park-homes
  "recently_added": "3-days", // Filter for properties listed in the last 24-hours, 3-days, 7-days, or "" for any time
  "radius": "5", // Search radius from the location, in miles (e.g., "5" for 5 miles)
  "furnished": "yes", // Specify if only furnished properties should be included: yes (furnished), no (not furnished)
  "sort_by": "update_date", // Sort the results by: "" (Recommended), price, price&direction=asc, update_date
  "page": 1 // Specify the starting page number for search results
}

Configure these properties for flexible, precise data extraction.
If any of these filter parameters are set, leave the urls field empty.

• location (string)
  Description: Specify the area to search (e.g., "swansea"). For best results, copy the location value directly from the website URL after conducting a search. Make sure the location is accurate by selecting a location on the website and then searching. Copy the location value directly from the URL. For example, for the URL '/to-rent/property/swansea/?page=2&sort-field=price', the correct location is 'swansea'.

• operation_type (string)
  Description: Select the type of property operation.

  • for-sale – Properties for sale
  • to-rent – Properties for rent
  • new-homes – New homes

• property_type (string)
  Description: Select the category of property you want to scrape.

  • property, detached, semi-detached, terraced, bungalow, flats-apartments, farms-land, park-homes

• recently_added (string)
  Description: Filter properties by how recently they were listed.

  • "" – Any time
  • 24-hours – Last 24 hours
  • 3-days – Last 3 days
  • 7-days – Last 7 days

• radius (string)
  Description: Search within a specific distance from the location. Select a value from:

  • "0.25", "0.5", "1", "2", "3", "4", "5", "7.5", "10", "15", "20", "30", "40"
    (Units: miles)

• furnished (string)
  Description: Specify if you want only furnished or unfurnished properties.

  • yes – Furnished
  • no – Not furnished

• sort_by (string)
  Description: Sort results by property attributes.

  • "" – Recommended
  • price – Price low to high
  • price&direction=asc – Price high to low
  • update_date – Most recent

• page (integer)
  Description: Starting page number for results. Useful for resuming or pagination.

---

General Options

• max_items_per_url (integer)
  Description: Limit the number of properties to scrape from each URL or filter result.
  Default: 20

• max_retries_per_url (integer)
  Description: Maximum retry attempts for failed requests or when scraping is blocked.
  Default: 2

• proxy (object)
  Description: Configure how your scraper connects.

  • useApifyProxy: Use the built-in Apify residential proxies (recommended for large/bulk scraping).
  • apifyProxyGroups: List of proxy groups, e.g., ["RESIDENTIAL"].
  • apifyProxyCountry: Country code to match the website’s region (e.g., SG for Singapore, AU for Australia).
  • By default, you can set "useApifyProxy": false to run without proxies for small or local extractions.

---

Comprehensive Output Data Structure

You get the output from the Onthemarket.com Property Search Scraper stored in a tab. The following is an example of the Information Fields collected after running the Actor.

[ // List of property information
  {
  "features": [
    "Tenure: Freehold",
    "Mid terrace property",
    "Open plan hallway/lounge/dining room",
    "Kitchen"
  ],
  "address": "Vivian Road, Sketty, Swansea",
  "humanised_property_type": "Terraced house",
  "price_qualifier": "Offers in region of",
  "images": [
    {
      "default": "https://media.onthemarket.com/properties/18028428/1571978285/image-0-480x320.jpg",
      "portrait?": false,
      "webp": "https://media.onthemarket.com/properties/18028428/1571978285/image-0-480x320.webp"
    },
    {
      "default": "https://media.onthemarket.com/properties/18028428/1571978285/image-1-480x320.jpg",
      "portrait?": false,
      "webp": "https://media.onthemarket.com/properties/18028428/1571978285/image-1-480x320.webp"
    },
    {
      "default": "https://media.onthemarket.com/properties/18028428/1571978296/image-2-480x320.jpg",
      "portrait?": false,
      "webp": "https://media.onthemarket.com/properties/18028428/1571978296/image-2-480x320.webp"
    },
    {
      "default": "https://media.onthemarket.com/properties/18028428/1571978285/image-3-480x320.jpg",
      "portrait?": false,
      "webp": "https://media.onthemarket.com/properties/18028428/1571978285/image-3-480x320.webp"
    },
    {
      "default": "https://media.onthemarket.com/properties/18028428/1571978285/image-4-480x320.jpg",
      "portrait?": false,
      "webp": "https://media.onthemarket.com/properties/18028428/1571978285/image-4-480x320.webp"
    }
  ],
  "details_url": "/details/18028428/",
  "main_label": "Spotlight Property",
  "fees_description": null,
  "short_price": "£215k",
  "bedrooms": 3,
  "id": "18028428",
  "days_since_added_reduced": "Added > 14 days",
  "spotlight": true,
  "fees_label": null,
  "bathrooms": 1,
  "cover_image": {
    "default": "https://media.onthemarket.com/properties/18028428/1571978285/image-0-480x320.jpg",
    "webp": "https://media.onthemarket.com/properties/18028428/1571978285/image-0-480x320.webp"
  },
  "location": {
    "lon": -3.983216,
    "lat": 51.621341
  },
  "price": "£215,000",
  "from_url": "https://www.onthemarket.com/for-sale/property/swansea?recently-added=&radius=10&furnished=&sort-field=price&page=1"
}, // ... Many other property details
]

The scraper generates structured JSON output containing extensive property information extracted from each listing. The output provides a comprehensive dataset suitable for various analytical applications and business intelligence purposes.

Output Field Explanations

Basic Property Information:

• ID: Unique identifier for the property used for internal tracking and URL generation.
• Details URL: Link to the full property details page, allowing quick access to complete listing information.
• Title: Full headline of the property as displayed in search results, usually including property type and key features.
• Short Price: Concise display of the listing price, e.g., "$350,000".

Location and Address:

• Display Address: Public-facing address shown to potential buyers, possibly partially obscured for privacy reasons.
• Location: General geographic area or district where the property is situated.
• Location Details: Extended info about the neighborhood or specific location.

Property Specifications:

• Bedrooms: Number of bedrooms, essential for categorization and targeting buyers.
• Bedrooms Text: Descriptive text such as "3 bedroom house" or "2 bed apartment."
• Features: List of amenities and features like garden, parking, architecture style.
• Show Features: Boolean indicating if additional features are displayed.

Financial Information:

• Price: The current listed price, typically in GBP or relevant currency.
• Price Qualifier: Contextual info such as "Offers Over", "Guide Price", or "Auction."
• For Sale: Boolean indicating if the property is actively listed for sale.

Visual Content:

• Images: Collection of images showcasing the property.
• Images Count: Total number of images, indicating listing completeness.
• Cover Image: Main image used in search results and thumbnails.
• Videos: Virtual tours or promotional videos available.
• Floorplans: Floor plans or layout diagrams.
• Floorplans Count: Number of floorplan images or documents.

Agent and Marketing:

• Agent: Details of the real estate agent or agency responsible.
• Property Badge: Marketing badges like "New", "Reduced", "Under Offer."
• Property Labels: Additional tags for categorization and visibility.
• New Home Flag: Indicator for newly built or new development properties.
• Days Since Added/Reduced: Time since the listing was added or price was reduced, useful for market analysis.

Enhanced Features:

• Online Viewing / Virtual Tours: Availability of virtual viewings or 3D tours.
• Spotlight: Marking as a premium or promoted listing.

Implementation Guide and Best Practices

Step-by-Step Usage Instructions

Step 1: Configuration Setup
Begin by preparing your input JSON according to your data collection needs, based on the available properties.

• Scrape by URLs:
  Input one or more OnTheMarket.com property list page URLs in the urls property. Filtering options will be disabled in this mode.
• Scrape by Search Filters:
  Alternatively, use search filter properties (such as location, operation_type, property_type, radius, etc.) for targeted and dynamic collection. Leave urls empty or unset to use filters.

Adjust max_retries_per_url for error resilience and set max_items_per_url according to your required dataset size. Set up proxy parameters (including location) for optimal market access and to maintain anonymous, reliable scraping.

Step 2: Input Preparation

• For URL-based scraping, search and gather specific property listing URLs from OnTheMarket.com, using the platform’s own filters to preselect property type, location, price range, and other criteria before copying URLs.
• For search filter scraping, specify the desired field values directly in the JSON configuration, e.g., targeting certain cities, property types, date ranges, furnishing status, etc.

Step 3: Execution and Monitoring
Launch the scraper with your prepared input parameters. Monitor the real-time progress, which provides feedback on property extraction status and any encountered issues. Extraction duration depends on property volume and network/server speed.

---

Best Practices for Optimal Results

• Stagger your scraping requests to avoid server-side rate limits and maintain access.
• Use residential proxies, matching the proxy region to your market geography for best accuracy and availability.
• Monitor scraper logs to quickly spot and address extraction errors.
• For large-scale jobs, consider dividing your task into smaller batches for manageability and more reliable completion.
• Always perform data validation and quality checks on outputs to ensure they fit business requirements.

---

Common Issues and Solutions

• Transient Extraction Failures:
  Network interruptions or server hiccups may cause failures. These are typically handled by increasing max_retries_per_url. If problems persist, review and update your proxy setup or adjust request rate parameters.

• Data Consistency & Updates:
  Property listings may change during scraping. To ensure data accuracy, implement timestamp checks and consider periodically re-running extractions for important datasets.

Flexible Scraping Modes:
Remember, the scraper works either by direct URL input or by configurable search filters—but not both at the same time. Build your JSON according to your needs and the available properties for robust, high-quality data collection.

Business Value and Practical Applications

The OnTheMarket.com Property Search Scraper delivers significant time savings compared to manual data collection methods. Tasks that previously required hours or days of manual browsing can be completed in minutes, freeing up valuable resources for analysis and decision-making activities.

Real estate investors leverage the extracted data for market trend analysis, identifying undervalued properties, and tracking price movements across different areas. Property developers use comprehensive location and feature data to assess market demand and plan new developments. Estate agents utilize competitive analysis capabilities to benchmark their listings against market standards and optimize pricing strategies.

The structured data output integrates seamlessly with business intelligence tools, spreadsheet applications, and custom analysis frameworks. This compatibility enables sophisticated market analysis, automated reporting, and data-driven decision making across various real estate business functions.

Conclusion

OnTheMarket.com's position as one of the UK's most user-friendly property portals makes it an invaluable data source for real estate professionals and researchers. The OnTheMarket.com Property Search Scraper transforms this wealth of information into actionable business intelligence through automated, reliable data extraction.

Whether you're conducting market research, analyzing investment opportunities, or monitoring competitive landscapes, this specialized scraper provides the comprehensive property data needed to make informed decisions in the dynamic UK real estate market. Transform your property research capabilities today with efficient, scalable data extraction from one of the UK's leading property portals.

Related Actors

• Onthemarket.com Property Details Scraper: A specialized data extraction tool engineered to harvest detailed property information from Onthemarket's dominant Dutch real estate marketplace.

Your feedback

We are always working to improve Actors' performance. So, if you have any technical feedback about Onthemarket.com Property Search Scraper or simply found a bug, please create an issue on the Actor's Issues tab in Apify Console.