# Gumtree Scraper (`rover-omniscraper/gumtree-scraper`) Actor Powerful Gumtree UK property scraper extracting rental listings with titles, prices, descriptions, images, GPS coordinates & seller phone numbers. HTTP-based solution bypassing bot detection. Apify-compatible with Supabase integration. Fast, reliable & cookieless mode supported. - **URL**: https://apify.com/rover-omniscraper/gumtree-scraper.md - **Developed by:** [Rover Omniscraper](https://apify.com/rover-omniscraper) (community) - **Categories:** Automation, Lead generation, Integrations - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $8.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Gumtree Property Scraper An Apify-compatible scraper that extracts property rental listings from Gumtree UK, including titles, prices, descriptions, images, map coordinates, and **seller phone numbers**. ### How It Works The scraper uses HTTP requests with session cookies to fetch Gumtree pages, then parses the HTML with BeautifulSoup. This approach completely bypasses Gumtree's Radware bot detection (which blocks headless browsers). #### Scraping Flow 1. **Search**: Fetches search result pages for the configured location/filters 2. **Parse Tiles**: Extracts listing URLs, prices, and post dates from search results 3. **Listing Details**: Visits each listing page to extract full data (title, attributes, description, images, coordinates) 4. **Phone Numbers**: If cookies are provided, fetches the phone reveal URL (`?srn=true`) to extract seller phone numbers 5. **Storage**: Pushes results to Apify Dataset and optionally to Supabase ### Providing Cookies (Required for Phone Numbers) Cookies authenticate your Gumtree session. **Without cookies, the scraper still works** but cannot extract phone numbers. #### Step-by-Step 1. **Install a cookie export extension** in your browser: - Chrome: [Cookie-Editor](https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm) or [EditThisCookie](https://chromewebstore.google.com/detail/editthiscookie/fngmhnnpilhplaeedifhccceomclgfbg) - Firefox: [Cookie-Editor](https://addons.mozilla.org/en-US/firefox/addon/cookie-editor/) 2. **Log into Gumtree** at [www.gumtree.com](https://www.gumtree.com) with your account 3. **Export cookies**: - Click the cookie extension icon - Click **Export** → **Export as JSON** (copies to clipboard) - The exported JSON is an array of cookie objects 4. **Use the cookies**: - **Apify UI**: Paste the JSON array into the "Gumtree Session Cookies" input field - **Local testing**: Save the JSON to `cookies.json` in the project root #### Cookie Format The exported JSON looks like this (array of objects): ```json [ { "domain": ".gumtree.com", "name": "gt_rememberMe", "value": "...", "path": "/", "secure": true, "httpOnly": true, "expirationDate": 1806936367 }, { "domain": ".gumtree.com", "name": "gt_s", "value": "id:...", "path": "/", "session": true } ] ```` #### Important Notes - **Export ALL cookies** — don't cherry-pick. The scraper needs the full set - **Session cookies expire** — re-export cookies if you get bot challenges or empty phone numbers - Key cookies: `gt_rememberMe`, `gt_s`, `GTSELLERSESSIONID`, `conversationsToken`, `waap_id` - The `waap_id` cookie (Radware) helps avoid bot challenges ### Local Testing ```bash ## Create virtual environment python -m venv .venv .venv\Scripts\activate ## Windows ## source .venv/bin/activate ## Linux/Mac ## Install dependencies pip install -r requirements.txt ## Save your cookies (see above) ## Then run: python test_local.py ``` Results are saved to `test_output.json`. ### Input Parameters | Field | Type | Required | Default | Description | |-------|------|----------|---------|-------------| | `location` | string | Yes | — | Search location (e.g. "London") | | `propertyCategory` | string | No | `property-to-rent` | Category slug | | `minPrice` | int | No | — | Minimum price filter | | `maxPrice` | int | No | — | Maximum price filter | | `minBedrooms` | int | No | — | Minimum bedrooms | | `maxBedrooms` | int | No | — | Maximum bedrooms | | `distance` | int | No | — | Search radius in miles | | `maxPages` | int | No | 6 | Max search result pages | | `maxDaysOld` | int | No | 7 | Skip listings older than N days | | `cookies` | array | No | — | Session cookies (see above) | | `supabaseUrl` | string | No | — | Supabase project URL | | `supabaseKey` | string | No | — | Supabase service role key | ### Output Schema Each listing produces a record like: ```json { "listing_id": "uuid", "title": "2 Bed Flat, Newton Place, E14", "price_per_month": 2400.0, "price_currency": "£", "period": "pm", "last_updated": "3 hours ago", "platform_name": "Gumtree", "listing_url": "https://www.gumtree.com/p/...", "address": "London", "property_type": "Flat", "bedroom": 2, "images": ["url1", "url2", "..."], "description": "...", "scrape_timestamp": "2026-04-05T20:06:38", "longitude": -0.026247, "latitude": 51.4924, "seller_type": "Private", "phone_number": "02037441177" } ``` ### Architecture ``` src/ ├── main.py ## Apify Actor entry point ├── scraper.py ## HTTP client, search pagination, listing processing ├── extractors.py ## BeautifulSoup-based HTML data extraction ├── storage.py ## Optional Supabase integration └── utils.py ## Price parsing, date filtering, coordinate extraction ``` - **No headless browser** — uses `httpx` for HTTP requests, `BeautifulSoup` + `lxml` for parsing - Completely avoids Radware/ShieldSquare bot detection - \~2 seconds per listing (HTTP fetch + parse) # Actor input Schema ## `location` (type: `string`): Location to search (e.g., 'London', 'Manchester', postcode) ## `propertyCategory` (type: `string`): Category slug for property search (e.g., 'property-to-rent', 'property-for-sale', 'room-to-rent') ## `minPrice` (type: `integer`): Minimum price filter ## `maxPrice` (type: `integer`): Maximum price filter ## `minBedrooms` (type: `integer`): Minimum number of bedrooms ## `maxBedrooms` (type: `integer`): Maximum number of bedrooms ## `distance` (type: `integer`): Search radius in miles from location ## `maxPages` (type: `integer`): Maximum number of search result pages to scrape ## `maxDaysOld` (type: `integer`): Stop scraping listings older than this many days ## `cookies` (type: `array`): Paste your Gumtree session cookies here to unlock seller phone numbers. Use a browser extension like 'EditThisCookie' or 'Cookie-Editor' to export cookies as JSON from gumtree.com after logging in. Each cookie needs: name, value, domain, path. ## `gumtreeEmail` (type: `string`): Your Gumtree account email. Used only if cookies are not provided. The scraper will log in automatically and extract phone numbers. ## `gumtreePassword` (type: `string`): Your Gumtree account password. Used only if cookies are not provided. ## `supabaseUrl` (type: `string`): Supabase project URL for storing results and duplicate checking ## `supabaseKey` (type: `string`): Supabase service role key ## `proxyConfiguration` (type: `object`): Apify proxy configuration. Residential proxies recommended. ## Actor input object example ```json { "location": "London", "propertyCategory": "property-to-rent", "minPrice": 500, "maxPrice": 3000, "minBedrooms": 1, "maxBedrooms": 5, "distance": 10, "maxPages": 6, "maxDaysOld": 7 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("rover-omniscraper/gumtree-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("rover-omniscraper/gumtree-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call rover-omniscraper/gumtree-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=rover-omniscraper/gumtree-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Gumtree Scraper", "description": "Powerful Gumtree UK property scraper extracting rental listings with titles, prices, descriptions, images, GPS coordinates & seller phone numbers. HTTP-based solution bypassing bot detection. Apify-compatible with Supabase integration. Fast, reliable & cookieless mode supported.", "version": "0.0", "x-build-id": "OJ2N653ivAwZkrsnJ" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/rover-omniscraper~gumtree-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-rover-omniscraper-gumtree-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/rover-omniscraper~gumtree-scraper/runs": { "post": { "operationId": "runs-sync-rover-omniscraper-gumtree-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/rover-omniscraper~gumtree-scraper/run-sync": { "post": { "operationId": "run-sync-rover-omniscraper-gumtree-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "location" ], "properties": { "location": { "title": "Location", "type": "string", "description": "Location to search (e.g., 'London', 'Manchester', postcode)", "default": "London" }, "propertyCategory": { "title": "Property Category", "type": "string", "description": "Category slug for property search (e.g., 'property-to-rent', 'property-for-sale', 'room-to-rent')", "default": "property-to-rent" }, "minPrice": { "title": "Minimum Price", "type": "integer", "description": "Minimum price filter", "default": 500 }, "maxPrice": { "title": "Maximum Price", "type": "integer", "description": "Maximum price filter", "default": 3000 }, "minBedrooms": { "title": "Minimum Bedrooms", "type": "integer", "description": "Minimum number of bedrooms", "default": 1 }, "maxBedrooms": { "title": "Maximum Bedrooms", "type": "integer", "description": "Maximum number of bedrooms", "default": 5 }, "distance": { "title": "Distance (miles)", "type": "integer", "description": "Search radius in miles from location", "default": 10 }, "maxPages": { "title": "Max Pages", "minimum": 1, "maximum": 20, "type": "integer", "description": "Maximum number of search result pages to scrape", "default": 6 }, "maxDaysOld": { "title": "Max Listing Age (days)", "minimum": 1, "type": "integer", "description": "Stop scraping listings older than this many days", "default": 7 }, "cookies": { "title": "Gumtree Session Cookies (Recommended)", "type": "array", "description": "Paste your Gumtree session cookies here to unlock seller phone numbers. Use a browser extension like 'EditThisCookie' or 'Cookie-Editor' to export cookies as JSON from gumtree.com after logging in. Each cookie needs: name, value, domain, path." }, "gumtreeEmail": { "title": "Gumtree Email", "type": "string", "description": "Your Gumtree account email. Used only if cookies are not provided. The scraper will log in automatically and extract phone numbers." }, "gumtreePassword": { "title": "Gumtree Password", "type": "string", "description": "Your Gumtree account password. Used only if cookies are not provided." }, "supabaseUrl": { "title": "Supabase URL", "type": "string", "description": "Supabase project URL for storing results and duplicate checking" }, "supabaseKey": { "title": "Supabase Service Key", "type": "string", "description": "Supabase service role key" }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "Apify proxy configuration. Residential proxies recommended." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```