# Wayfair Scraper (`crawlerbros/wayfair-scraper`) Actor Scrape Wayfair product listings, product details, category browses, and brand pages from Wayfair.com (US), Wayfair.ca (Canada), Wayfair.co.uk (UK), and Wayfair.ie (Ireland). Extract title, SKU, brand, price, currency, rating, review count, image, breadcrumbs, and product URL. - **URL**: https://apify.com/crawlerbros/wayfair-scraper.md - **Developed by:** [Crawler Bros](https://apify.com/crawlerbros) (community) - **Categories:** E-commerce, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 11 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing from $3.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Wayfair Scraper Scrape Wayfair product listings, product detail pages, category browses, and brand pages from Wayfair's regional storefronts. Extract title, SKU, brand, price (incl. previous/sale price), currency, rating, review count, breadcrumbs, image, and product URL. ### Why this scraper - **No login, no cookies, no API key required** — works against the public Wayfair site. - **5 modes** — `search`, `byUrl`, `byCategory`, `byBrand`, `byProduct`. - **4 regional storefronts** — `wayfair.com` (US), `wayfair.ca` (CA), `wayfair.co.uk` (UK), `wayfair.ie` (IE). - **HTTP-only via `curl_cffi` chrome131 impersonation** — fast, low-memory, no browser overhead. - **Apify residential proxy by default** — Wayfair blocks Apify datacenter IPs with 429s, so the actor uses residential proxy out of the box. - **Filters** — price range, minimum rating, sort order. - **Optional PDP enrichment** — set `fetchProductDetails=true` to fetch the product detail page for every listing card and add brand, full description, breadcrumbs, and high-res image. ### Input | Field | Type | Description | |---|---|---| | `mode` | enum | `search`, `byUrl`, `byCategory`, `byBrand`, `byProduct` | | `country` | enum | `com`, `ca`, `co.uk`, `ie` | | `text` | string | Search keyword (`search` mode) | | `category` | enum | Top-level Wayfair category (`byCategory` mode) — Furniture, Outdoor, Decor & Pillows, Rugs, Kitchen & Tabletop, Bed & Bath, Lighting, Storage & Organization, Appliances, Home Improvement, Baby & Kids, Holiday Decor, Office, Pet | | `urls` | array | Direct Wayfair URLs (`byUrl` / `byBrand` modes) | | `skus` | array | Wayfair SKUs (e.g. `W118077895`, `CXVA1530`) for `byProduct` mode | | `sortBy` | enum | `relevance`, `price_low_to_high`, `price_high_to_low`, `top_rated`, `newest`, `best_seller` | | `priceMin` / `priceMax` | int | Drop products outside the price range | | `ratingMin` | float | Drop products with rating below this (0.0-5.0) | | `fetchProductDetails` | bool | Enrich every listing card with PDP data | | `maxItems` | int | Hard cap on emitted records | | `maxPagesPerListing` | int | Max paginated listing pages to crawl per seed | | `proxyConfiguration` | proxy | Apify proxy configuration (defaults to RESIDENTIAL — required for cloud runs) | | `brand` | string | Brand name for `byBrand` mode (e.g. "Latitude Run") | ### Output Each record represents a single Wayfair product: ```json { "sku": "CXVA1530", "title": "Latitude Run Ilkeston 81.6\" Wide Vegan Leather Manual Reclining Sofa", "brand": "Latitude Run", "productUrl": "https://www.wayfair.com/furniture/pdp/latitude-run-ilkeston-...-cxva1530.html", "canonicalUrl": "https://www.wayfair.com/furniture/pdp/latitude-run-ilkeston-...-cxva1530.html", "price": 689.99, "previousPrice": 1098.99, "currency": "USD", "ratingValue": 4.63, "reviewCount": 648, "imageUrl": "https://assets.wfcdn.com/im/.../Ilkeston+81.6+...+Cup+Holders.jpg", "description": "Choose a delivery day that works for you...", "breadcrumbs": ["Furniture", "Living Room Furniture", "Sofas"], "category": "Furniture", "subCategory": "Sofas", "promoFlag": "72-Hour Clearout", "country": "com", "siteName": "Wayfair", "recordType": "product", "scrapedAt": "2026-05-08T17:24:33.123456+00:00", "sourceUrl": "https://www.wayfair.com/keyword.php?keyword=sofa" } ```` Empty / null fields are omitted from the record (no sentinel values). ### Modes #### Search Free-text keyword search across the country's catalog. ```json { "mode": "search", "country": "com", "text": "sofa", "sortBy": "top_rated", "maxItems": 50 } ``` #### By URL Paste any Wayfair search results, category, brand, or product detail URL. The scraper auto-detects the URL kind and the country from the host. ```json { "mode": "byUrl", "urls": [ "https://www.wayfair.com/furniture/cat/sofas-c413894.html", "https://www.wayfair.com/furniture/pdp/latitude-run-ilkeston-...-cxva1530.html" ] } ``` #### By Category Browse a top-level Wayfair category. ```json { "mode": "byCategory", "country": "com", "category": "lighting", "sortBy": "newest" } ``` #### By Brand Search by brand name. The scraper performs a brand-scoped keyword search. ```json { "mode": "byBrand", "country": "com", "brand": "Latitude Run" } ``` #### By Product Look up products by SKU. ```json { "mode": "byProduct", "country": "com", "skus": ["W118077895", "CXVA1530"] } ``` ### FAQs **Q: Does this work without a proxy?** Locally yes (from your home/office IP), but on Apify cloud Wayfair returns 429 from every Apify datacenter IP. The actor defaults to RESIDENTIAL proxy for that reason. **Q: Does it support Wayfair.de?** No. Wayfair shut down its German storefront in 2020; the `wayfair.de` domain redirects to a permanent announcement page. **Q: Do search results include all fields?** Search-result HTML embeds full data (title, price, currency, rating, reviewCount) for the first 3-5 cards above-the-fold, and partial data (title, sku, productUrl, image) for the remaining 45+ cards on the page. To get full data for every card, enable `fetchProductDetails=true`. **Q: How many products per page?** ~48 unique SKUs per Wayfair listing page. **Q: How does the price filter work?** Products without a price are kept (the user opted into a filter, not into requiring the field). Products with a price outside `[priceMin, priceMax]` are dropped. **Q: Does the scraper extract reviews?** Not in this version. The Wayfair review widget loads via XHR after click; this scraper focuses on product-level data (title, price, rating aggregate, review count). Per-review extraction is on the roadmap. **Q: What about international currency?** Each country has its native currency: USD (com), CAD (ca), GBP (co.uk), EUR (ie). The scraper extracts the actual currency code embedded in the JSON for each listing. ### Limitations - **wayfair.de is not supported** (storefront closed). - **Reviews-per-product** are not yet extracted (only the aggregate `ratingValue` and `reviewCount`). - **Brand pages**: Wayfair's brand URL format varies; the easiest way to scrape a brand is via `byUrl` with the brand listing URL copied from the site. - **Product detail data** for sponsored/ad placements may be sparser than for organic results. ### Test plan Run `python -m pytest tests/ -q` from the actor folder. The suite covers 100+ unit tests across URL handling, listing extraction (US/CA/UK fixtures), product record extraction (real PDP fixture), filter helpers (per-type edge cases), and recursive null-walking. # Actor input Schema ## `mode` (type: `string`): Discovery axis. search: keyword search. byUrl: paste any Wayfair search/category/brand/product URL(s). byCategory: top-level category browse. byBrand: brand listing URL(s). byProduct: SKU lookup (product detail page). ## `country` (type: `string`): Wayfair regional domain. com=US, ca=Canada, co.uk=UK, ie=Ireland. (Wayfair.de shut down in 2020 and is not supported.) ## `text` (type: `string`): Free-text keyword (used in search mode). Examples: "sofa", "office chair", "throw pillow", "area rug". ## `category` (type: `string`): Top-level Wayfair category (resolves to a category browse URL). Used for byCategory mode. ## `urls` (type: `array`): Direct Wayfair URLs (search results, category page, or product detail page). Domain auto-detected from URL. ## `brand` (type: `string`): Brand / manufacturer name (e.g. "Latitude Run", "Wade Logan", "Hokku Designs"). The scraper performs a brand-scoped search on Wayfair. ## `skus` (type: `array`): Wayfair product SKUs (e.g. "W118077895", "CXVA1530"). The scraper resolves each SKU to its detail page. ## `sortBy` (type: `string`): Sort order on listing pages. Wayfair maps these to its sortby URL parameter. ## `priceMin` (type: `integer`): Drop products priced below this value (in the destination country's currency). ## `priceMax` (type: `integer`): Drop products priced above this value (in the destination country's currency). ## `ratingMin` (type: `number`): Drop products with rating below this value (0.0-5.0). Products with no rating are kept (the user opted into the filter, not into requiring the field). ## `fetchProductDetails` (type: `boolean`): For each listing card, fetch the product detail page to enrich with brand, full description, breadcrumbs, and high-resolution image. Adds 1 HTTP request per product. ## `maxItems` (type: `integer`): Hard cap on emitted product records. ## `maxPagesPerListing` (type: `integer`): Maximum paginated listing pages (?page=N) to crawl per search/category/brand seed (~48 products per page). ## `proxyConfiguration` (type: `object`): Apify proxy configuration. Wayfair blocks Apify datacenter IPs, so RESIDENTIAL proxy is required. Default is RESIDENTIAL. Leaving blank disables proxy and the actor will likely 429 on every request from Apify cloud. ## Actor input object example ```json { "mode": "search", "country": "com", "text": "sofa", "category": "furniture", "urls": [], "brand": "", "skus": [], "sortBy": "relevance", "fetchProductDetails": false, "maxItems": 5, "maxPagesPerListing": 2, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } ``` # Actor output Schema ## `products` (type: `string`): Dataset containing Wayfair product records. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "search", "country": "com", "text": "sofa", "category": "furniture", "urls": [], "brand": "", "skus": [], "sortBy": "relevance", "fetchProductDetails": false, "maxItems": 5, "maxPagesPerListing": 2, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } }; // Run the Actor and wait for it to finish const run = await client.actor("crawlerbros/wayfair-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "search", "country": "com", "text": "sofa", "category": "furniture", "urls": [], "brand": "", "skus": [], "sortBy": "relevance", "fetchProductDetails": False, "maxItems": 5, "maxPagesPerListing": 2, "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"], }, } # Run the Actor and wait for it to finish run = client.actor("crawlerbros/wayfair-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "search", "country": "com", "text": "sofa", "category": "furniture", "urls": [], "brand": "", "skus": [], "sortBy": "relevance", "fetchProductDetails": false, "maxItems": 5, "maxPagesPerListing": 2, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } }' | apify call crawlerbros/wayfair-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=crawlerbros/wayfair-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Wayfair Scraper", "description": "Scrape Wayfair product listings, product details, category browses, and brand pages from Wayfair.com (US), Wayfair.ca (Canada), Wayfair.co.uk (UK), and Wayfair.ie (Ireland). Extract title, SKU, brand, price, currency, rating, review count, image, breadcrumbs, and product URL.", "version": "1.0", "x-build-id": "KiSUfscwwPdwljRhy" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/crawlerbros~wayfair-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-crawlerbros-wayfair-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/crawlerbros~wayfair-scraper/runs": { "post": { "operationId": "runs-sync-crawlerbros-wayfair-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/crawlerbros~wayfair-scraper/run-sync": { "post": { "operationId": "run-sync-crawlerbros-wayfair-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "mode": { "title": "Scrape mode", "enum": [ "search", "byUrl", "byCategory", "byBrand", "byProduct" ], "type": "string", "description": "Discovery axis. search: keyword search. byUrl: paste any Wayfair search/category/brand/product URL(s). byCategory: top-level category browse. byBrand: brand listing URL(s). byProduct: SKU lookup (product detail page).", "default": "search" }, "country": { "title": "Country / domain", "enum": [ "com", "ca", "co.uk", "ie" ], "type": "string", "description": "Wayfair regional domain. com=US, ca=Canada, co.uk=UK, ie=Ireland. (Wayfair.de shut down in 2020 and is not supported.)", "default": "com" }, "text": { "title": "Search keyword (search mode)", "type": "string", "description": "Free-text keyword (used in search mode). Examples: \"sofa\", \"office chair\", \"throw pillow\", \"area rug\".", "default": "sofa" }, "category": { "title": "Category (byCategory mode)", "enum": [ "furniture", "outdoor", "decor-pillows", "rugs", "kitchen-tabletop", "bed-bath", "lighting", "storage-organization", "appliances", "home-improvement", "baby-kids", "holiday-decor", "office", "pet" ], "type": "string", "description": "Top-level Wayfair category (resolves to a category browse URL). Used for byCategory mode.", "default": "furniture" }, "urls": { "title": "URLs (byUrl mode)", "type": "array", "description": "Direct Wayfair URLs (search results, category page, or product detail page). Domain auto-detected from URL.", "default": [], "items": { "type": "string" } }, "brand": { "title": "Brand name (byBrand mode)", "type": "string", "description": "Brand / manufacturer name (e.g. \"Latitude Run\", \"Wade Logan\", \"Hokku Designs\"). The scraper performs a brand-scoped search on Wayfair.", "default": "" }, "skus": { "title": "Product SKUs (byProduct mode)", "type": "array", "description": "Wayfair product SKUs (e.g. \"W118077895\", \"CXVA1530\"). The scraper resolves each SKU to its detail page.", "default": [], "items": { "type": "string" } }, "sortBy": { "title": "Sort by (search / byCategory / byBrand)", "enum": [ "relevance", "price_low_to_high", "price_high_to_low", "top_rated", "newest", "best_seller" ], "type": "string", "description": "Sort order on listing pages. Wayfair maps these to its sortby URL parameter.", "default": "relevance" }, "priceMin": { "title": "Minimum price", "minimum": 0, "maximum": 1000000, "type": "integer", "description": "Drop products priced below this value (in the destination country's currency)." }, "priceMax": { "title": "Maximum price", "minimum": 0, "maximum": 1000000, "type": "integer", "description": "Drop products priced above this value (in the destination country's currency)." }, "ratingMin": { "title": "Minimum rating", "minimum": 0, "maximum": 5, "type": "number", "description": "Drop products with rating below this value (0.0-5.0). Products with no rating are kept (the user opted into the filter, not into requiring the field)." }, "fetchProductDetails": { "title": "Fetch product detail pages", "type": "boolean", "description": "For each listing card, fetch the product detail page to enrich with brand, full description, breadcrumbs, and high-resolution image. Adds 1 HTTP request per product.", "default": false }, "maxItems": { "title": "Max items", "minimum": 1, "maximum": 5000, "type": "integer", "description": "Hard cap on emitted product records.", "default": 25 }, "maxPagesPerListing": { "title": "Max listing pages", "minimum": 1, "maximum": 25, "type": "integer", "description": "Maximum paginated listing pages (?page=N) to crawl per search/category/brand seed (~48 products per page).", "default": 3 }, "proxyConfiguration": { "title": "Apify proxy configuration", "type": "object", "description": "Apify proxy configuration. Wayfair blocks Apify datacenter IPs, so RESIDENTIAL proxy is required. Default is RESIDENTIAL. Leaving blank disables proxy and the actor will likely 429 on every request from Apify cloud.", "default": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```