# Best Buy Product Scraper (`cirkit/bestbuy-product-scraper`) Actor Fast Best Buy product scraper. Pull product details (title, prices, savings, SKU, BSIN, model number, brand, ratings, reviews, specifications, images, category path) by SKU, direct URL, search keyword, or category. Reads Best Buy's GraphQL gateway directly, no headless browser. - **URL**: https://apify.com/cirkit/bestbuy-product-scraper.md - **Developed by:** [Crikit](https://apify.com/cirkit) (community) - **Categories:** E-commerce - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Best Buy Product Scraper Fast, structured product data from Best Buy. Pull current price, regular price, savings, SKU, BSIN, brand, model number, ratings, review count, full specifications, image gallery, category path, and availability for any product on bestbuy.com. Three discovery modes plus two precise modes: pass a list of SKUs, a list of product URLs, search keywords, or category IDs. Output is a flat JSON record per product. ### What you get per product - **Identifiers** - `skuId`, `bsin`, `upc`, `modelNumber` - **Pricing** - `currentPrice`, `regularPrice`, `totalSavings`, `totalSavingsPercent`, `onSale` - **Catalog** - `title`, `brand`, `categoryPath`, `whatItIs`, `releaseDate` - **Reviews** - `averageRating`, `reviewCount` - **Media** - `primaryImage`, `images` (gallery) - **Detail** - `specifications` (flat key-value map), `highlights`, `features`, `dimensions` - **Status** - `availability` (active / archived / coming-soon) ### How it works The Best Buy Product Scraper reads Best Buy's public GraphQL gateway directly. It does NOT spin up a headless browser, so each request costs about 30 ms of compute and 10 KB of bandwidth. The fallback path parses the product page HTML directly when the gateway hiccups. Under the hood: - `curl_cffi` with Chrome 131 TLS impersonation to slip past Akamai Bot Manager - US residential proxy rotation (via Apify Proxy) with auto-rotation every 4 requests per IP - PDP HTML fallback when the GraphQL gateway is rate-limited, so successful runs degrade gracefully - Two-layer SKU discovery: the search page's embedded GraphQL response gives ~18 organic results per page (vs 4 from naive SSR scraping) #### A note on pull-through Best Buy uses Akamai Bot Manager with an aggressive policy on automated traffic. Even with all the mitigations above, expect roughly: - **SKU mode and direct URL mode:** 50-70% of submitted SKUs returned on typical runs. - **Search mode:** ~18 organic SKUs returned per result page; per-SKU pull-through then matches SKU mode. - **Category mode (leaf `pcmcat...` IDs):** similar yield to search mode. For the highest pull-through, set `requestConcurrency: 1` (the default) and run smaller batches (10-25 SKUs at a time). Bump `requestConcurrency` to 2 or 3 when you want faster runs and can tolerate lower per-SKU pull-through; the actor input has a tunable knob for exactly this trade-off. ### Input modes #### Mode A: SKUs (fastest, most precise) ```json { "skuIds": ["6447382", "6505725", "6620467"] } ```` Each SKU is fetched directly through the public GraphQL gateway. No discovery overhead. #### Mode B: Direct product URLs ```json { "productUrls": [ "https://www.bestbuy.com/site/apple-airpods-pro-2.../6447382.p?skuId=6447382", "https://www.bestbuy.com/product/sony-wh-1000xm6.../J7XSRH5RCF" ] } ``` Both URL formats are supported. The scraper extracts the SKU and reuses the same fast GraphQL path. #### Mode C: Keyword search ```json { "searchQueries": ["airpods", "oled tv", "lego star wars"], "maxResults": 18 } ``` The scraper walks Best Buy's search results, harvests organic SKUs (~18 per page from the embedded SSR data), and fetches each one's full detail. `maxResults` caps the total **per search query**. For maxResults > 18 you may not get additional results because Best Buy's SSR only includes the first page of organic results in the static HTML; deeper pages lazy-load via JavaScript and aren't reachable from an HTTP-only scraper. For high-volume use, prefer Mode D (category browse, ~24 SKUs per leaf category page) or Mode A (explicit SKU lists). #### Mode D: Category browse ```json { "categoryIds": ["pcmcat144700050004"], "maxResults": 24 } ``` Category IDs come from any Best Buy PLP URL (the `?id=` query parameter). **Use leaf category IDs (`pcmcat...`), not parent IDs (`abcat...`).** Parent categories are landing pages that link to sub-categories rather than listing actual products; the scraper detects this and returns zero results so you can switch to the correct ID. Leaf categories typically expose ~24 organic SKUs per page in the SSR data. You can mix modes in a single run. Results are deduplicated by SKU. ### Common Best Buy categories | Category ID | Description | |-------------|-------------| | `pcmcat144700050004` | All Headphones | | `pcmcat138500050001` | Cell Phones | | `abcat0502000` | Computers & Tablets | | `abcat0500000` | TV & Home Theater | | `pcmcat209400050001` | Smart Home | | `abcat0204000` | Headphones (parent) | ### Pricing Pay-per-event: **$0.002 per product record** (charged once per item written to the dataset). No subscription, no minimums. A test run of 10 products costs $0.02. A bulk run of 10,000 products costs $20. ### Tips - For **price monitoring** runs every day or every hour, set `includeSpecifications: false` to trim payload size by ~80%. The fast-path record still includes price, rating, availability, and primary image. - For **competitive analysis**, leave `includeSpecifications: true` (default) and pass `searchQueries` matching the product category you want to map. - For **catalog enrichment**, pass your existing SKU list directly via `skuIds` - fastest and cheapest mode. - **Marketplace items** sold by third-party sellers return slightly fewer fields. The scraper writes whatever Best Buy exposes anonymously. ### Output sample ```json { "skuId": "6447382", "bsin": "JJGCQ88C8X", "upc": "195949704529", "title": "Apple - AirPods Pro 2, Wireless Active Noise Cancelling Earbuds with Hearing Aid Feature - White", "brand": "Apple", "modelNumber": "MTJV3LL/A/MTJV3AM/A", "currentPrice": 249.99, "regularPrice": 249.99, "totalSavings": 0, "totalSavingsPercent": 0, "currency": "USD", "onSale": false, "averageRating": 4.8, "reviewCount": 31906, "availability": "active", "categoryPath": ["Headphones", "All Headphones"], "primaryCategoryId": "pcmcat144700050004", "primaryImage": "https://pisces.bbystatic.com/image2/BestBuy_US/images/products/6447/6447382_sd.jpg", "specifications": { "Noise Cancelling (Active)": "true", "Sound Mode": "Stereo", "Battery Life": "6 hours", "Bluetooth": "5.3" }, "scrapedAt": "2026-05-24T20:00:00Z" } ``` ### Limits - **Geo:** Best Buy serves US customers; the scraper runs through US residential proxies (configurable). - **Rate:** ~6-15 products per minute per actor instance, depending on Best Buy's defensive posture. Akamai rate-limits aggressively; the scraper retries with proxy rotation but each successful fetch usually takes 5-15 seconds end-to-end. - **Search/category depth:** ~18-24 organic SKUs per result page; deeper results require JS-rendered lazy loading and are not extracted. - **Stock data:** real-time inventory and "in stock at store X" requires Best Buy auth and is not extracted. The `availability` field returns Best Buy's coarse `dotComDisplayStatus`. - **Open box:** open-box pricing for individual conditions is a separate query; this actor returns the main-line price only. ### Why this Best Buy Product Scraper - **Fast:** GraphQL gateway path is 100x cheaper than headless browser actors. Each product is ~9 KB on the wire and ~150-300 ms of compute (excluding retries under anti-bot pressure). - **Resilient:** TLS impersonation defeats Best Buy's Akamai Bot Manager. When GraphQL is throttled, the scraper transparently falls back to parsing the PDP HTML so runs degrade gracefully instead of failing hard. - **Honest output:** every field maps directly to a Best Buy schema field. No invented values, no LLM-rewritten descriptions. - **Mix-and-match input modes:** SKU list, URL list, search, and category in one run. Built by the Cirkit team. Reach out via the Apify Support tab for questions. # Actor input Schema ## `skuIds` (type: `array`): List of Best Buy 7-digit SKU IDs (e.g. '6447382'). Bypasses search and fetches each SKU directly via the public GraphQL gateway. Fastest path. ## `productUrls` (type: `array`): List of Best Buy product URLs. Both legacy (https://www.bestbuy.com/site//.p?skuId=) and new (https://www.bestbuy.com/product//) formats supported. ## `searchQueries` (type: `array`): Keyword search queries (e.g. 'airpods', 'oled tv', 'lego'). The scraper enumerates SKUs from Best Buy's search results, then fetches each product's details. ## `categoryIds` (type: `array`): Best Buy category IDs (e.g. 'pcmcat144700050004' for headphones, 'abcat0204000' for headphones parent). Found on PLP URLs after '?id='. ## `maxResults` (type: `integer`): Cap on total products per (searchQuery) or per (categoryId). Discovery walks paginated results until this cap or end-of-results. ## `includeSpecifications` (type: `boolean`): When true (default), each product record includes the full spec table (specifications, highlights, features, dimensions, image gallery). Set false for a slim record (price/rating/availability only). ## `maxImagesPerProduct` (type: `integer`): Cap on image URLs returned per product. ## `requestConcurrency` (type: `integer`): How many product detail requests to run in parallel against Best Buy. Higher = faster runs but lower per-SKU pull-through under Akamai pressure. Default 1 gives the highest pull-through; try 2-3 if you care more about throughput than completeness. ## `proxyConfiguration` (type: `object`): US Residential proxies strongly recommended. Best Buy's Akamai Bot Manager closes HTTP/2 streams after 6-8 sustained requests per IP; the scraper rotates IP sessions automatically. ## Actor input object example ```json { "skuIds": [], "productUrls": [], "searchQueries": [ "airpods" ], "categoryIds": [], "maxResults": 5, "includeSpecifications": true, "maxImagesPerProduct": 10, "requestConcurrency": 1, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } } ``` # Actor output Schema ## `dataset` (type: `string`): Dataset of Best Buy product records. ## `datasetCsv` (type: `string`): Same dataset rendered as CSV. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchQueries": [ "airpods" ], "maxResults": 5, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } }; // Run the Actor and wait for it to finish const run = await client.actor("cirkit/bestbuy-product-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchQueries": ["airpods"], "maxResults": 5, "proxyConfiguration": { "useApifyProxy": True, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "US", }, } # Run the Actor and wait for it to finish run = client.actor("cirkit/bestbuy-product-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchQueries": [ "airpods" ], "maxResults": 5, "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } }' | apify call cirkit/bestbuy-product-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=cirkit/bestbuy-product-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Best Buy Product Scraper", "description": "Fast Best Buy product scraper. Pull product details (title, prices, savings, SKU, BSIN, model number, brand, ratings, reviews, specifications, images, category path) by SKU, direct URL, search keyword, or category. Reads Best Buy's GraphQL gateway directly, no headless browser.", "version": "0.1", "x-build-id": "hQu0LoKeXCSRLNCda" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/cirkit~bestbuy-product-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-cirkit-bestbuy-product-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/cirkit~bestbuy-product-scraper/runs": { "post": { "operationId": "runs-sync-cirkit-bestbuy-product-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/cirkit~bestbuy-product-scraper/run-sync": { "post": { "operationId": "run-sync-cirkit-bestbuy-product-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "skuIds": { "title": "Best Buy SKUs", "uniqueItems": true, "type": "array", "description": "List of Best Buy 7-digit SKU IDs (e.g. '6447382'). Bypasses search and fetches each SKU directly via the public GraphQL gateway. Fastest path.", "default": [], "items": { "type": "string" } }, "productUrls": { "title": "Direct Product URLs", "uniqueItems": true, "type": "array", "description": "List of Best Buy product URLs. Both legacy (https://www.bestbuy.com/site//.p?skuId=) and new (https://www.bestbuy.com/product//) formats supported.", "default": [], "items": { "type": "string" } }, "searchQueries": { "title": "Search Keywords", "uniqueItems": true, "type": "array", "description": "Keyword search queries (e.g. 'airpods', 'oled tv', 'lego'). The scraper enumerates SKUs from Best Buy's search results, then fetches each product's details.", "default": [], "items": { "type": "string" } }, "categoryIds": { "title": "Best Buy Category IDs", "uniqueItems": true, "type": "array", "description": "Best Buy category IDs (e.g. 'pcmcat144700050004' for headphones, 'abcat0204000' for headphones parent). Found on PLP URLs after '?id='.", "default": [], "items": { "type": "string" } }, "maxResults": { "title": "Max Products Per Search", "minimum": 1, "maximum": 5000, "type": "integer", "description": "Cap on total products per (searchQuery) or per (categoryId). Discovery walks paginated results until this cap or end-of-results.", "default": 50 }, "includeSpecifications": { "title": "Include Full Specifications", "type": "boolean", "description": "When true (default), each product record includes the full spec table (specifications, highlights, features, dimensions, image gallery). Set false for a slim record (price/rating/availability only).", "default": true }, "maxImagesPerProduct": { "title": "Max Image URLs Per Product", "minimum": 0, "maximum": 50, "type": "integer", "description": "Cap on image URLs returned per product.", "default": 10 }, "requestConcurrency": { "title": "Concurrent Product Fetches", "minimum": 1, "maximum": 5, "type": "integer", "description": "How many product detail requests to run in parallel against Best Buy. Higher = faster runs but lower per-SKU pull-through under Akamai pressure. Default 1 gives the highest pull-through; try 2-3 if you care more about throughput than completeness.", "default": 1 }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "US Residential proxies strongly recommended. Best Buy's Akamai Bot Manager closes HTTP/2 streams after 6-8 sustained requests per IP; the scraper rotates IP sessions automatically.", "default": { "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ], "apifyProxyCountry": "US" } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```