# Best Buy Product Catalog Scraper (`automation-lab/best-buy-product-catalog-scraper`) Actor

Scrape Best Buy US products by keyword, category, search URL, or product URL. Get SKUs, models, prices, stock, fulfillment, ratings, images, and catalog details.

- **URL**: https://apify.com/automation-lab/best-buy-product-catalog-scraper.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** E-commerce
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Best Buy Product Catalog Scraper

Collect structured Best Buy US product catalog data without a Best Buy API key.
Search by keyword, submit a Best Buy search or category URL, or enrich a known product URL.
The Actor returns one normalized row per SKU with product identity, pricing, availability, fulfillment, ratings, images, categories, highlights, and timestamps.

Best Buy Product Catalog Scraper is designed for repeatable ecommerce research rather than one-off page copying.
It deduplicates SKUs across inputs, follows result pagination, and uses the public catalog data embedded in Best Buy pages.

### What can Best Buy Product Catalog Scraper do?

- Discover products from one or more Best Buy search terms.
- Process public Best Buy search and category URLs.
- Resolve product URLs through their Best Buy SKU.
- Export canonical product URLs, SKUs, BSINs, brands, and model numbers.
- Capture current and regular price, savings, and USD currency.
- Capture purchase state plus shipping, pickup, and delivery eligibility.
- Capture rating, review count, image, seller, release date, and catalog highlights.
- Deduplicate products across all submitted sources.
- Stop exactly at your configured `maxItems` limit.
- Produce integration-ready JSON, CSV, Excel, XML, RSS, or JSONL through Apify datasets.

### Who is it for?

#### Ecommerce intelligence teams

Refresh assortment, price, promotion, and availability datasets on a recurring schedule.
Use stable SKUs to compare snapshots over time.

#### Brands and manufacturers

Monitor how products appear in Best Buy search, which models are active, and how ratings or pricing change.

#### Resellers and procurement analysts

Build shortlists using current price, store pickup, shipping eligibility, model, and rating signals.

#### Price-comparison products

Feed normalized Best Buy records into a multi-retailer catalog using SKU, model, canonical URL, and timestamps.

#### Researchers and developers

Prototype Best Buy catalog workflows without maintaining proxy sessions, hydration parsers, pagination, or dataset exports.

### Why use this Actor?

Best Buy pages contain useful product data in application hydration payloads, but those payloads are large and their delivery can be sensitive to network identity.
The Actor handles the operational details:

1. it creates stable US residential sessions;
2. it retries transient resets and challenge responses with a different session;
3. it parses product objects structurally instead of scraping presentation-only selectors;
4. it normalizes nested pricing and fulfillment fields;
5. it deduplicates SKUs before charging or saving records.

The HTTP-first architecture avoids browser rendering and unnecessary image downloads.
That keeps runs lighter while preserving the catalog fields visible in Best Buy's public search experience.

### What Best Buy product data is extracted?

| Field | Meaning |
| --- | --- |
| `url` | Canonical SKU-specific Best Buy product URL |
| `sourceUrl` | Search or URL source that produced the product |
| `sku` | Best Buy SKU |
| `bsin` | Best Buy product identifier used in newer URLs |
| `title`, `shortTitle` | Full and short product names |
| `brand`, `model`, `color` | Brand, manufacturer model, and displayed color/variant |
| `condition` | Catalog condition, usually `new` |
| `categories` | Product type/category labels |
| `currentPrice`, `regularPrice` | Current and regular displayed prices |
| `currency` | `USD` for Best Buy US |
| `savings`, `savingsPercent` | Current displayed discount |
| `availability` | Purchase button state such as `ADD_TO_CART` |
| `fulfillment` | Shipping, pickup, delivery, store, and ZIP signals |
| `rating`, `reviewCount` | Aggregate customer feedback |
| `primaryImage`, `images` | Product image URLs |
| `highlights` | Public catalog highlights |
| `specifications` | Labeled specifications when present in the catalog response |
| `releaseDate`, `seller` | Release and seller classification metadata |
| `scrapedAt` | UTC extraction timestamp |

Every upstream field can be absent for some products.
Treat nullable fields as expected rather than as an extraction failure.

### How to scrape Best Buy products

1. Open the Actor in Apify Console.
2. Keep the prefilled `wireless headphones` search or enter your own terms.
3. Optionally add Best Buy product, search, or category URLs.
4. Choose a practical `maxItems` limit.
5. Keep the default proxy configuration unless you have tested another route.
6. Click **Start**.
7. Open the **Products** dataset view.
8. Export the results or connect the dataset to your workflow.

A small first run is the easiest way to confirm that the selected terms return the products you expect.

### Input parameters

#### `searchTerms`

An array of Best Buy keyword searches.
Examples include `wireless headphones`, `gaming laptop`, or `oled tv`.
Each term is paginated independently.

#### `startUrls`

Optional Best Buy US URLs.
Supported intent includes:

- product URLs containing `skuId` or a `/sku/{id}` path;
- Best Buy search URLs;
- public Best Buy category/listing URLs that expose catalog products.

URLs on non-Best-Buy hosts fail validation.
A product URL is resolved using its SKU against the catalog search surface.

#### `maxItems`

Maximum unique product rows across the whole run.
The default is 20 and the supported range is 1–1,000.

#### `maxPagesPerSource`

Pagination safety limit for each search term or URL.
The default is 5 and the maximum is 20.
The run also stops as soon as `maxItems` is reached.

#### `proxyConfiguration`

Apify proxy settings.
When omitted, the Actor uses sticky US residential sessions because that is the tested reliable route.
Changing proxy geography can alter availability and fulfillment data or increase blocking.

### Example input

```json
{
  "searchTerms": ["wireless headphones", "noise cancelling earbuds"],
  "startUrls": [
    {
      "url": "https://www.bestbuy.com/site/apple-airpods-4-white/6447384.p?skuId=6447384"
    }
  ],
  "maxItems": 25,
  "maxPagesPerSource": 3
}
```

Search terms and URLs can be combined.
If the same SKU appears more than once, it is saved and charged only once.

### Best Buy product output example

A current product record has this shape:

```json
{
  "url": "https://www.bestbuy.com/product/apple-airpods-4-white/JJGCQ83JQ5/sku/6447384",
  "sourceUrl": "https://www.bestbuy.com/site/searchpage.jsp?st=airpods",
  "sku": "6447384",
  "bsin": "JJGCQ83JQ5",
  "title": "Apple - AirPods 4 - White",
  "shortTitle": "AirPods 4",
  "brand": "Apple",
  "model": "MXP63LL/A",
  "color": "White",
  "condition": "new",
  "categories": ["Headphones", "Wireless Headphones"],
  "currentPrice": 129.99,
  "regularPrice": 129.99,
  "currency": "USD",
  "availability": "ADD_TO_CART",
  "rating": 4.7,
  "reviewCount": 10362,
  "scrapedAt": "2025-01-15T12:00:00.000Z"
}
```

Prices, ratings, counts, fulfillment, and availability change over time.
The example documents the output shape, not a price promise.

### How much does it cost to scrape Best Buy products?

The Actor uses pay per event pricing:

- a one-time **Start** event per run;
- one **Product** event for each unique row saved.

At the current BRONZE rate, the start is **$0.005** and each product is **$0.00087125**.
A 10-product run is about **$0.01371**.
A 100-product run is about **$0.09213**.
A 1,000-product run is about **$0.87625**.

Higher Apify subscription tiers receive lower per-product prices as displayed on the Actor page.
Actual output count can be lower than `maxItems` when Best Buy has fewer matching products.
Proxy and compute usage are covered by the pay-per-event model rather than billed as a separate Actor event.

### Tips for reliable catalog collection

- Start with specific buyer keywords if broad searches contain irrelevant products.
- Use SKU-based product URLs when you need stable entity enrichment.
- Schedule the same input and compare rows by `sku` for price or stock changes.
- Keep the default US residential proxy route.
- Increase `maxPagesPerSource` only when you also need a larger `maxItems` value.
- Inspect `sourceUrl` when diagnosing why a product appeared.
- Use `scrapedAt` to separate snapshots.
- Store historic snapshots outside a single run dataset if you need long-term trends.

### Limits and failure behavior

Best Buy controls the public source and can change its page or hydration structure.
The Actor retries transient resets, temporary server errors, rate limits, and recognizable challenge pages with bounded session rotation.
It fails the run after retry exhaustion instead of silently returning a misleading empty dataset.

Fulfillment is location-sensitive.
The returned store and ZIP reflect the location context Best Buy supplied to the session, not every US location.

Some products omit regular price, savings, model, category, release date, fulfillment, highlights, or labeled specifications.
Those fields remain nullable.

The Actor covers Best Buy US public catalog data.
It does not log into customer accounts, access member-only history, place orders, or bypass private access controls.

### Automate price and availability monitoring

A common recurring workflow is:

1. save a Task with your product URLs or search terms;
2. schedule it daily or weekly;
3. use a webhook when the run succeeds;
4. fetch the dataset by SKU;
5. compare `currentPrice`, `availability`, `fulfillment`, `rating`, and `reviewCount` with the previous snapshot;
6. send alerts only for meaningful changes.

Apify integrations can send results to Google Sheets, Slack, Zapier, Make, webhooks, cloud storage, or your own API.
For large recurring catalogs, keep one stable Task input so changes are attributable to source data rather than input drift.

### Use the Apify API with cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~best-buy-product-catalog-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchTerms":["wireless headphones"],"maxItems":20}'
```

To wait for completion and receive dataset items directly:

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~best-buy-product-catalog-scraper/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchTerms":["wireless headphones"],"maxItems":20}'
```

Keep API tokens in environment variables or a secret manager.

### Use the Apify API with JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/best-buy-product-catalog-scraper').call({
  searchTerms: ['gaming laptop'],
  maxItems: 50,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.map(({ sku, currentPrice, availability }) => ({
  sku,
  currentPrice,
  availability,
})));
```

### Use the Apify API with Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("automation-lab/best-buy-product-catalog-scraper").call(
    run_input={"searchTerms": ["oled tv"], "maxItems": 30}
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["sku"], item.get("currentPrice"), item.get("availability"))
```

### Use Best Buy Product Catalog Scraper through MCP

Add the Apify MCP server to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/best-buy-product-catalog-scraper"
```

#### Claude Desktop

Add this MCP server configuration to Claude Desktop:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/best-buy-product-catalog-scraper"
    }
  }
}
```

#### Cursor

Add the same `apify` MCP server URL in Cursor's MCP settings.

#### VS Code

Add the same HTTP MCP server to your VS Code MCP configuration and enable it for the workspace.

#### Example prompts for MCP

Use prompts such as:

- “Find 20 highly rated wireless headphones on Best Buy and return SKU, model, price, and pickup eligibility.”
- “Extract the current Best Buy catalog row for this product URL and summarize its fulfillment options.”
- “Build a 50-item gaming laptop dataset for weekly price monitoring.”

### Responsible use and legality

Best Buy product pages are public, but public availability does not remove every legal or contractual obligation.
Use the Actor only for data you are authorized to collect and process.

- Review Best Buy's current terms and applicable robots guidance.
- Use conservative limits and schedules.
- Do not collect or infer private customer information.
- Do not use the output for deceptive pricing or inventory claims.
- Respect trademark, database, consumer-protection, and privacy law.
- Keep time-sensitive price and availability claims timestamped.

You are responsible for your use of the data and for complying with laws that apply to your location and use case.

### Troubleshooting

#### The run returned no products

Check that the keyword has results on Best Buy US.
For URLs, confirm the host is `bestbuy.com` and that the product or listing remains public.
Try a small `maxItems` value with the default proxy configuration.

#### The run failed after several retries

Best Buy may have temporarily blocked or reset every bounded session.
Wait before rerunning, keep the US residential configuration, and inspect the run log for status and retry reason.
The Actor intentionally fails instead of presenting a challenge page as zero valid results.

#### Fulfillment differs from my browser

Store pickup, delivery, and shipping are location-sensitive.
The Actor reports the public session's store/ZIP context.
It does not claim nationwide stock for a single product row.

#### A specification is missing

Best Buy's search catalog does not expose complete labeled specifications for every product.
Use `model`, `categories`, and `highlights` when available, and treat `specifications` as optional.

### Related Automation Lab Actors

- [Best Buy Reviews Scraper](https://apify.com/automation-lab/best-buy-reviews-scraper) — collect individual review text and reviewer-level feedback for Best Buy products after catalog discovery.

Use this catalog Actor for product-level assortment, price, stock, and aggregate rating rows.
Use the reviews Actor when your workflow needs the underlying review records.

### FAQ

#### Does this Actor require a Best Buy API key?

No.
It extracts public Best Buy US catalog data without a Best Buy developer credential.

#### Can I scrape a specific product URL?

Yes.
Submit a public Best Buy product URL in `startUrls`.
The Actor extracts its SKU and resolves the current catalog record.

#### Can I combine queries and URLs?

Yes.
All sources share one run-level `maxItems` limit and SKU deduplication set.

#### Does `maxItems` guarantee that many rows?

No.
It is an upper bound.
A narrower query, removed product, exhausted pagination, or Best Buy source behavior can yield fewer records.

#### Are results real time?

Results reflect the public catalog response received during the run.
Use `scrapedAt` and avoid describing old exports as current.

#### Does the Actor download product images?

No.
It returns image URLs but does not download image binaries, which reduces transfer and storage cost.

#### Does it scrape customer reviews?

It returns aggregate rating and review count.
Use the related Best Buy Reviews Scraper for individual review records.

#### Is a browser used for every product?

No.
The Actor is HTTP-first and parses server-rendered application data.
This avoids browser cost and per-product page rendering while the public catalog surface remains useful.

# Actor input Schema

## `searchTerms` (type: `array`):

Best Buy US keyword searches. Each term is paginated independently and product SKUs are deduplicated.

## `startUrls` (type: `array`):

Optional Best Buy US product, search, or category URLs. Product URLs are resolved through Best Buy's catalog search using their SKU.

## `maxItems` (type: `integer`):

Maximum number of unique product records saved across all searches and URLs.

## `maxPagesPerSource` (type: `integer`):

Safety limit for result pages requested for each search term or URL.

## `proxyConfiguration` (type: `object`):

Proxy settings for Best Buy. If omitted, the Actor uses sticky US residential sessions because they are the reliable tested route.

## Actor input object example

```json
{
  "searchTerms": [
    "wireless headphones"
  ],
  "startUrls": [],
  "maxItems": 20,
  "maxPagesPerSource": 5,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ],
    "apifyProxyCountry": "US"
  }
}
```

# Actor output Schema

## `dataset` (type: `string`):

Best Buy products returned by the configured searches and URLs.

## `overview` (type: `string`):

Table view of the most useful product fields.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchTerms": [
        "wireless headphones"
    ],
    "startUrls": [],
    "maxItems": 20,
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ],
        "apifyProxyCountry": "US"
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/best-buy-product-catalog-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchTerms": ["wireless headphones"],
    "startUrls": [],
    "maxItems": 20,
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
        "apifyProxyCountry": "US",
    },
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/best-buy-product-catalog-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchTerms": [
    "wireless headphones"
  ],
  "startUrls": [],
  "maxItems": 20,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ],
    "apifyProxyCountry": "US"
  }
}' |
apify call automation-lab/best-buy-product-catalog-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=automation-lab/best-buy-product-catalog-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/jz4rbO5qd6Q2wiY0Q/builds/6xr92Pf8xmlV9OlBe/openapi.json