# Shopify Store Products Scraper (`zerahdata/shopify-store-products-scraper`) Actor

Extract full product catalogs from any Shopify store: titles, prices, sale prices, variants, SKUs, stock and images. Export to Excel.

- **URL**: https://apify.com/zerahdata/shopify-store-products-scraper.md
- **Developed by:** [Zerah Data](https://apify.com/zerahdata) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Shopify Store Products Scraper

Extract the full, public product catalog from any Shopify store - titles, prices,
variants, images, inventory signals, and more - and track how that catalog changes
over time, without writing a single line of scraping code.

The actor reads the storefront `products.json` endpoint that Shopify exposes on
every store by default, paginates through the entire catalog, and normalizes
the result into a clean, flat dataset that is ready for a spreadsheet, BI tool,
or your own pipeline. Turn on change tracking and the same actor becomes a
price/stock monitor: it remembers what it saw last time and reports exactly
what changed.

### Why this actor

- **Monitoring built in, not left to you.** Turn on change tracking and every
  run after the first reports new products, removed products, price drops and
  increases (with percent change), sales starting/ending, and stock changes -
  no need to download two datasets and diff them yourself.
- **Insight rows, not just raw JSON.** An optional per-store summary row gives
  you product/variant counts, % on sale, average discount, price
  min/median/max, in-stock %, top vendors and product types, and newest
  products in one row - a one-screen read of a store's catalog health.
- **One simple price.** A single pay-per-result event, no stacked add-on fees
  for extra fields - change rows and store summaries are priced the same as a
  product row.
- **Collection URLs actually scrape just that collection.** Point it at
  `/collections/sale` and only that collection's products are fetched - full
  catalog URLs aren't silently substituted.
- **Automatic fallback when a store blocks the main feed.** Some stores
  return 401/403/429 for `/products.json` specifically from Apify's
  datacenter IPs. The actor retries with a fresh browser fingerprint and (if
  a proxy is configured) a new proxy session, then tries
  `/collections/all/products.json`, then falls back to discovering products
  via the store's `/sitemap.xml` and fetching each product's AJAX JSON - all
  before giving up. Every row records which method produced it (`source`).
- **Built-in self-check.** Every run writes a `RUN_REPORT` record (in the
  run's key-value store) with inputs processed, row counts by type, errors by
  code, warnings (e.g. a store returning 0 products), and a validation pass
  confirming required fields were non-empty in every output row.
- **Partial results, not all-or-nothing.** If a store fails midway through a
  large catalog, everything already fetched is kept and marked `partial` in
  the error row - no need to rerun to get the pages it already managed to
  fetch.
- **No surprise catalog truncation.** There is no low default cap hiding most
  of a store's catalog - leave **Max products per store** at `0` and you get
  everything.

### What it does

Give it one or more store URLs and it will:

- Resolve the store (accepts a full URL or a specific collection URL).
- Follow redirects if the store has moved to a different domain.
- Page through the entire product catalog (or a capped number of products).
- Fall back to an alternative pagination method automatically if a store disables
  the default one partway through.
- Fall back further - fresh browser fingerprint/proxy session, an alternative
  endpoint, then sitemap + per-product AJAX JSON - if the main feed is blocked
  outright or returns HTML instead of JSON. See "Fallback chain" below.
- Pull the store's active currency.
- Detect and clearly report stores that are not running Shopify, or that have
  disabled/blocked the public products feed, instead of silently failing.
- Output one row per product (with a nested variants array) or one row per
  variant, depending on how you configure it.
- Optionally compare this run's catalog to the last run's and output rows
  describing exactly what changed.
- Optionally output one aggregate "store summary" row per store.
- Write a `RUN_REPORT` self-check record at the end of every run.

### Use cases

- **Price & stock monitoring** - enable **Track changes between runs** and
  schedule the actor; each run reports price drops/increases, sales
  starting/ending, restocks and stockouts, and new/removed products.
- **Competitor research** - see what products, vendors, and tags a store carries,
  plus a one-row summary of its assortment and pricing.
- **Catalog export / migration** - pull a full product list out of a Shopify
  store you manage or are migrating away from.
- **Market research** - analyze product assortment, naming, and pricing patterns
  across many stores in one run.

### How to use

1. Add one or more full store URLs to **Store URLs**, e.g.:

   - `https://www.allbirds.com`
   - `https://www.allbirds.com/collections/mens-shoes` (scrapes only that collection)

   Use full URLs here (`https://...`). If you're calling the Apify API
   directly rather than using the Console UI and only have bare domains
   (e.g. `allbirds.com`), put those in the separate **Store domains** field
   instead - the list editor used by **Store URLs** rejects bare domains
   when submitted via the API. Both fields are merged into one list of
   stores to scrape.
2. (Optional) Set **Max products per store** to cap how many products are
   fetched per store. Leave at `0` to fetch the entire catalog.
3. (Optional) Enable **Output one row per variant** if you want a row for
   every size/color/variant combination instead of one row per product.
4. (Optional) Enable **Track changes between runs** to turn this into a
   monitor - see "Change tracking" below.
5. (Optional) Turn on a proxy under **Proxy configuration** if you are
   scraping many stores or very large catalogs.
6. Run the actor.

#### Example input

```json
{
  "startUrls": [
    { "url": "https://www.allbirds.com" },
    { "url": "https://www.gymshark.com" }
  ],
  "storeDomains": ["colourpop.com"],
  "maxProductsPerStore": 0,
  "includeVariants": false,
  "proxyConfiguration": { "useApifyProxy": false }
}
```

#### Example input with monitoring

```json
{
  "startUrls": [{ "url": "https://www.allbirds.com" }],
  "trackChanges": true,
  "monitorName": "allbirds-watch",
  "outputMode": "changesOnly",
  "includeStoreSummary": true
}
```

Run this on a schedule. The first run saves a baseline snapshot and reports no
changes; every run after that reports what moved since the previous run.

### Output example

Default mode (one row per product):

```json
{
  "type": "product",
  "storeUrl": "https://www.allbirds.com",
  "storeDomain": "www.allbirds.com",
  "productId": 1234567890,
  "title": "Men's Wool Runner",
  "handle": "mens-wool-runner",
  "url": "https://www.allbirds.com/products/mens-wool-runner",
  "vendor": "Allbirds",
  "productType": "Shoes",
  "tags": ["men", "running", "wool"],
  "descriptionText": "A comfortable, breathable everyday sneaker...",
  "price": 98,
  "compareAtPrice": null,
  "currency": "USD",
  "available": true,
  "images": ["https://cdn.shopify.com/.../image1.jpg"],
  "featuredImage": "https://cdn.shopify.com/.../image1.jpg",
  "createdAt": "2021-01-01T00:00:00-08:00",
  "updatedAt": "2024-05-01T00:00:00-07:00",
  "publishedAt": "2021-01-02T00:00:00-08:00",
  "options": [{ "name": "Size", "values": ["7", "8", "9", "10"] }],
  "variants": [
    {
      "id": 9876543210,
      "title": "9",
      "sku": "WR-M-9",
      "price": 98,
      "compareAtPrice": null,
      "available": true,
      "inventoryQuantity": 12,
      "option1": "9",
      "option2": null,
      "option3": null,
      "grams": 300,
      "requiresShipping": true,
      "barcode": null,
      "featuredImage": null
    }
  ],
  "minPrice": 98,
  "maxPrice": 98,
  "anyAvailable": true,
  "variantCount": 8,
  "onSale": false,
  "discountPercent": null,
  "inStockVariantCount": 8,
  "totalVariantCount": 8,
  "source": "products_json",
  "scrapedAt": "2026-01-01T12:00:00.000Z"
}
```

`source` records which method produced the row: `products_json` (the
default feed), `collections_all`, or `sitemap_ajax` (see "Fallback chain"
below).

With **Output one row per variant** enabled, each row has the same product
fields plus `variantId`, `variantTitle`, `sku`, `price`, `compareAtPrice`,
`available`, `inventoryQuantity`, `option1`/`option2`/`option3`, `variantImage`,
`onSale` (true when `compareAtPrice` is higher than `price`), `discountPercent`,
and `totalVariantCount`/`inStockVariantCount` for the parent product.

If a store can't be scraped, you'll get a clear error row instead of a silent
gap in your data:

```json
{
  "type": "error",
  "storeUrl": "https://www.wikipedia.org",
  "error": "NOT_SHOPIFY",
  "message": "...",
  "partial": false,
  "scrapedAt": "2026-01-01T12:00:00.000Z"
}
```

`partial: true` means the store failed partway through, but every product
page fetched before the failure is already in the dataset.

### Fallback chain

A small number of stores return 401/403/429 for `/products.json` specifically
from Apify's datacenter IPs (both direct requests and the Apify datacenter
proxy group), even though the same request succeeds from other IPs. When the
main feed is blocked (401/403, or 429 that persists after retries) or returns
an HTML page instead of JSON, the actor works through the following chain
before giving up on a store, and stops at the first method that returns real
data:

1. **Retry `/products.json`** (or the collection-scoped equivalent) up to 3
   times, each attempt using a freshly generated browser fingerprint and, if
   a proxy is configured, a brand-new proxy session.
2. **`/collections/all/products.json`** as an alternative endpoint.
3. **Sitemap fallback** - read `/sitemap.xml`, follow its
   `sitemap_products_*.xml` entries to collect product URLs, then fetch each
   product's `/products/<handle>.js` AJAX JSON directly (at low concurrency,
   respecting **Max products per store**). Prices on this endpoint are in
   cents; the actor normalizes them to the same decimal units and field names
   used by `/products.json` before they reach the dataset.

Steps 2 and 3 only run for whole-store crawls (not for a URL scoped to one
specific collection), since neither can be filtered to a single collection.
Each dataset row records which method produced it in its `source` field. If
every method fails, the store gets an `error` row with `error: "BLOCKED"`
advising you to enable **Proxy configuration** with the **RESIDENTIAL** proxy
group for that store - datacenter IPs (Apify's own or Apify Proxy's
datacenter group) are what tends to get blocked.

### Change tracking (monitoring mode)

Enable **Track changes between runs** (`trackChanges: true`) to turn repeated
runs into a monitor. The actor keeps a compact snapshot per store (per
product/variant: price, compare-at price, availability, title) in a named
key-value store (`shopify-monitor-<monitorName>`), and compares the new
snapshot to the one saved last time. Every difference becomes a `change` row:

```json
{
  "type": "change",
  "changeType": "PRICE_DROP",
  "storeUrl": "https://www.allbirds.com",
  "storeDomain": "www.allbirds.com",
  "productId": "1234567890",
  "productTitle": "Men's Wool Runner",
  "url": "https://www.allbirds.com/products/mens-wool-runner",
  "variantId": "9876543210",
  "variantTitle": "9",
  "sku": "WR-M-9",
  "field": "price",
  "oldValue": 108,
  "newValue": 98,
  "percentChange": -9.3,
  "currency": "USD",
  "scrapedAt": "2026-01-01T12:00:00.000Z"
}
```

`changeType` is one of: `NEW_PRODUCT`, `REMOVED_PRODUCT`, `PRICE_DROP`,
`PRICE_INCREASE`, `SALE_STARTED`, `SALE_ENDED`, `OUT_OF_STOCK`, `BACK_IN_STOCK`.
Change rows are charged the same as a product row.

- **Monitor name**: give runs that should be compared against each other the
  same `monitorName` (e.g. a scheduled task). Leave it empty and the actor
  derives one from your store URLs automatically.
- **First run**: there is nothing to compare against yet, so it only saves a
  baseline and reports zero changes - this is expected.
- **`REMOVED_PRODUCT` and partial crawls**: if **Max products per store** is
  set above `0`, the actor can't tell "removed" apart from "not reached this
  time", so removed-product detection is skipped and a warning is written to
  the run report instead of a false positive.
- **`outputMode`**: set to `"changesOnly"` to suppress regular product/variant
  rows and keep only `change` rows (plus the store summary and error rows,
  which are controlled separately).

### Store summary rows

With **Include per-store summary row** enabled (default), each store adds one
`storeSummary` row:

```json
{
  "type": "storeSummary",
  "storeUrl": "https://www.allbirds.com",
  "storeDomain": "www.allbirds.com",
  "currency": "USD",
  "productCount": 412,
  "variantCount": 1830,
  "onSaleProductCount": 37,
  "onSalePercent": 9,
  "avgDiscountPercent": 22.4,
  "priceMin": 20,
  "priceMedian": 98,
  "priceMax": 250,
  "inStockProductCount": 390,
  "inStockPercent": 94.7,
  "topVendors": [{ "name": "Allbirds", "count": 412 }],
  "topProductTypes": [{ "name": "Shoes", "count": 300 }],
  "newestProducts": [{ "title": "...", "url": "...", "createdAt": "..." }],
  "collectionCount": 48,
  "sampleCollections": [{ "title": "Men's Shoes", "handle": "mens-shoes" }],
  "scrapedAt": "2026-01-01T12:00:00.000Z"
}
```

`collectionCount`/`sampleCollections` come from one extra cheap request to the
store's collections feed (no per-collection product counts, which would need
one request per collection). This row is charged as a single result.

### Run report (self-check)

At the end of every run, the actor writes a `RUN_REPORT` record to the
default key-value store with:

- `inputsProcessed`: how many store URLs were given.
- `rows`: counts of products, variants, changes, store summaries, and errors
  actually written to the dataset.
- `errorsByCode`: how many stores failed with each error code.
- `warnings`: notable non-fatal conditions, e.g. a store returning 0
  products, or removed-product detection being skipped for a capped crawl.
- `validation`: whether every pushed row had its required fields (title, URL,
  store domain, etc.) non-empty.
- `chargedResultCount`: how many billable results this run produced.

### Tips

- **Collection URLs**: point **Store URLs** at a specific collection (e.g.
  `/collections/sale`) to scrape just that subset of a catalog instead of
  the entire store.
- **Variant mode**: enable it when you need SKU-level or size/color-level
  pricing and inventory, such as for feed generation or inventory tracking.
- **Monitoring**: keep **Monitor name** fixed across scheduled runs if you
  plan to add or remove store URLs over time, so tracking stays continuous
  per store regardless of which other stores are in the same run.
- **Proxies**: for very large stores or when scraping many stores in one run,
  enable a proxy to reduce the chance of rate limiting or temporary blocks.
  If a store ends up `BLOCKED` even with a datacenter proxy, switch to the
  **RESIDENTIAL** proxy group - some stores specifically block datacenter IPs.
- **Bare domains via the API**: if you're calling the Apify API directly and
  only have bare domains (no `https://`), put them in **Store domains**
  instead of **Store URLs** - that field's editor rejects bare domains when
  used that way.

### Pricing

This actor uses **pay-per-result** pricing: **$1.00 per 1,000 results**, with
no start fee. A result is one product row, one variant row (when **Output
one row per variant** is enabled), one `change` row, or one `storeSummary`
row. Error rows for stores that can't be scraped are never charged. You can
cap spending with the run's maximum cost setting, and the actor stops as soon
as that limit is reached.

### FAQ

**Is this legal?** The actor only reads data that a Shopify store already
publishes publicly on its own storefront (the same `products.json` feed used
by the store's own theme). It does not access private data, log in, or bypass
any access controls.

**What happens if a URL isn't a Shopify store?** You'll get an error row with
`error: "NOT_SHOPIFY"` for that store instead of a failed run - all other
stores in the same run are unaffected.

**What other errors can show up?** `PRODUCTS_JSON_DISABLED` (the store is on
Shopify but has turned off or restricted the public products feed),
`BLOCKED` (the store rejected every method in the fallback chain - see
"Fallback chain" above; try enabling **Proxy configuration** with the
**RESIDENTIAL** group), and `FETCH_FAILED` (a network-level failure, such as
a DNS error or timeout).

**Why did I get no `change` rows on my first monitored run?** The first run
with a given monitor name has nothing to compare against - it saves the
baseline. Run it again later and changes will appear.

**What are the limits?** Only products that are published to the Online Store
sales channel are visible through this feed - unpublished, draft, or
password-protected-storefront products won't appear. Inventory quantities are
only included when a store chooses to expose them publicly; otherwise that
field will be `null`.

# Actor input Schema

## `startUrls` (type: `array`):

Shopify store URLs to scrape. Use full URLs, e.g. https://www.allbirds.com, or a collection URL (e.g. https://store.com/collections/sale) to scrape only that collection's products.json. Note: when calling the Apify API directly (not the Console UI), this field's editor rejects bare domains like 'allbirds.com' - use the 'Store domains' field below instead if you only have bare domains.

## `storeDomains` (type: `array`):

Optional additional stores, merged with 'Store URLs' above. Unlike 'Store URLs', this field accepts bare domains (e.g. colourpop.com) with no 'https://' prefix required - useful when calling the Apify API directly, since the 'Store URLs' editor rejects bare domains there.

## `maxProductsPerStore` (type: `integer`):

Maximum number of products to fetch from each store. Set to 0 for unlimited (scrape the entire catalog).

## `includeVariants` (type: `boolean`):

If enabled, the dataset will contain one row per product variant (with product fields flattened onto each variant). If disabled (default), the dataset contains one row per product with a nested 'variants' array and summary price/availability fields.

## `proxyConfiguration` (type: `object`):

Optional proxy configuration. Using Apify Proxy (residential or datacenter) is recommended for scraping many or large stores to avoid IP blocking.

## `maxConcurrency` (type: `integer`):

Maximum number of stores to scrape in parallel.

## `requestDelayMs` (type: `integer`):

Optional delay in milliseconds between consecutive page requests to the same store, to reduce the chance of being rate-limited or blocked.

## `trackChanges` (type: `boolean`):

If enabled, the actor saves a compact snapshot of each store's products (price, compare-at price, availability, title) in a named key-value store, and compares it to the snapshot from the previous run with this same 'Monitor name'. Differences are output as extra dataset rows with type 'change' (new/removed products, price drops/increases, sales starting/ending, stock changes). The first run with a given monitor name only saves a baseline - it has nothing to compare against yet, so it reports no changes.

## `monitorName` (type: `string`):

Name identifying this monitoring setup, used to pick which saved snapshot to compare against (and update) on each run. Only relevant when 'Track changes between runs' is enabled. Leave empty to auto-derive one from the store URLs - but set a fixed value if you plan to change the list of store URLs between runs and still want continuous tracking.

## `outputMode` (type: `string`):

'All' outputs product/variant rows plus change rows (when change tracking is on). 'Changes only' suppresses product/variant rows and outputs only change rows - useful once you only care about what moved, not the full catalog. Store summary and error rows are controlled separately and are not affected by this setting.

## `includeStoreSummary` (type: `boolean`):

If enabled (default), each store gets one extra dataset row of type 'storeSummary' with aggregate insights: product/variant counts, % on sale, average discount, price min/median/max, in-stock %, top vendors and product types, newest products, currency, and a cheap store-wide collection count. Charged as a single result, same as any other row.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.allbirds.com"
    }
  ],
  "storeDomains": [],
  "maxProductsPerStore": 0,
  "includeVariants": false,
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "maxConcurrency": 3,
  "requestDelayMs": 0,
  "trackChanges": false,
  "monitorName": "",
  "outputMode": "all",
  "includeStoreSummary": true
}
```

# Actor output Schema

## `overview` (type: `string`):

All scraped product rows (or variant rows when per-variant output is enabled), plus error rows for stores that could not be scraped.

## `insights` (type: `string`):

Change-tracking rows (type 'change', only present when 'Track changes between runs' is enabled) and per-store summary rows (type 'storeSummary').

## `runReport` (type: `string`):

A RUN_REPORT record written to the default key-value store at the end of every run: inputs processed, row counts by type, errors by code, warnings (e.g. a store returning 0 products), and a validation pass confirming required fields were non-empty in output rows.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.allbirds.com"
        }
    ],
    "storeDomains": [],
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("zerahdata/shopify-store-products-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://www.allbirds.com" }],
    "storeDomains": [],
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("zerahdata/shopify-store-products-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.allbirds.com"
    }
  ],
  "storeDomains": [],
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call zerahdata/shopify-store-products-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,zerahdata/shopify-store-products-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ltSpo1q9g19nXM5wb/builds/YthDuNcDzogekI1I5/openapi.json