# Shopify Product Catalog Scraper (`theta_corvi/shopify-product-catalog-scraper`) Actor

Exports the complete product catalog of any public Shopify store - every product, every variant, prices, options and images - and verifies the export against the store's own published product count so you know whether you got all of it.

- **URL**: https://apify.com/theta\_corvi/shopify-product-catalog-scraper.md
- **Developed by:** [Nyx Ward](https://apify.com/theta_corvi) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 products

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Shopify Product Catalog Scraper

Read the full product catalog and every price of the Shopify stores you name —
one store or two hundred — and **know, per store, whether you got all of it**.

You give it a list of store domains. It gives you one clean row per product,
with the store on every row, and a run report saying, per store, whether the
export matched the number of products the store itself says it has published.

### Built for watching a set of stores

`storeUrls` is a list, not a field. One run covers a whole watchlist, and every
row carries its `store`, `currency`, `minPrice`, `maxPrice` and `scrapedAt`, so
a single CSV export pivots straight into a price sheet.

```json
{
  "storeUrls": [
    "allbirds.com",
    "brooklinen.com",
    "tentree.com",
    "chubbiesshorts.com"
  ],
  "maxProductsPerStore": 0
}
```

A store that cannot be read does not stop the run and costs you nothing per
product. It comes back with a status and a reason, and the other stores finish.

### Why "complete" is the point

If you are comparing today's prices to last week's, a silently short read is
worse than a failure. Twelve products missing from an export look exactly like
twelve products delisted. You act on a signal that never happened.

Shopify's storefront product feed makes that easy to get wrong. It has no
cursor. It is paged by offset, it carries no pagination metadata, and it
silently caps every page at 250 items no matter what you ask for. A catalog
edited while it is being read can shift under the offset, and nothing in the
response tells you it happened.

This Actor reads the store's own published product count first, deduplicates
every product by id, and compares the two at the end. If the count is short it
walks the catalog a second time and merges the results. Then it tells you the
outcome either way:

```
"complete": true,  "expectedProductCount": 293, "productCount": 293
```

**No other tool in this category reports that number.** Most report nothing at
all. It is the difference between a price sheet you can act on and one you have
to spot-check by hand.

### Input

| Field | Required | What it does |
|---|---|---|
| `storeUrls` | **yes** | One or more store domains. `allbirds.com`, `https://allbirds.com/collections/mens` and `weareallbirds.myshopify.com` all work — only the host is used |
| `maxProductsPerStore` | no | Safety cap per store. `0` (default) means no cap |
| `proxyConfiguration` | no | Apify proxy. If a store answers `BLOCKED`, re-run it with the RESIDENTIAL group |

This Actor never discovers stores on its own. It reads the ones you name, and
only those.

```json
{
  "storeUrls": ["allbirds.com", "deathwishcoffee.com"],
  "maxProductsPerStore": 0
}
```

### Output — one row per product

```json
{
  "schemaVersion": 1,
  "store": "allbirds.com",
  "storeUrl": "https://www.allbirds.com",
  "currency": "USD",
  "scrapedAt": "2026-08-30T21:00:00Z",
  "productId": 7218356060240,
  "handle": "mens-strider-explore",
  "title": "Men's Strider Explore - Natural Black",
  "url": "https://www.allbirds.com/products/mens-strider-explore",
  "vendor": "Allbirds",
  "productType": "Shoes",
  "tags": ["classic", "mens"],
  "publishedAt": "2025-10-01T09:35:00-07:00",
  "createdAt": "2025-09-10T12:47:47-07:00",
  "updatedAt": "2026-08-30T14:28:13-07:00",
  "bodyHtml": "<p>...</p>",
  "optionNames": ["Color", "Size"],
  "featuredImageUrl": "https://cdn.shopify.com/.../left.png",
  "imageUrls": ["https://cdn.shopify.com/.../left.png"],
  "imageCount": 5,
  "variantCount": 13,
  "minPrice": 130.0,
  "maxPrice": 130.0,
  "available": true,
  "variants": [
    {
      "variantId": 41334293889104,
      "title": "Natural Black / 9",
      "sku": "A11768M090",
      "price": 130.0,
      "compareAtPrice": null,
      "available": true,
      "position": 3,
      "grams": 954,
      "requiresShipping": true,
      "taxable": true,
      "options": { "Color": "Natural Black", "Size": "9" },
      "imageUrl": "https://cdn.shopify.com/.../left.png"
    }
  ]
}
```

The four things this saves you writing yourself:

1. **Option names resolved.** The feed keeps option *names* on the product and
   option *values* on the variant, joined by nothing but array position. Here
   they arrive as `{"Color": "Natural Black", "Size": "9"}`.
2. **Prices as numbers.** The feed sends `"130.00"` as a string, with no
   currency anywhere. Here they are floats, with the store's currency on every
   row.
3. **Variant images resolved.** The variant→image link lives in two different
   places and stores use one or the other, sometimes both in the same catalog.
   Both are read.
4. **Deduplicated and counted.** See above.

### The run report

The dataset stays one shape so a CSV export is usable. Everything about a
*store* rather than a product goes to the key-value record `OUTPUT`:

```json
{
  "scrapedAt": "2026-08-30T21:00:00Z",
  "storesRequested": 2,
  "storesRead": 1,
  "productsExported": 293,
  "stores": [
    { "store": "allbirds.com", "status": "OK", "complete": true,
      "expectedProductCount": 293, "productCount": 293,
      "duplicatesDropped": 0, "pagesFetched": 2, "passes": 1,
      "message": "Exported 293 products, which matches the 293 the store itself reports published. Catalog complete." },
    { "store": "example.com", "status": "NOT_SHOPIFY",
      "message": "No Shopify product feed at this domain (HTTP 404...). Check the domain." }
  ]
}
```

#### Every way a store can end

| Status | What it means |
|---|---|
| `OK` | The catalog was read. Check `complete` for whether it was whole |
| `NOT_SHOPIFY` | No Shopify product feed at this domain |
| `HEADLESS_STOREFRONT` | A Shopify store whose product feed is switched off — a headless or custom storefront. No tool can read it through this endpoint |
| `PASSWORD_PROTECTED` | Behind Shopify's password gate; the catalog is not public |
| `BLOCKED` | The store refused the request (HTTP 401/403/430). Re-run with the RESIDENTIAL proxy group |
| `RATE_LIMITED` | Throttled, and still throttled after three attempts |
| `UNREACHABLE` | No answer after three attempts |
| `ROBOTS_DISALLOWED` | The store's `robots.txt` disallows this endpoint. Skipped, and not charged |

### Limits, stated plainly

- **25,000 products per store.** That ceiling is Shopify's, not ours: past it
  the endpoint answers `HTTP 400 Page * Limit exceeds the 25000 limit`. Larger
  catalogs come back with `depthLimited: true` and `complete: false`.
- **Public catalog fields only.** Inventory quantities, cost, barcodes and the
  rest of the Admin API are not in this feed and are not invented here.
- **Published products only** — what a visitor to the storefront can see.

### What it does not do

Stated up front so you do not have to find out:

- **No change detection between runs.** It exports the catalog as it stands.
  Diffing consecutive runs is yours to do — schedule the runs and compare the
  datasets, or filter on `updatedAt`.
- **No store discovery.** It reads the domains you name and only those.
- **No headless storefronts.** A Shopify store that has switched the feed off
  comes back `HEADLESS_STOREFRONT`. No tool reads those through this endpoint.

#### How many stores actually expose the feed

Measured 2026-09-03 on a **sample of 12 domains, n = 12**: 8 exposed a readable
feed, 1 was a headless storefront, 3 had no Shopify feed at all.

That sample was assembled to exercise this scraper — large well-known brands,
one negative control, one at the size ceiling — **not** to estimate the Shopify
population, and it should not be read as a coverage rate for it. Run it on your
own list with `maxProductsPerStore: 1` and the run report answers the question
for your domains, one page per store.

### Pricing

You are charged per product exported. A store that cannot be read costs you
nothing beyond the run itself.

### Support

Open an issue on the Actor page. This Actor is maintained by Nyx Ward.

# Actor input Schema

## `storeUrls` (type: `array`):

One line per store. A bare domain (allbirds.com), a full URL (https://allbirds.com/collections/mens) or a myshopify domain all work - only the host is used. This field is required: the Actor never discovers stores on its own, it only reads the ones you name.

## `maxProductsPerStore` (type: `integer`):

Safety cap on how many products are exported per store. Leave at 0 for no cap. Whatever you set, the storefront endpoint itself cannot reach past 25000 products per store - that ceiling is Shopify's, not ours, and it is reported when it is hit.

## `proxyConfiguration` (type: `object`):

Some stores sit behind a WAF that answers datacenter addresses with HTTP 403. If a store comes back BLOCKED, re-run it with the RESIDENTIAL proxy group.

## Actor input object example

```json
{
  "storeUrls": [
    "allbirds.com"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `products` (type: `string`):

No description

## `runReport` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "storeUrls": [
        "allbirds.com"
    ],
    "maxProductsPerStore": 0,
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("theta_corvi/shopify-product-catalog-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "storeUrls": ["allbirds.com"],
    "maxProductsPerStore": 0,
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("theta_corvi/shopify-product-catalog-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "storeUrls": [
    "allbirds.com"
  ],
  "maxProductsPerStore": 0,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call theta_corvi/shopify-product-catalog-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,theta_corvi/shopify-product-catalog-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/GmKThf556Zf0gU2sn/builds/ySuRACoVOaaaO3YoU/openapi.json