# Shopify Products Scraper — Full Catalog Export (`sturdydata/shopify-products-scraper`) Actor

Export the complete product catalog of any Shopify store: titles, handles, vendors, tags, prices, variants with SKUs and availability, images and timestamps. Uses the store's own JSON endpoints (fast, no browser) and automatically falls back to product sitemaps to get past the 5,000-product pagin...

- **URL**: https://apify.com/sturdydata/shopify-products-scraper.md
- **Developed by:** [Sturdy Data](https://apify.com/sturdydata) (community)
- **Categories:** E-commerce
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## Shopify Products Scraper — Full Catalog Export

Export the **complete product catalog** of any Shopify store: titles, handles, vendors, product types, tags, prices, every variant with SKU / price / compare-at price / availability, image URLs and timestamps.

Built for: competitive pricing intelligence, assortment analysis, dropshipping research, MAP monitoring, catalog syncing.

### Why this scraper

Generic Shopify actors all share the same structural flaw: they paginate `/products.json` and silently stop at the **~5,000-product cap**. On a 30,000-SKU store you get a sixth of the catalog and no warning.

This actor:

- Uses the store's own JSON endpoints — fast, exact data, no browser, no HTML parsing errors.
- **Detects the 5,000-product wall and automatically switches to product sitemaps** (`sitemap_products_*.xml`) + per-product JSON, exporting the *full* catalog.
- Also handles stores that disable `/products.json` entirely (sitemap path kicks in).
- **Checks each store's robots.txt first** and skips stores that disallow these paths — compliance by construction.
- Per-store `RUN_SUMMARY` (key-value store) reports exactly how many products came from which path. No silent truncation, ever.

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `storeUrls` | array | — | Store homepages (custom domains fine). |
| `maxProductsPerStore` | integer | `5000` | Raise it for big catalogs — the sitemap fallback can go far beyond 5,000. |
| `includeVariants` | boolean | `true` | Full variant list per product, or counts only. |
| `proxyConfiguration` | object | Apify Proxy | Recommended: automatic. |

```json
{
    "storeUrls": ["https://www.allbirds.com"],
    "maxProductsPerStore": 20000
}
````

### Output schema

One dataset item per product:

```json
{
    "store": "www.allbirds.com",
    "storeUrl": "https://www.allbirds.com",
    "id": "7205181653072",
    "handle": "mens-cruiser-terralux-anthracite",
    "title": "Men's Cruiser Terralux - Anthracite",
    "url": "https://www.allbirds.com/products/mens-cruiser-terralux-anthracite",
    "vendor": "Allbirds",
    "productType": "Shoes",
    "tags": ["mens", "cruiser"],
    "priceMin": 120,
    "priceMax": 120,
    "available": true,
    "variantsCount": 14,
    "variants": [
        { "id": "41448181662", "title": "8", "sku": "A10611M080", "price": 120, "compareAtPrice": null, "available": true }
    ],
    "images": ["https://cdn.shopify.com/s/files/..."],
    "imagesCount": 6,
    "publishedAt": "2026-03-02T10:00:00-05:00",
    "createdAt": "2026-02-28T09:12:00-05:00",
    "updatedAt": "2026-07-01T02:11:00-04:00",
    "source": "products_json",
    "scrapedAt": "2026-07-04T10:00:00.000Z"
}
```

Notes: prices are numbers in the store's own currency (currency code is not exposed by these endpoints — it is the store's display currency). `source` tells you which path produced the item (`products_json` or `product_js` sitemap fallback). Fields a store does not publish are `null`.

### Pricing

**Pay per result. 1 result = 1 product** (with all its variants included).

| Products | Cost at $2.50 / 1,000 |
|---|---|
| 1,000 | $2.50 |
| 10,000 | $25.00 |
| 50,000 | $125.00 |

### Compliance & fair use

Public storefront endpoints only, robots.txt checked per store before any scraping, 1 request/second per store. No login walls, no checkout/cart endpoints, no customer data — catalog facts only.

### Using from API / MCP / AI agents

```bash
curl -X POST "https://api.apify.com/v2/acts/<ACTOR_ID>/run-sync-get-dataset-items?token=<TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{ "storeUrls": ["https://www.allbirds.com"], "maxProductsPerStore": 100 }'
```

Fully described input schema — works as an AI-agent tool via the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp).

### FAQ

**How do I know if the site is a Shopify store?** Check `<domain>/products.json` in your browser — if you see JSON, it works. If unsure, just run it: non-Shopify stores produce zero results and a clear note in `RUN_SUMMARY`.

**Why fewer products than the store claims?** Check the `RUN_SUMMARY` key-value store record (Storage → Key-value store in the run's console). Each store gets a `note` field that explains exactly what happened whenever results are zero or capped — e.g. robots.txt blocked scraping (`robotsBlocked: true`), the store isn't running standard Shopify, or its storefront doesn't expose the usual product endpoints. `note` is `null` when everything went as expected.

**Do you return product descriptions?** Not in v0.1 (keeps items small). If you need `body_html`, open an issue — it ships as an opt-in flag.

### Changelog

- **0.1.1** (2026-07) — `RUN_SUMMARY` now includes a `note` field per store that explains *why* a run returned zero or fewer products than expected (bot-blocked, non-Shopify storefront, missing sitemap, etc.) — no more silent zero-result runs.
- **0.1.0** (2026-07) — Initial release: products.json export, automatic sitemap fallback past the 5K cap, robots.txt pre-check, per-store run summary, variant-level data.

***

Maintained by **Sturdy Data** — boring, reliable scrapers. Report issues in the Issues tab; we respond within 24 hours.

# Actor input Schema

## `storeUrls` (type: `array`):

Homepage URLs of Shopify stores, e.g. https://www.allbirds.com. Any store built on Shopify works, including custom domains.

## `maxProductsPerStore` (type: `integer`):

Safety cap per store. The sitemap fallback means even 50,000+ product stores can be exported in full — raise this cap accordingly.

## `includeVariants` (type: `boolean`):

When enabled, each product includes every variant (SKU, price, compare-at price, availability). Turn off for a lighter dataset with counts only.

## `proxyConfiguration` (type: `object`):

Proxy to use. Apify Proxy (automatic) is recommended — some stores sit behind aggressive CDNs.

## Actor input object example

```json
{
  "storeUrls": [
    "https://www.allbirds.com"
  ],
  "maxProductsPerStore": 5000,
  "includeVariants": true,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "storeUrls": [
        "https://www.allbirds.com"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("sturdydata/shopify-products-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "storeUrls": ["https://www.allbirds.com"],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("sturdydata/shopify-products-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "storeUrls": [
    "https://www.allbirds.com"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call sturdydata/shopify-products-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=sturdydata/shopify-products-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Shopify Products Scraper — Full Catalog Export",
        "description": "Export the complete product catalog of any Shopify store: titles, handles, vendors, tags, prices, variants with SKUs and availability, images and timestamps. Uses the store's own JSON endpoints (fast, no browser) and automatically falls back to product sitemaps to get past the 5,000-product pagin...",
        "version": "0.1",
        "x-build-id": "Pga3OxFvwk12tlHD1"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/sturdydata~shopify-products-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-sturdydata-shopify-products-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/sturdydata~shopify-products-scraper/runs": {
            "post": {
                "operationId": "runs-sync-sturdydata-shopify-products-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/sturdydata~shopify-products-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-sturdydata-shopify-products-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "storeUrls"
                ],
                "properties": {
                    "storeUrls": {
                        "title": "Shopify store URLs",
                        "type": "array",
                        "description": "Homepage URLs of Shopify stores, e.g. https://www.allbirds.com. Any store built on Shopify works, including custom domains.",
                        "items": {
                            "type": "string"
                        }
                    },
                    "maxProductsPerStore": {
                        "title": "Max products per store",
                        "minimum": 1,
                        "maximum": 200000,
                        "type": "integer",
                        "description": "Safety cap per store. The sitemap fallback means even 50,000+ product stores can be exported in full — raise this cap accordingly.",
                        "default": 5000
                    },
                    "includeVariants": {
                        "title": "Include full variant list",
                        "type": "boolean",
                        "description": "When enabled, each product includes every variant (SKU, price, compare-at price, availability). Turn off for a lighter dataset with counts only.",
                        "default": true
                    },
                    "proxyConfiguration": {
                        "title": "Proxy configuration",
                        "type": "object",
                        "description": "Proxy to use. Apify Proxy (automatic) is recommended — some stores sit behind aggressive CDNs.",
                        "default": {
                            "useApifyProxy": true
                        }
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```