Shopify Products Scraper — Full Catalog Export

Export the complete product catalog of any Shopify store: titles, handles, vendors, product types, tags, prices, every variant with SKU / price / compare-at price / availability, image URLs and timestamps.

Built for: competitive pricing intelligence, assortment analysis, dropshipping research, MAP monitoring, catalog syncing.

Why this scraper

Generic Shopify actors all share the same structural flaw: they paginate /products.json and silently stop at the ~5,000-product cap. On a 30,000-SKU store you get a sixth of the catalog and no warning.

This actor:

• Uses the store's own JSON endpoints — fast, exact data, no browser, no HTML parsing errors.
• Detects the 5,000-product wall and automatically switches to product sitemaps (sitemap_products_*.xml) + per-product JSON, exporting the full catalog.
• Also handles stores that disable /products.json entirely (sitemap path kicks in).
• Checks each store's robots.txt first and skips stores that disallow these paths — compliance by construction.
• Per-store RUN_SUMMARY (key-value store) reports exactly how many products came from which path. No silent truncation, ever.

Input

Field	Type	Default	Notes
storeUrls	array	—	Store homepages (custom domains fine).
maxProductsPerStore	integer	5000	Raise it for big catalogs — the sitemap fallback can go far beyond 5,000.
includeVariants	boolean	true	Full variant list per product, or counts only.
proxyConfiguration	object	Apify Proxy	Recommended: automatic.

{
    "storeUrls": ["https://www.allbirds.com"],
    "maxProductsPerStore": 20000
}

Output schema

One dataset item per product:

{
    "store": "www.allbirds.com",
    "storeUrl": "https://www.allbirds.com",
    "id": "7205181653072",
    "handle": "mens-cruiser-terralux-anthracite",
    "title": "Men's Cruiser Terralux - Anthracite",
    "url": "https://www.allbirds.com/products/mens-cruiser-terralux-anthracite",
    "vendor": "Allbirds",
    "productType": "Shoes",
    "tags": ["mens", "cruiser"],
    "priceMin": 120,
    "priceMax": 120,
    "available": true,
    "variantsCount": 14,
    "variants": [
        { "id": "41448181662", "title": "8", "sku": "A10611M080", "price": 120, "compareAtPrice": null, "available": true }
    ],
    "images": ["https://cdn.shopify.com/s/files/..."],
    "imagesCount": 6,
    "publishedAt": "2026-03-02T10:00:00-05:00",
    "createdAt": "2026-02-28T09:12:00-05:00",
    "updatedAt": "2026-07-01T02:11:00-04:00",
    "source": "products_json",
    "scrapedAt": "2026-07-04T10:00:00.000Z"
}

Notes: prices are numbers in the store's own currency (currency code is not exposed by these endpoints — it is the store's display currency). source tells you which path produced the item (products_json or product_js sitemap fallback). Fields a store does not publish are null.

Pricing

Pay per result. 1 result = 1 product (with all its variants included).

Products	Cost at $2.50 / 1,000
1,000	$2.50
10,000	$25.00
50,000	$125.00

Compliance & fair use

Public storefront endpoints only, robots.txt checked per store before any scraping, 1 request/second per store. No login walls, no checkout/cart endpoints, no customer data — catalog facts only.

Using from API / MCP / AI agents

curl -X POST "https://api.apify.com/v2/acts/<ACTOR_ID>/run-sync-get-dataset-items?token=<TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{ "storeUrls": ["https://www.allbirds.com"], "maxProductsPerStore": 100 }'

Fully described input schema — works as an AI-agent tool via the Apify MCP server.

FAQ

How do I know if the site is a Shopify store? Check <domain>/products.json in your browser — if you see JSON, it works. If unsure, just run it: non-Shopify stores produce zero results and a clear note in RUN_SUMMARY.

Why fewer products than the store claims? Check the RUN_SUMMARY key-value store record (Storage → Key-value store in the run's console). Each store gets a note field that explains exactly what happened whenever results are zero or capped — e.g. robots.txt blocked scraping (robotsBlocked: true), the store isn't running standard Shopify, or its storefront doesn't expose the usual product endpoints. note is null when everything went as expected.

Do you return product descriptions? Not in v0.1 (keeps items small). If you need body_html, open an issue — it ships as an opt-in flag.

Changelog

• 0.1.1 (2026-07) — RUN_SUMMARY now includes a note field per store that explains why a run returned zero or fewer products than expected (bot-blocked, non-Shopify storefront, missing sitemap, etc.) — no more silent zero-result runs.
• 0.1.0 (2026-07) — Initial release: products.json export, automatic sitemap fallback past the 5K cap, robots.txt pre-check, per-store run summary, variant-level data.

---

Maintained by Sturdy Data — boring, reliable scrapers. Report issues in the Issues tab; we respond within 24 hours.