Shopify Products Scraper

Scrape products and variants from any public Shopify store using its built-in /products.json endpoint. No login, no Shopify Admin API, no browser.

Unlike scrapers that finish "successfully" with 0 results and no explanation, this Actor checks every store first and, if it cannot be scraped, tells you why in the log and in the SUMMARY record (not a Shopify store, password-protected, blocked by bot protection, rate limited, empty catalog, invalid URL…).

Input

Field	Type	Default	Description
shopUrls	string[]	— (required)	Store URLs, e.g. https://www.allbirds.com. URLs ending in /products.json are accepted; only the origin is used.
maxProductsPerShop	integer	100	Max products per shop (each product may yield several variant rows). 0 = as many as the store exposes.
includeVariants	boolean	true	true: one row per variant. false: one row per product, using the first variant's price.
maxShops	integer	5	Max shops per run (hard cap 20). Extra URLs are skipped and listed in SUMMARY.
requestDelayMs	integer	500	Delay between page requests to the same shop.
userAgent	string	browser UA	Optional custom User-Agent.

{
    "shopUrls": ["https://www.allbirds.com"],
    "maxProductsPerShop": 50,
    "includeVariants": true
}

Output

One dataset item per variant (or per product when includeVariants is false):

{
    "shopUrl": "https://www.allbirds.com",
    "shopDomain": "allbirds.com",
    "productId": "7340901859408",
    "variantId": "41980124004432",
    "variantTitle": "8",
    "title": "Women's Allbirds Flip Flop - Dusty Pink",
    "handle": "womens-allbirds-flip-flop-dusty-pink",
    "vendor": "Allbirds",
    "productType": "Shoes",
    "tags": ["womens", "..."],
    "price": 50,
    "compareAtPrice": null,
    "available": true,
    "sku": "AB0123W080",
    "barcode": null,
    "imageUrl": "https://cdn.shopify.com/...",
    "productUrl": "https://www.allbirds.com/products/womens-allbirds-flip-flop-dusty-pink?variant=41980124004432",
    "variantCount": 6,
    "createdAt": "2025-03-01T10:00:00-08:00",
    "updatedAt": "2026-10-01T10:00:00-07:00",
    "scrapedAt": "2026-10-29T12:00:00.000Z"
}

price / compareAtPrice are numbers in the store's currency (or null). tags is always an array. Rows are de-duplicated by shopDomain + productId + variantId.

SUMMARY

The default key-value store record SUMMARY contains:

{
    "shops": [{ "shopUrl": "...", "ok": true, "products": 50, "rows": 538, "pages": 1, "stopReason": "reached maxProductsPerShop=50", "elapsedMs": 3200 }],
    "products": 50,
    "rows": 538,
    "errors": [{ "shopUrl": "https://example.com", "reason": "http-404: not a Shopify store (no /products.json)" }],
    "skippedReasons": { "https://example.com": "http-404: not a Shopify store (no /products.json)" }
}

How it works

1. Probe https://<shop>/products.json?limit=1. 404 / non-JSON / password redirect / 401 / 403 / challenge page → the shop is skipped with a reason.
2. Paginate /products.json?limit=<n>&page=N until a page is short or empty, the product cap is reached, or a safety limit of 25 pages.
3. Retry 429 and 5xx with exponential backoff (max 3 retries, honours Retry-After).

Pricing

Pay per event: product-scraped is charged once per dataset row pushed. Skipped shops and empty runs cost nothing in events.

Limitations

• Only data exposed by the public /products.json endpoint is available — no Admin API, no orders, no customer data.
• Password-protected stores and non-Shopify sites are skipped (with a reason), never bypassed.
• No exact inventory quantities: only the available flag per variant is exposed publicly.
• Some stores sit behind Cloudflare or other bot protection and may block or rate-limit requests; such stores are reported as skipped.
• Some stores cap pagination (often around page 17–20 of 250 products) or disable /products.json entirely, so very large catalogs may be incomplete.
• Prices are in the store's default currency; multi-currency / market-specific prices are not resolved.