Shopify Store Products Scraper

Extract the full, public product catalog from any Shopify store - titles, prices, variants, images, inventory signals, and more - and track how that catalog changes over time, without writing a single line of scraping code.

The actor reads the storefront products.json endpoint that Shopify exposes on every store by default, paginates through the entire catalog, and normalizes the result into a clean, flat dataset that is ready for a spreadsheet, BI tool, or your own pipeline. Turn on change tracking and the same actor becomes a price/stock monitor: it remembers what it saw last time and reports exactly what changed.

Why this actor

• Monitoring built in, not left to you. Turn on change tracking and every run after the first reports new products, removed products, price drops and increases (with percent change), sales starting/ending, and stock changes - no need to download two datasets and diff them yourself.
• Insight rows, not just raw JSON. An optional per-store summary row gives you product/variant counts, % on sale, average discount, price min/median/max, in-stock %, top vendors and product types, and newest products in one row - a one-screen read of a store's catalog health.
• One simple price. A single pay-per-result event, no stacked add-on fees for extra fields - change rows and store summaries are priced the same as a product row.
• Collection URLs actually scrape just that collection. Point it at /collections/sale and only that collection's products are fetched - full catalog URLs aren't silently substituted.
• Automatic fallback when a store blocks the main feed. Some stores return 401/403/429 for /products.json specifically from Apify's datacenter IPs. The actor retries with a fresh browser fingerprint and (if a proxy is configured) a new proxy session, then tries /collections/all/products.json, then falls back to discovering products via the store's /sitemap.xml and fetching each product's AJAX JSON - all before giving up. Every row records which method produced it (source).
• Built-in self-check. Every run writes a RUN_REPORT record (in the run's key-value store) with inputs processed, row counts by type, errors by code, warnings (e.g. a store returning 0 products), and a validation pass confirming required fields were non-empty in every output row.
• Partial results, not all-or-nothing. If a store fails midway through a large catalog, everything already fetched is kept and marked partial in the error row - no need to rerun to get the pages it already managed to fetch.
• No surprise catalog truncation. There is no low default cap hiding most of a store's catalog - leave Max products per store at 0 and you get everything.

What it does

Give it one or more store URLs and it will:

• Resolve the store (accepts a full URL or a specific collection URL).
• Follow redirects if the store has moved to a different domain.
• Page through the entire product catalog (or a capped number of products).
• Fall back to an alternative pagination method automatically if a store disables the default one partway through.
• Fall back further - fresh browser fingerprint/proxy session, an alternative endpoint, then sitemap + per-product AJAX JSON - if the main feed is blocked outright or returns HTML instead of JSON. See "Fallback chain" below.
• Pull the store's active currency.
• Detect and clearly report stores that are not running Shopify, or that have disabled/blocked the public products feed, instead of silently failing.
• Output one row per product (with a nested variants array) or one row per variant, depending on how you configure it.
• Optionally compare this run's catalog to the last run's and output rows describing exactly what changed.
• Optionally output one aggregate "store summary" row per store.
• Write a RUN_REPORT self-check record at the end of every run.

Use cases

• Price & stock monitoring - enable Track changes between runs and schedule the actor; each run reports price drops/increases, sales starting/ending, restocks and stockouts, and new/removed products.
• Competitor research - see what products, vendors, and tags a store carries, plus a one-row summary of its assortment and pricing.
• Catalog export / migration - pull a full product list out of a Shopify store you manage or are migrating away from.
• Market research - analyze product assortment, naming, and pricing patterns across many stores in one run.

How to use

1. Add one or more full store URLs to Store URLs, e.g.:

   • https://www.allbirds.com
   • https://www.allbirds.com/collections/mens-shoes (scrapes only that collection)

   Use full URLs here (https://...). If you're calling the Apify API directly rather than using the Console UI and only have bare domains (e.g. allbirds.com), put those in the separate Store domains field instead - the list editor used by Store URLs rejects bare domains when submitted via the API. Both fields are merged into one list of stores to scrape.

2. (Optional) Set Max products per store to cap how many products are fetched per store. Leave at 0 to fetch the entire catalog.

3. (Optional) Enable Output one row per variant if you want a row for every size/color/variant combination instead of one row per product.

4. (Optional) Enable Track changes between runs to turn this into a monitor - see "Change tracking" below.

5. (Optional) Turn on a proxy under Proxy configuration if you are scraping many stores or very large catalogs.

6. Run the actor.

Example input

{
  "startUrls": [
    { "url": "https://www.allbirds.com" },
    { "url": "https://www.gymshark.com" }
  ],
  "storeDomains": ["colourpop.com"],
  "maxProductsPerStore": 0,
  "includeVariants": false,
  "proxyConfiguration": { "useApifyProxy": false }
}

Example input with monitoring

{
  "startUrls": [{ "url": "https://www.allbirds.com" }],
  "trackChanges": true,
  "monitorName": "allbirds-watch",
  "outputMode": "changesOnly",
  "includeStoreSummary": true
}

Run this on a schedule. The first run saves a baseline snapshot and reports no changes; every run after that reports what moved since the previous run.

Output example

Default mode (one row per product):

{
  "type": "product",
  "storeUrl": "https://www.allbirds.com",
  "storeDomain": "www.allbirds.com",
  "productId": 1234567890,
  "title": "Men's Wool Runner",
  "handle": "mens-wool-runner",
  "url": "https://www.allbirds.com/products/mens-wool-runner",
  "vendor": "Allbirds",
  "productType": "Shoes",
  "tags": ["men", "running", "wool"],
  "descriptionText": "A comfortable, breathable everyday sneaker...",
  "price": 98,
  "compareAtPrice": null,
  "currency": "USD",
  "available": true,
  "images": ["https://cdn.shopify.com/.../image1.jpg"],
  "featuredImage": "https://cdn.shopify.com/.../image1.jpg",
  "createdAt": "2021-01-01T00:00:00-08:00",
  "updatedAt": "2024-05-01T00:00:00-07:00",
  "publishedAt": "2021-01-02T00:00:00-08:00",
  "options": [{ "name": "Size", "values": ["7", "8", "9", "10"] }],
  "variants": [
    {
      "id": 9876543210,
      "title": "9",
      "sku": "WR-M-9",
      "price": 98,
      "compareAtPrice": null,
      "available": true,
      "inventoryQuantity": 12,
      "option1": "9",
      "option2": null,
      "option3": null,
      "grams": 300,
      "requiresShipping": true,
      "barcode": null,
      "featuredImage": null
    }
  ],
  "minPrice": 98,
  "maxPrice": 98,
  "anyAvailable": true,
  "variantCount": 8,
  "onSale": false,
  "discountPercent": null,
  "inStockVariantCount": 8,
  "totalVariantCount": 8,
  "source": "products_json",
  "scrapedAt": "2026-01-01T12:00:00.000Z"
}

source records which method produced the row: products_json (the default feed), collections_all, or sitemap_ajax (see "Fallback chain" below).

With Output one row per variant enabled, each row has the same product fields plus variantId, variantTitle, sku, price, compareAtPrice, available, inventoryQuantity, option1/option2/option3, variantImage, onSale (true when compareAtPrice is higher than price), discountPercent, and totalVariantCount/inStockVariantCount for the parent product.

If a store can't be scraped, you'll get a clear error row instead of a silent gap in your data:

{
  "type": "error",
  "storeUrl": "https://www.wikipedia.org",
  "error": "NOT_SHOPIFY",
  "message": "...",
  "partial": false,
  "scrapedAt": "2026-01-01T12:00:00.000Z"
}

partial: true means the store failed partway through, but every product page fetched before the failure is already in the dataset.

Fallback chain

A small number of stores return 401/403/429 for /products.json specifically from Apify's datacenter IPs (both direct requests and the Apify datacenter proxy group), even though the same request succeeds from other IPs. When the main feed is blocked (401/403, or 429 that persists after retries) or returns an HTML page instead of JSON, the actor works through the following chain before giving up on a store, and stops at the first method that returns real data:

1. Retry /products.json (or the collection-scoped equivalent) up to 3 times, each attempt using a freshly generated browser fingerprint and, if a proxy is configured, a brand-new proxy session.
2. /collections/all/products.json as an alternative endpoint.
3. Sitemap fallback - read /sitemap.xml, follow its sitemap_products_*.xml entries to collect product URLs, then fetch each product's /products/<handle>.js AJAX JSON directly (at low concurrency, respecting Max products per store). Prices on this endpoint are in cents; the actor normalizes them to the same decimal units and field names used by /products.json before they reach the dataset.

Steps 2 and 3 only run for whole-store crawls (not for a URL scoped to one specific collection), since neither can be filtered to a single collection. Each dataset row records which method produced it in its source field. If every method fails, the store gets an error row with error: "BLOCKED" advising you to enable Proxy configuration with the RESIDENTIAL proxy group for that store - datacenter IPs (Apify's own or Apify Proxy's datacenter group) are what tends to get blocked.

Change tracking (monitoring mode)

Enable Track changes between runs (trackChanges: true) to turn repeated runs into a monitor. The actor keeps a compact snapshot per store (per product/variant: price, compare-at price, availability, title) in a named key-value store (shopify-monitor-<monitorName>), and compares the new snapshot to the one saved last time. Every difference becomes a change row:

{
  "type": "change",
  "changeType": "PRICE_DROP",
  "storeUrl": "https://www.allbirds.com",
  "storeDomain": "www.allbirds.com",
  "productId": "1234567890",
  "productTitle": "Men's Wool Runner",
  "url": "https://www.allbirds.com/products/mens-wool-runner",
  "variantId": "9876543210",
  "variantTitle": "9",
  "sku": "WR-M-9",
  "field": "price",
  "oldValue": 108,
  "newValue": 98,
  "percentChange": -9.3,
  "currency": "USD",
  "scrapedAt": "2026-01-01T12:00:00.000Z"
}

changeType is one of: NEW_PRODUCT, REMOVED_PRODUCT, PRICE_DROP, PRICE_INCREASE, SALE_STARTED, SALE_ENDED, OUT_OF_STOCK, BACK_IN_STOCK. Change rows are charged the same as a product row.

• Monitor name: give runs that should be compared against each other the same monitorName (e.g. a scheduled task). Leave it empty and the actor derives one from your store URLs automatically.
• First run: there is nothing to compare against yet, so it only saves a baseline and reports zero changes - this is expected.
• REMOVED_PRODUCT and partial crawls: if Max products per store is set above 0, the actor can't tell "removed" apart from "not reached this time", so removed-product detection is skipped and a warning is written to the run report instead of a false positive.
• outputMode: set to "changesOnly" to suppress regular product/variant rows and keep only change rows (plus the store summary and error rows, which are controlled separately).

Store summary rows

With Include per-store summary row enabled (default), each store adds one storeSummary row:

{
  "type": "storeSummary",
  "storeUrl": "https://www.allbirds.com",
  "storeDomain": "www.allbirds.com",
  "currency": "USD",
  "productCount": 412,
  "variantCount": 1830,
  "onSaleProductCount": 37,
  "onSalePercent": 9,
  "avgDiscountPercent": 22.4,
  "priceMin": 20,
  "priceMedian": 98,
  "priceMax": 250,
  "inStockProductCount": 390,
  "inStockPercent": 94.7,
  "topVendors": [{ "name": "Allbirds", "count": 412 }],
  "topProductTypes": [{ "name": "Shoes", "count": 300 }],
  "newestProducts": [{ "title": "...", "url": "...", "createdAt": "..." }],
  "collectionCount": 48,
  "sampleCollections": [{ "title": "Men's Shoes", "handle": "mens-shoes" }],
  "scrapedAt": "2026-01-01T12:00:00.000Z"
}

collectionCount/sampleCollections come from one extra cheap request to the store's collections feed (no per-collection product counts, which would need one request per collection). This row is charged as a single result.

Run report (self-check)

At the end of every run, the actor writes a RUN_REPORT record to the default key-value store with:

• inputsProcessed: how many store URLs were given.
• rows: counts of products, variants, changes, store summaries, and errors actually written to the dataset.
• errorsByCode: how many stores failed with each error code.
• warnings: notable non-fatal conditions, e.g. a store returning 0 products, or removed-product detection being skipped for a capped crawl.
• validation: whether every pushed row had its required fields (title, URL, store domain, etc.) non-empty.
• chargedResultCount: how many billable results this run produced.

Tips

• Collection URLs: point Store URLs at a specific collection (e.g. /collections/sale) to scrape just that subset of a catalog instead of the entire store.
• Variant mode: enable it when you need SKU-level or size/color-level pricing and inventory, such as for feed generation or inventory tracking.
• Monitoring: keep Monitor name fixed across scheduled runs if you plan to add or remove store URLs over time, so tracking stays continuous per store regardless of which other stores are in the same run.
• Proxies: for very large stores or when scraping many stores in one run, enable a proxy to reduce the chance of rate limiting or temporary blocks. If a store ends up BLOCKED even with a datacenter proxy, switch to the RESIDENTIAL proxy group - some stores specifically block datacenter IPs.
• Bare domains via the API: if you're calling the Apify API directly and only have bare domains (no https://), put them in Store domains instead of Store URLs - that field's editor rejects bare domains when used that way.

Pricing

This actor uses pay-per-result pricing: $1.00 per 1,000 results, with no start fee. A result is one product row, one variant row (when Output one row per variant is enabled), one change row, or one storeSummary row. Error rows for stores that can't be scraped are never charged. You can cap spending with the run's maximum cost setting, and the actor stops as soon as that limit is reached.

FAQ

Is this legal? The actor only reads data that a Shopify store already publishes publicly on its own storefront (the same products.json feed used by the store's own theme). It does not access private data, log in, or bypass any access controls.

What happens if a URL isn't a Shopify store? You'll get an error row with error: "NOT_SHOPIFY" for that store instead of a failed run - all other stores in the same run are unaffected.

What other errors can show up? PRODUCTS_JSON_DISABLED (the store is on Shopify but has turned off or restricted the public products feed), BLOCKED (the store rejected every method in the fallback chain - see "Fallback chain" above; try enabling Proxy configuration with the RESIDENTIAL group), and FETCH_FAILED (a network-level failure, such as a DNS error or timeout).

Why did I get no change rows on my first monitored run? The first run with a given monitor name has nothing to compare against - it saves the baseline. Run it again later and changes will appear.

What are the limits? Only products that are published to the Online Store sales channel are visible through this feed - unpublished, draft, or password-protected-storefront products won't appear. Inventory quantities are only included when a store chooses to expose them publicly; otherwise that field will be null.