# Shopify Store Products & Prices Scraper (`hridayrungta/shopify-products-scraper`) Actor

Extract the public product catalogue of any Shopify store: every product and variant with price, compare-at price, SKU, availability, vendor, type, tags, images and dates. Monitor many stores and emit only price changes. One row per variant, $0.30 per 1,000.

- **URL**: https://apify.com/hridayrungta/shopify-products-scraper.md
- **Developed by:** [Hriday Rungta](https://apify.com/hridayrungta) (community)
- **Categories:** E-commerce
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$0.30 / 1,000 variant scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Shopify Store Products & Prices Scraper

Turn any Shopify storefront into a clean, structured product feed. Give the Actor a list of store URLs and it returns **every product and variant** with price, compare-at (strike-through) price, SKU, availability, vendor, type, tags, images and dates - one row per variant, ready for spreadsheets, BI tools or your own code.

Point it at ten competitor stores on a daily schedule with **Price changes only** switched on and it becomes a price monitor: each run emits only the variants whose price changed since yesterday, with the old and new value side by side.

It reads the public JSON product feed that every Shopify online store exposes (`/products.json`), so there is no browser, no HTML parsing and no login. Stores that switched the feed off are handled by a sitemap fallback.

> **Independent tool.** Uses the public product feed each Shopify store exposes to shoppers and search engines. Not affiliated with, endorsed by or supported by Shopify Inc. Product data belongs to the individual merchants; you are responsible for respecting each store's terms of service and applicable law.

### What it extracts

One row per **variant** (or per product with `includeVariants: false`):

| Field | Description |
|---|---|
| `storeDomain` | Storefront hostname |
| `productId`, `productHandle`, `productTitle` | Shopify product ID, URL slug, title |
| `vendor`, `productType`, `tags[]` | Brand, merchant-defined type, tags |
| `variantId`, `variantTitle`, `variantOptions` | Variant ID, e.g. `M / Black`, and `{ Size: "M", Color: "Black" }` |
| `sku`, `barcode` | SKU; barcode only where the fallback feed provides it |
| `price`, `compareAtPrice`, `currency` | Current price, original price when on sale, ISO currency (from the store's `/meta.json` or `/cart.js`; `null` if neither is exposed) |
| `available` | Purchasable right now (`true`/`false`) |
| `inventoryQuantity`, `requiresShipping`, `grams` | Stock count is usually `null` - Shopify's public feed does not publish it |
| `productUrl`, `variantUrl`, `imageUrl` | Product page, page pre-selected to the variant, main image |
| `images[]` | All product images (`includeImages`) |
| `bodyHtml`, `description` | Raw HTML and plain-text description (`includeBodyHtml`) |
| `createdAt`, `updatedAt`, `publishedAt` | ISO-8601 timestamps from the store |
| `changeType`, `priceChange {old,new}`, `compareAtPriceChange {old,new}` | Only in `newOnly` / `priceChangesOnly` mode |
| `scrapedAt` | ISO-8601 |

No personal data is collected: storefront catalogues contain products, not people.

### Use cases

- **Price monitoring across competitor stores** - schedule daily runs in *Price changes only* mode and get a feed of every price drop, markup, sale start and sale end across dozens of Shopify brands, with old and new prices.
- **DTC market research** - map a niche: which vendors, product types and price bands do the leading Shopify brands sell; how big are their catalogues; how often do they publish new products (`publishedAt`).
- **Dropshipping and product research** - pull full catalogues from stores in your niche to compare assortments, spot best-seller tags and find gaps.
- **Retail analytics and repricing** - join competitor variants on SKU or barcode with your own catalogue and feed a repricing engine.
- **Agencies and consultants** - audit a prospect's catalogue (missing images, `compareAtPrice` errors, unavailable variants, tag hygiene) before a pitch, or track a client's competitors as a managed service.

### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `storeUrls` | array | - | Store domains or URLs, one per line. Required. |
| `collectionHandles` | array | all | Only these collections (handles or `/collections/x` URLs), applied to every store. |
| `outputMode` | `all` / `newOnly` / `priceChangesOnly` | `all` | See *Daily price-change recipe*. |
| `maxProductsPerStore` | integer | `5000` | Cap per store (Shopify's public feed ends at 25,000). |
| `maxItems` | integer | `10000` | Hard cap on saved rows across all stores = cost cap. |
| `includeVariants` | boolean | `true` | One row per variant; off = one row per product (cheapest variant's price, plus `variantCount`). |
| `includeImages` | boolean | `false` | Add `images[]`. |
| `includeBodyHtml` | boolean | `false` | Add `bodyHtml` and `description`. |
| `emitAllOnFirstRun` | boolean | `false` | In change modes, output the whole catalogue on a store's first run as well. |
| `snapshotStoreName` | string | `shopify-products-snapshots` | Named key-value store holding last-seen prices per store. |
| `requestsPerSecond` | integer | `2` | Per-store throttle (1-5). |
| `storeConcurrency` | integer | `3` | Stores scraped in parallel. |
| `maxRetries` | integer | `4` | Backoff retries for network errors, 5xx, 403, 429, 430. |
| `allowHtmlFallback` | boolean | `true` | Sitemap / `/collections/all` fallback for stores that disabled the JSON feed. |
| `proxyConfiguration` | object | none | Optional; only needed if a store rate-limits you. |

Example - full catalogues of three stores:

```json
{
  "storeUrls": ["https://www.allbirds.com", "gymshark.com", "https://colourpop.com/collections/lips"],
  "maxProductsPerStore": 1000,
  "maxItems": 20000
}
```

### Sample output

```json
{
  "storeDomain": "example-socks.com",
  "productId": "7654321001",
  "productHandle": "merino-crew-sock",
  "productTitle": "Merino Crew Sock",
  "vendor": "Example Socks Co",
  "productType": "Socks",
  "tags": ["merino", "crew", "bestseller"],
  "variantId": "43000000001",
  "variantTitle": "S / Black",
  "variantOptions": { "Size": "S", "Color": "Black" },
  "sku": "MCS-S-BLK",
  "barcode": null,
  "price": 18,
  "compareAtPrice": 24,
  "currency": "USD",
  "available": true,
  "inventoryQuantity": null,
  "requiresShipping": true,
  "grams": 85,
  "productUrl": "https://example-socks.com/products/merino-crew-sock",
  "variantUrl": "https://example-socks.com/products/merino-crew-sock?variant=43000000001",
  "imageUrl": "https://cdn.shopify.com/s/files/1/0001/0001/products/mcs-black.jpg?v=1740000000",
  "createdAt": "2025-03-01T15:00:00.000Z",
  "updatedAt": "2026-09-10T22:22:41.000Z",
  "publishedAt": "2025-03-02T14:15:00.000Z",
  "scrapedAt": "2026-09-11T06:00:02.114Z"
}
```

In `priceChangesOnly` mode each row additionally carries:

```json
{ "changeType": "price-change", "priceChange": { "old": 20, "new": 18 }, "compareAtPriceChange": { "old": 24, "new": 24 } }
```

Export as JSON, CSV, Excel or XML from the Apify Console or API, or push rows to Google Sheets, Slack, e-mail or a webhook with an integration. The dataset has two built-in views: **Variants** and **Price changes**.

### Daily price-change recipe

1. Create a run with your competitor list in `storeUrls`, `outputMode: "priceChangesOnly"`, `maxItems` comfortably above your total variant count.
2. Run it once. The first run saves a **baseline** of every variant's price into the key-value store `shopify-products-snapshots` (one record per store) and emits nothing - or the whole catalogue if `emitAllOnFirstRun` is on.
3. Save it as a **Schedule** (e.g. daily at 06:00). From now on every run outputs only:
   - variants whose `price` or `compareAtPrice` changed (`changeType: "price-change"`, `priceChange: { old, new }`), and
   - variants that did not exist before (`changeType: "new"`, `priceChange: { old: null, new }`).
4. Add a **Slack / e-mail / Google Sheets integration** to the schedule to deliver the changes. Unchanged variants cost nothing: you are billed only for rows that are actually saved.

Use `newOnly` instead to get a feed of new product launches. Use a different `snapshotStoreName` per watch-list to keep several independent monitors. Variants that vanish from a store are kept in the snapshot, so a temporarily hidden product does not re-trigger as "new" when it returns.

### Pricing

Pay-per-event: one `variant-scraped` event per saved row at **$0.0003 ($0.30 per 1,000 rows)**, no start fee. A 2,000-variant catalogue costs $0.60; a daily price-change run across 20 stores that finds 150 changes costs under 5 cents. `maxItems` is your hard cost cap and the run also stops when your Apify "max total charge" limit is reached. Unchanged variants, duplicates and skipped stores are never billed.

### FAQ

**Which stores work?**
Any Shopify online store whose `/products.json` feed is enabled (the default; the vast majority of stores). Headless storefronts (Hydrogen and other custom front ends) and password-protected stores expose no public feed and are reported as `feed-disabled` / `password-protected` in the run's `STORE-SUMMARIES` record. Non-Shopify sites are detected via Shopify's response headers and skipped as `not-shopify`.

**What if a store disabled `/products.json`?**
The Actor reads product handles from the store's sitemap (or `/collections/all` pages) and fetches each product's `/products/{handle}.js` feed instead. That costs one request per product, so it is slower; prices and availability are identical, `updatedAt` is not available in that feed.

**Why is `currency` null for some stores?**
It comes from the store's `/meta.json` (or `/cart.js`). A few stores block those endpoints. Prices are always in the store's default presentment currency.

**Can I get stock levels?**
Usually not. Shopify's public feed exposes `available` (in stock or not) but not quantities; `inventoryQuantity` is filled only when a store happens to publish it.

**How complete is the catalogue?**
`/products.json` returns published products visible to the online store sales channel, 250 per page, up to page 100 (25,000 products). Draft, archived and channel-hidden products are not included. Use `collectionHandles` to target subsets of very large stores.

**Is this legal?**
The Actor reads the same public endpoints a browser, Google and price-comparison engines use, without login or circumvention, at a polite rate (2 requests per second per store by default). It collects no personal data. Shopify's own terms bind merchants, not visitors, but individual stores may have their own terms of use: you are responsible for respecting them and for how you use the data. This tool is not affiliated with Shopify Inc.

**What about rate limits?**
Shopify answers 429 or 430 to aggressive clients. The Actor throttles per store, honours `Retry-After`, backs off exponentially and rotates proxy sessions if a proxy is configured. Keep `requestsPerSecond` at 2 unless you know a store tolerates more.

**Does it get metafields, reviews or collection membership?**
No. Those are not in the public product feed. Collection membership can be derived by running with `collectionHandles`.

### Local development

```bash
npm install
npm test                  # offline unit tests against documented fixtures
APIFY_LOCAL_STORAGE_DIR=$PWD/storage node src/main.js
```

# Actor input Schema

## `storeUrls` (type: `array`):

Shopify storefronts to scrape, one per line: a domain (`example-store.com`), a homepage URL or a `*.myshopify.com` address. Any path or query string is ignored. Non-Shopify sites are detected and skipped.

## `collectionHandles` (type: `array`):

Only scrape these collections, e.g. `sale`, `new-arrivals` or a full `/collections/mens` URL. Applies to every store. Leave empty to scrape the whole catalogue via `/products.json`.

## `outputMode` (type: `string`):

`all`: every variant every run. `newOnly`: only variants not seen in an earlier run. `priceChangesOnly`: new variants plus variants whose price or compare-at price changed since the last run (rows carry `priceChange {old,new}`). Change modes keep one snapshot per store in the key-value store named below; the first run saves a baseline and emits nothing unless **Emit everything on first run** is on.

## `maxProductsPerStore` (type: `integer`):

Stop paging a store after this many products (each product can have many variants). Shopify's public feed itself stops at page 100 = 25,000 products.

## `maxItems` (type: `integer`):

Hard cap on saved rows across all stores. This is also your cost cap: you are billed once per saved row.

## `includeVariants` (type: `boolean`):

On (default): one row per variant with `variantId`, `variantTitle`, `sku`, `price`. Off: one row per product using its cheapest variant's price, plus `variantCount`.

## `includeImages` (type: `boolean`):

Add `images[]` with every product image URL. `imageUrl` (the variant's or product's main image) is always included.

## `includeBodyHtml` (type: `boolean`):

Add `bodyHtml` (raw HTML) and `description` (plain text). Off by default to keep rows small.

## `emitAllOnFirstRun` (type: `boolean`):

In `newOnly` / `priceChangesOnly` mode, also output the full catalogue when a store has no snapshot yet (every row gets `changeType: "new"`). Off: the first run only saves the baseline.

## `snapshotStoreName` (type: `string`):

Name of the named key-value store that holds the last-seen prices per store domain (created if missing). Use different names to keep independent watch-lists.

## `requestsPerSecond` (type: `integer`):

Throttle per storefront (1-5). Shopify answers 429/430 to aggressive clients; 2 is safe for virtually every store.

## `storeConcurrency` (type: `integer`):

How many stores to scrape at the same time (each with its own throttle).

## `maxRetries` (type: `integer`):

Retries with exponential backoff (honouring Retry-After) for network errors, 5xx, 403, 429 and 430 responses.

## `allowHtmlFallback` (type: `boolean`):

When a store has disabled `/products.json`, read product handles from its sitemap and, failing that, from `/collections/all` pages, then fetch each product's `.js` feed. Costs one request per product.

## `proxyConfiguration` (type: `object`):

Optional. The product feed is public and works without a proxy for most stores; enable Apify Proxy if a store answers 429/430 or 403.

## Actor input object example

```json
{
  "storeUrls": [
    "https://www.allbirds.com",
    "https://www.gymshark.com"
  ],
  "collectionHandles": [],
  "outputMode": "all",
  "maxProductsPerStore": 200,
  "maxItems": 500,
  "includeVariants": true,
  "includeImages": false,
  "includeBodyHtml": false,
  "emitAllOnFirstRun": false,
  "snapshotStoreName": "shopify-products-snapshots",
  "requestsPerSecond": 2,
  "storeConcurrency": 3,
  "maxRetries": 4,
  "allowHtmlFallback": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "storeUrls": [
        "https://www.allbirds.com",
        "https://www.gymshark.com"
    ],
    "collectionHandles": [],
    "maxProductsPerStore": 200,
    "maxItems": 500
};

// Run the Actor and wait for it to finish
const run = await client.actor("hridayrungta/shopify-products-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "storeUrls": [
        "https://www.allbirds.com",
        "https://www.gymshark.com",
    ],
    "collectionHandles": [],
    "maxProductsPerStore": 200,
    "maxItems": 500,
}

# Run the Actor and wait for it to finish
run = client.actor("hridayrungta/shopify-products-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "storeUrls": [
    "https://www.allbirds.com",
    "https://www.gymshark.com"
  ],
  "collectionHandles": [],
  "maxProductsPerStore": 200,
  "maxItems": 500
}' |
apify call hridayrungta/shopify-products-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,hridayrungta/shopify-products-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/0g1eysRNcmeHfrm8v/builds/HIKd6PmggyCu2O04j/openapi.json