# Shopify Products.json Scraper — Price & Stock Monitor (`mayevil/shopify-catalog-monitor`) Actor

Scrape public Shopify /products.json catalogs (optional collections). One dataset row per variant. Monitor price_up/down and stock_in/out between runs. No login, no browser. Soft-fails blocked (403) stores.

- **URL**: https://apify.com/mayevil/shopify-catalog-monitor.md
- **Developed by:** [David Tu](https://apify.com/mayevil) (community)
- **Categories:** E-commerce, Automation, Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.70 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

Scrape **public Shopify store catalogs** via `/products.json` and optionally detect **price / stock / new-product changes** between runs. **One dataset row per variant.** Optional collection handles. **No login, no browser, no residential proxies.** Soft-fails stores that return HTTP 403.

Also see the [Greenhouse / Lever / Ashby Jobs Scraper + Monitor](https://apify.com/mayevil/greenhouse-lever-jobs-scraper) and [Workday Jobs Scraper + Monitor](https://apify.com/mayevil/workday-jobs-monitor) for hiring-signal monitoring.

### What does this Shopify products.json scraper do?

- 🛒 **Multi-store catalog export.** Page through `/products.json?limit=250` (or selected collections).
- 💲 **One row per variant.** Price, compare-at, SKU, availability, image, product URL.
- 📡 **Monitor mode.** Remembers variant fingerprints and emits only `new` / `price_up` / `price_down` / `stock_in` / `stock_out` / `removed`.
- 🎯 **Filters.** Title, vendor, product type, tags, in-stock only, caps.
- 🛡️ **Polite.** Per-host throttling, retries on 429/5xx, per-shop `RUN_SUMMARY` (`ok` / `blocked` / `not_found` / `error` / `skipped`).

### Who uses it?

#### Ecommerce analysts and brand teams

Export competitor catalogs and track **price_up / price_down** and **stock_in / stock_out** on a schedule without building your own crawler.

#### Price intelligence and MAP monitoring

Watch a watchlist of Shopify stores; pay only for **change rows** in monitor mode.

#### Product data / aggregators

Ingest structured variant-level rows (SKU, price, availability, image URL) from public `/products.json`.

### Input

| Format | Example |
|---|---|
| Domain | `allbirds.com` |
| With www | `www.allbirds.com` |
| Full URL | `https://www.allbirds.com` |

Path and query are ignored; the Actor always hits the shop origin.

#### Example: sample export

```json
{
  "shops": ["allbirds.com"],
  "maxProductsPerShop": 10,
  "maxProductsTotal": 50
}
```

#### Example: daily price & stock monitor

```json
{
  "shops": ["allbirds.com"],
  "monitorMode": true,
  "monitorStateKey": "footwear",
  "outputPriceChanges": true,
  "outputStockChanges": true,
  "outputRemoved": true,
  "skipExistingOnFirstRun": true
}
```

### Output example

```json
{
  "productKey": "shopify:www.allbirds.com:7340901859408:42146889203792",
  "status": "active",
  "shop": "www.allbirds.com",
  "shopName": "Allbirds",
  "productId": "7340901859408",
  "handle": "womens-allbirds-flip-flop-dusty-pink",
  "title": "Women's Allbirds Flip Flop - Dusty Pink",
  "vendor": "Allbirds",
  "productType": "Shoes",
  "tags": ["shoprunner"],
  "variantId": "42146889203792",
  "sku": "A12513W100",
  "variantTitle": "10",
  "price": 25,
  "compareAtPrice": 50,
  "currency": "USD",
  "available": true,
  "grams": 594,
  "imageUrl": "https://cdn.shopify.com/...",
  "productUrl": "https://www.allbirds.com/products/womens-allbirds-flip-flop-dusty-pink",
  "collectionHandle": null,
  "sourceApiUrl": "https://www.allbirds.com/products.json?limit=250&page=1",
  "scrapedAt": "2026-10-09T08:00:00.000Z"
}
```

A **`RUN_SUMMARY`** record in the run key-value store lists every shop with status and counts.

### Monitor mode

- State lives in key-value store `shopify-catalog-monitor-state` under your **Monitor state key**.
- Fingerprint per variant: `{ price, available, title }`.
- First run: all matching variants as `new`, or baseline-only if **Skip existing on first run** is on.
- Later runs: only changes (and optionally `removed`).
- Variants not output because of max/budget caps are **not** marked seen / updated, so the next run delivers them.

### Pricing

**$1.00 per 1,000 variant rows**, pay per result (pay-per-event). You pay only for rows saved to your dataset, plus a tiny Apify start fee per run ($0.00005). There is no monthly rental, and **in monitor mode you pay only for change rows.**

| Apify plan discount | Price per 1,000 variant rows |
|---|---|
| Free / Starter | $1.00 |
| Bronze | $0.90 |
| Silver | $0.80 |
| Gold and above | $0.70 |

Examples at $1.00 / 1,000:

- Sample of 10 products × ~5 variants ≈ 50 rows ≈ **$0.05**.
- A daily monitor that emits ~200 change rows/day ≈ 6,000 rows/month ≈ **$6/month**.

Set **Max products per shop** / **Max variant rows in total** or the run's **maximum cost** to cap spending.

### FAQ

#### What is `/products.json`?

Most public Shopify storefronts expose a paginated JSON catalog at `https://{shop}/products.json`. This Actor reads that feed (and optional `/collections/{handle}/products.json`) — no theme HTML scraping.

#### Why one row per variant?

Shopify products often have many sizes/colors. Billing and monitoring are **per variant row** (price and stock change at variant level).

#### A shop shows `blocked` (403). Why?

Some brands block anonymous `/products.json` access. The Actor **soft-fails** those stores (status `blocked` in `RUN_SUMMARY`) and continues with others. v1 does not spend residential proxies.

#### Can I scrape only one collection?

Yes. Set **Collection handles** to crawl `/collections/{handle}/products.json` instead of the full catalog.

#### Do I need login or a browser?

No. Public JSON only. Password-protected / headless stores are not supported.

#### How is this different from a full-page catalog dump?

You get **variant-level** rows plus optional **monitor mode** (`price_up` / `price_down` / `stock_in` / `stock_out` / `new` / `removed`) so scheduled runs stay cheap.

### Limitations

- **Public JSON only.** Does not scrape theme HTML, cart, checkout, or customer accounts.
- **Some brands return 403** on `/products.json`. Those are reported as `blocked` in `RUN_SUMMARY`. No residential proxy spend in v1.
- **Currency** comes from `/meta.json` when available; otherwise `null`.
- **Password-protected / headless stores** are not supported.
- Large catalogs (tens of thousands of products) take more wall time; use caps or collection handles.

### Related

| Actor | Use for |
|---|---|
| [Greenhouse / Lever / Ashby Jobs Scraper + Monitor](https://apify.com/mayevil/greenhouse-lever-jobs-scraper) | ATS job boards + hiring signals |
| [Workday Jobs Scraper + Monitor](https://apify.com/mayevil/workday-jobs-monitor) | `*.myworkdayjobs.com` CXS career boards |

# Actor input Schema

## `shops` (type: `array`):

One store per line. Accepted: allbirds.com, www.allbirds.com, https://www.allbirds.com. Path/query are ignored.

## `collectionHandles` (type: `array`):

If set, crawl only these collections via /collections/{handle}/products.json instead of the full catalog. Leave empty for /products.json.

## `includeCollectionsIndex` (type: `boolean`):

Calls /collections.json and records the count in RUN_SUMMARY (does not emit dataset rows).

## `titleKeywords` (type: `array`):

Case-insensitive substrings. Leave empty to keep all titles.

## `titleExcludeKeywords` (type: `array`):

Case-insensitive substrings.

## `vendorKeywords` (type: `array`):

Matched against the product vendor field, e.g. "Allbirds".

## `productTypeKeywords` (type: `array`):

Matched against product_type, e.g. "Shoes", "Apparel".

## `tagKeywords` (type: `array`):

Matched against product tags (any tag containing the substring).

## `inStockOnly` (type: `boolean`):

Keep only variants where available === true.

## `monitorMode` (type: `boolean`):

Remembers variant fingerprints (price, available, title) and outputs new / price / stock / removed rows. You only pay for changes.

## `monitorStateKey` (type: `string`):

Name of the watchlist memory. Use a different key per schedule.

## `skipExistingOnFirstRun` (type: `boolean`):

Baseline only on first sight of a shop in monitor mode.

## `outputPriceChanges` (type: `boolean`):

In monitor mode, emit status price_up / price_down when a variant price changes.

## `outputStockChanges` (type: `boolean`):

In monitor mode, emit status stock_in / stock_out when availability flips.

## `outputRemoved` (type: `boolean`):

In monitor mode, emit status "removed" for variants that disappeared since the last run.

## `maxProductsPerShop` (type: `integer`):

Cap on distinct products per store (all variants of kept products are still output). 0 = no cap.

## `maxProductsTotal` (type: `integer`):

Stop after this many dataset items (cost control). 0 = no cap. Each variant is one row.

## `maxConcurrency` (type: `integer`):

How many stores are processed in parallel. Per-host JSON calls are also throttled.

## `maxRequestsPerMinute` (type: `integer`):

Upper bound on how many shops are started per minute.

## Actor input object example

```json
{
  "shops": [
    "allbirds.com"
  ],
  "collectionHandles": [],
  "includeCollectionsIndex": false,
  "titleKeywords": [],
  "titleExcludeKeywords": [],
  "vendorKeywords": [],
  "productTypeKeywords": [],
  "tagKeywords": [],
  "inStockOnly": false,
  "monitorMode": false,
  "monitorStateKey": "default",
  "skipExistingOnFirstRun": false,
  "outputPriceChanges": true,
  "outputStockChanges": true,
  "outputRemoved": false,
  "maxProductsPerShop": 0,
  "maxProductsTotal": 5000,
  "maxConcurrency": 5,
  "maxRequestsPerMinute": 60
}
```

# Actor output Schema

## `products` (type: `string`):

No description

## `runSummary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "shops": [
        "allbirds.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("mayevil/shopify-catalog-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "shops": ["allbirds.com"] }

# Run the Actor and wait for it to finish
run = client.actor("mayevil/shopify-catalog-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "shops": [
    "allbirds.com"
  ]
}' |
apify call mayevil/shopify-catalog-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,mayevil/shopify-catalog-monitor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/84Lgqw9W0c1wbblFo/builds/p7rGhoAcADrElmcyf/openapi.json