# Shopify Scraper: Products, Variants & Store Summary (`everyotherfriday/shopify-catalog`) Actor

Export Shopify products with variants, prices, availability and images. Filter by collection, price or update date, and get store summaries with catalogue size, vendors, price ranges and detected apps. Built for price monitoring, competitor research and DTC prospecting.

- **URL**: https://apify.com/everyotherfriday/shopify-catalog.md
- **Developed by:** [Paul Vasquez](https://apify.com/everyotherfriday) (community)
- **Categories:** E-commerce, Lead generation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 store summaries

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Shopify Catalogue and Store Summary

Collect public Shopify product catalogues without an Admin API key. This actor returns normalized products, optional variant and image details, and a compact storefront summary. Use it for catalogue comparisons, assortment research, availability monitoring, and periodic exports. It reads public storefront HTTP endpoints; it does not sign in, purchase products, or access merchant administration.

### Quick start

Use Python 3.12. Create a virtual environment, install `requirements.txt`, then run `python -m src`. For a local SDK run, place input at `storage/key_value_stores/default/INPUT.json`. Alternatively, use `apify run` after installing the Apify CLI. The supplied `INPUT.json` selects Allbirds and tentree (Gymshark detail endpoints returned 404) with a 300-product cap per store. `validation/run_live.ps1` copies that input into a fresh local storage directory, runs the real SDK, and records counts and elapsed time in `validation/results.json`.

The daily-test input requires no merchant keys. Its observed runtime is recorded in VALIDATION.md; remote rate limits and outages can increase it. The Dockerfile uses Apify's Python 3.12 base image. No deployment is performed by the validation script.

### Inputs and selection

`storeUrls` is a required array of storefront root URLs. Bare hostnames receive HTTPS; paths, query strings, fragments, and embedded credentials are rejected. Duplicate stores are removed. `collections` optionally restricts discovery to collection handles, with duplicate products removed across those collections. Omit it to read the general catalogue.

`maxProductsPerStore` defaults to 2000 matching products. `includeVariants`, `includeImages`, and `includeStoreSummary` default to true. Disabling variant or image output returns empty arrays while retaining product-level prices and availability. `onlyAvailable` requires explicitly available products. `priceMin` and `priceMax` are inclusive bounds: a product qualifies when its variant price range overlaps the requested interval. Unknown prices fail an active price filter. Prices use each storefront's currency; no exchange conversion occurs.

`updatedAfter` accepts an exclusive ISO date or timestamp. Naive dates use UTC. Missing or invalid source update dates fail that filter. `timeoutSecs` defaults to 20 seconds per HTTP operation. `proxyConfiguration` defaults to direct requests with `useApifyProxy: false`; an Apify proxy or custom proxy URLs can be configured when a particular storefront requires them.

### Discovery and fallback

The actor detects Shopify through response headers or recognizable homepage assets. Sites without that evidence produce a free error row. Products come from `/products.json?limit=250&page=N`, or the corresponding `/collections/{handle}/products.json` route. Collection counts come from paginated `/collections.json`. Each route has a 100-page scan ceiling, and repeated pages stop discovery. The returned-product cap applies after filtering, so restrictive filters can require substantially more requests.

If the general product feed is unavailable or malformed, discovery falls back to `/sitemap_products_1.xml` and individual `/products/{handle}.js` requests, with a warning. Only the first product sitemap is covered, up to 25,000 candidate handles. This fallback can be slower and may lack update dates. Failed collection routes do not use an unrestricted sitemap because membership cannot be inferred. HTTP 429 and server errors receive two retries with one-second and two-second backoff. Transport failures receive the same retries; other client errors do not. Five consecutive fallback detail failures stop that fallback.

### Results

Product rows include identity, handle, title, vendor, type, tags, timestamps, canonical product URL, price bounds, currency, availability, public inventory, variants, images, description, options, and source URL. Variant records include SKU, comparison price, options, weight, and shipping requirements. Ajax amounts are converted from cents; feed decimal amounts are retained. Hidden inventory remains null. Currency comes from explicit product data or cheap homepage metadata, otherwise null. Image dimensions and alternative text remain null when unavailable. `bodyHtml` is limited to 2000 characters; `bodyText` contains the plain-text description.

`STORE-<host>` contains the same summary offered as a dataset row. Aggregates describe successfully emitted, filtered products, not necessarily the whole store. Summaries include collection count, top 20 vendors with counts, top 20 product types, price range, currency, known-availability share, latest product update, warnings, and detected apps. App fingerprints include Klaviyo, Judge.me, Yotpo, Recharge, Gorgias, Loox, and Shop Pay. These are HTML hints, not proof of active subscriptions. `SUMMARY` records per-store counts and timings.

### Pricing and validation

Configure `product-returned` at $0.0004 and `store-summary` at $0.002 in Apify Console. One charge request precedes each successful paid row. Error rows and zero-product summaries are free; local SDK runs do not bill. Charge-limit checks stop paid output. A storage failure after charging cannot roll back that charge. Disable synthetic start and dataset events before publication.

Run `python -m unittest discover -s tests -v` and `apify validate-schema .actor/input_schema.json`. Mocked tests cover normalization, filters, pagination, retries, detection, fallback, and charging. See VALIDATION.md for live evidence and deployment limitations. Shopify's [Ajax product reference](https://shopify.dev/docs/api/ajax/reference/product) documents the detail endpoint and its public-data limitations.

### Example output

Recorded local validation output from [storage/live-20260926-062111/datasets/default/000000001.json](storage/live-20260926-062111/datasets/default/000000001.json), dataset row 1. Fields are omitted for brevity; retained values are unchanged. This is a historical example, not a current-source claim.

```json
{
    "rowType":  "product",
    "store":  "https://www.allbirds.com",
    "productId":  7340901859408,
    "title":  "Women\u0027s Allbirds Flip Flop - Dusty Pink",
    "vendor":  "Allbirds",
    "productType":  "Shoes",
    "priceMin":  25.0,
    "priceMax":  25.0,
    "currency":  "USD",
    "available":  true,
    "totalInventory":  null
}
```

### Use cases

- An ecommerce merchandising manager compares returned footwear assortments across selected stores using product types, variants, and prices.
- A retail pricing analyst saves repeated catalogue exports and compares the same product IDs within each storefront currency.
- An inventory planning consultant reviews public availability flags to identify products needing manual stock checks with the merchant.
- An agency account strategist reviews store summaries and detected app hints before preparing a storefront assessment for a client.

### Pricing example

1,000 product rows and 10 nonempty store summaries cost (1,000 x $0.0004) + (10 x $0.002) = **$0.42** in declared events. Rates come from [the local event declaration](.actor/pay_per_event.json). This calculation is an event subtotal, not a measured invoice; local validation does not bill.

### Limitations

Public availability does not reveal hidden stock quantities. Summaries cover emitted products, so caps and filters change the apparent assortment. Endpoint blocking and the bounded sitemap fallback can leave gaps. Compare currencies explicitly and investigate warnings before drawing store-wide conclusions.

# Actor input Schema

## `storeUrls` (type: `array`):

Storefront root URLs, without paths or credentials.

## `collections` (type: `array`):

Optional collection handles. Products are deduplicated across collections.

## `maxProductsPerStore` (type: `integer`):

Maximum matching unique products returned per store; scanning capped at 100 pages per route.

## `includeVariants` (type: `boolean`):

Include public variant details.

## `includeImages` (type: `boolean`):

Include public image details.

## `onlyAvailable` (type: `boolean`):

Return only products with at least one available variant.

## `includeStoreSummary` (type: `boolean`):

Save and return aggregates over emitted products.

## `priceMin` (type: `number`):

Inclusive lower price bound; matches overlapping product price ranges.

## `priceMax` (type: `number`):

Inclusive upper price bound; matches overlapping product price ranges.

## `updatedAfter` (type: `string`):

Exclusive ISO date/time cutoff. Products without valid update dates are excluded.

## `timeoutSecs` (type: `integer`):

Timeout per HTTP operation; two retries on 429/5xx.

## `proxyConfiguration` (type: `object`):

Optional proxy configuration. Direct requests are the default.

## Actor input object example

```json
{
  "storeUrls": [
    "https://www.allbirds.com",
    "https://www.tentree.com"
  ],
  "collections": [],
  "maxProductsPerStore": 10,
  "includeVariants": true,
  "includeImages": true,
  "onlyAvailable": false,
  "includeStoreSummary": true,
  "timeoutSecs": 20,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `dataset` (type: `string`):

All rows in JSON

## `csv` (type: `string`):

All rows in CSV

## `stores` (type: `string`):

STORE-host records and SUMMARY timings

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "storeUrls": [
        "https://www.allbirds.com",
        "https://www.tentree.com"
    ],
    "maxProductsPerStore": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("everyotherfriday/shopify-catalog").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "storeUrls": [
        "https://www.allbirds.com",
        "https://www.tentree.com",
    ],
    "maxProductsPerStore": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("everyotherfriday/shopify-catalog").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "storeUrls": [
    "https://www.allbirds.com",
    "https://www.tentree.com"
  ],
  "maxProductsPerStore": 10
}' |
apify call everyotherfriday/shopify-catalog --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,everyotherfriday/shopify-catalog"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ux2VKhGQ4hpV4DxBY/builds/eDFsY9oyBRKVVhYuC/openapi.json