Shopify Catalogue and Store Summary

Collect public Shopify product catalogues without an Admin API key. This actor returns normalized products, optional variant and image details, and a compact storefront summary. Use it for catalogue comparisons, assortment research, availability monitoring, and periodic exports. It reads public storefront HTTP endpoints; it does not sign in, purchase products, or access merchant administration.

Quick start

Use Python 3.12. Create a virtual environment, install requirements.txt, then run python -m src. For a local SDK run, place input at storage/key_value_stores/default/INPUT.json. Alternatively, use apify run after installing the Apify CLI. The supplied INPUT.json selects Allbirds and tentree (Gymshark detail endpoints returned 404) with a 300-product cap per store. validation/run_live.ps1 copies that input into a fresh local storage directory, runs the real SDK, and records counts and elapsed time in validation/results.json.

The daily-test input requires no merchant keys. Its observed runtime is recorded in VALIDATION.md; remote rate limits and outages can increase it. The Dockerfile uses Apify's Python 3.12 base image. No deployment is performed by the validation script.

Inputs and selection

storeUrls is a required array of storefront root URLs. Bare hostnames receive HTTPS; paths, query strings, fragments, and embedded credentials are rejected. Duplicate stores are removed. collections optionally restricts discovery to collection handles, with duplicate products removed across those collections. Omit it to read the general catalogue.

maxProductsPerStore defaults to 2000 matching products. includeVariants, includeImages, and includeStoreSummary default to true. Disabling variant or image output returns empty arrays while retaining product-level prices and availability. onlyAvailable requires explicitly available products. priceMin and priceMax are inclusive bounds: a product qualifies when its variant price range overlaps the requested interval. Unknown prices fail an active price filter. Prices use each storefront's currency; no exchange conversion occurs.

updatedAfter accepts an exclusive ISO date or timestamp. Naive dates use UTC. Missing or invalid source update dates fail that filter. timeoutSecs defaults to 20 seconds per HTTP operation. proxyConfiguration defaults to direct requests with useApifyProxy: false; an Apify proxy or custom proxy URLs can be configured when a particular storefront requires them.

Discovery and fallback

The actor detects Shopify through response headers or recognizable homepage assets. Sites without that evidence produce a free error row. Products come from /products.json?limit=250&page=N, or the corresponding /collections/{handle}/products.json route. Collection counts come from paginated /collections.json. Each route has a 100-page scan ceiling, and repeated pages stop discovery. The returned-product cap applies after filtering, so restrictive filters can require substantially more requests.

If the general product feed is unavailable or malformed, discovery falls back to /sitemap_products_1.xml and individual /products/{handle}.js requests, with a warning. Only the first product sitemap is covered, up to 25,000 candidate handles. This fallback can be slower and may lack update dates. Failed collection routes do not use an unrestricted sitemap because membership cannot be inferred. HTTP 429 and server errors receive two retries with one-second and two-second backoff. Transport failures receive the same retries; other client errors do not. Five consecutive fallback detail failures stop that fallback.

Results

Product rows include identity, handle, title, vendor, type, tags, timestamps, canonical product URL, price bounds, currency, availability, public inventory, variants, images, description, options, and source URL. Variant records include SKU, comparison price, options, weight, and shipping requirements. Ajax amounts are converted from cents; feed decimal amounts are retained. Hidden inventory remains null. Currency comes from explicit product data or cheap homepage metadata, otherwise null. Image dimensions and alternative text remain null when unavailable. bodyHtml is limited to 2000 characters; bodyText contains the plain-text description.

STORE-<host> contains the same summary offered as a dataset row. Aggregates describe successfully emitted, filtered products, not necessarily the whole store. Summaries include collection count, top 20 vendors with counts, top 20 product types, price range, currency, known-availability share, latest product update, warnings, and detected apps. App fingerprints include Klaviyo, Judge.me, Yotpo, Recharge, Gorgias, Loox, and Shop Pay. These are HTML hints, not proof of active subscriptions. SUMMARY records per-store counts and timings.

Pricing and validation

Configure product-returned at $0.0004 and store-summary at $0.002 in Apify Console. One charge request precedes each successful paid row. Error rows and zero-product summaries are free; local SDK runs do not bill. Charge-limit checks stop paid output. A storage failure after charging cannot roll back that charge. Disable synthetic start and dataset events before publication.

Run python -m unittest discover -s tests -v and apify validate-schema .actor/input_schema.json. Mocked tests cover normalization, filters, pagination, retries, detection, fallback, and charging. See VALIDATION.md for live evidence and deployment limitations. Shopify's Ajax product reference documents the detail endpoint and its public-data limitations.

Example output

Recorded local validation output from storage/live-20260926-062111/datasets/default/000000001.json, dataset row 1. Fields are omitted for brevity; retained values are unchanged. This is a historical example, not a current-source claim.

{
    "rowType":  "product",
    "store":  "https://www.allbirds.com",
    "productId":  7340901859408,
    "title":  "Women\u0027s Allbirds Flip Flop - Dusty Pink",
    "vendor":  "Allbirds",
    "productType":  "Shoes",
    "priceMin":  25.0,
    "priceMax":  25.0,
    "currency":  "USD",
    "available":  true,
    "totalInventory":  null
}

Use cases

• An ecommerce merchandising manager compares returned footwear assortments across selected stores using product types, variants, and prices.
• A retail pricing analyst saves repeated catalogue exports and compares the same product IDs within each storefront currency.
• An inventory planning consultant reviews public availability flags to identify products needing manual stock checks with the merchant.
• An agency account strategist reviews store summaries and detected app hints before preparing a storefront assessment for a client.

Pricing example

1,000 product rows and 10 nonempty store summaries cost (1,000 x $0.0004) + (10 x $0.002) = $0.42 in declared events. Rates come from .actor/pay_per_event.json. This calculation is an event subtotal, not a measured invoice; local validation does not bill.

Limitations

Public availability does not reveal hidden stock quantities. Summaries cover emitted products, so caps and filters change the apparent assortment. Endpoint blocking and the bounded sitemap fallback can leave gaps. Compare currencies explicitly and investigate warnings before drawing store-wide conclusions.