# Shopify Scraper — Stores & Products (`spool/shopify-store-products-scraper`) Actor Shopify store scraper and Shopify products scraper in one run: every product and variant with GTIN barcodes, plus store-level launch velocity, price positioning, discounting, brand model and the apps the store runs. - **URL**: https://apify.com/spool/shopify-store-products-scraper.md - **Developed by:** [Spool](https://apify.com/spool) (community) - **Categories:** E-commerce, Lead generation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.40 / 1,000 products This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Shopify Scraper — Stores & Products **Point it at a Shopify store. Get every product and variant with GTIN barcodes, *and* a read on the business itself — how fast it ships, where it prices, how hard it discounts, and which apps it pays for.** ``` store deathwishcoffee.com 0.03 new/day median $40 dormant · 8 apps · Dawn theme product Coconut Caramel Coffee / 1 bag 810063344313 $13.99 subscribe & save 10% product Valhalla Java / 5 lb bag 810063342159 $89.99 in stock, tracked ``` Most Shopify scrapers give you rows of products. This gives you the catalog **and** what the catalog says about the merchant, from the same read. *** ### Two records, one run Set `output` and you get either or both: **The store record** — one row per store. Launch velocity measured properly, price bands in USD, discount depth, stock pressure, brand model, catalog scale, and the apps and theme the storefront loads. **The product records** — one row per variant (or per product), each with the GTIN barcode, price, stock, subscription plan and collection membership. Use `store` alone for a fast, cheap scan of a hundred competitors. Use `products` for a pure catalog extract. Use `both` when you want the business and its catalog together. *** ### The barcode `/products.json` — the endpoint nearly every Shopify scraper reads — **does not contain the barcode**. GTIN / UPC / EAN is what connects a Shopify product to the rest of commerce: to Amazon, to Google Shopping, to a distributor's price list, to your own ERP. Without it you have product titles and manual matching. It lives one layer down, at `/products/{handle}.js`, which costs one request per product. Measured across eight stores it came back on **97–100% of variants for six of them**. The other two are the merchant's doing, not the endpoint's: Rothy's is 81% — its newest products are fully barcoded, its oldest half are not — and Death Wish Coffee is 26%: ``` rothys.com 81% organicbasics.com 97% allbirds.com 98% gymshark.com 100% finisterre.com 100% kith.com 100% tentree.com 100% deathwishcoffee.com 26% ``` The same endpoint carries four more things `/products.json` lacks: **subscription plans** with the real subscriber price, **wholesale volume breaks**, **order minimums**, and **`inventory_management`** — which decides whether "in stock" means anything, because when a merchant does not track inventory `available` is a default rather than a fact. `inventoryTracked` tells you which kind you are looking at. *** ### Velocity that is actually right The store record reports how fast the store ships new product. Getting that right took three attempts, because the obvious methods are all wrong. `published_at` looks like a launch date and is not — it is rewritten in bulk whenever a store replatforms. Allbirds spans **7.6 years by `created_at` and 83 days by `published_at`**. Deriving a rate from it is wrong in both directions at once: ``` from published_at measured properly allbirds.com 3.54/day 0.00/day overstated without bound gymshark.com 12.39/day 0.00/day overstated without bound finisterre.com 0.28/day 1.86/day 6.6x too low rothys.com 0.22/day 0.81/day 3.7x too low ``` So both rates are reported: `perDay` counts products genuinely **created** in the last 90 days, `publishPerDay` counts what was **published**. The gap is the finding — Gymshark has created nothing since May while publishing sixteen a day, which is re-merchandising, not dying, and reads as `republishing`. **And it says when it does not know.** On a store too large to read in full, `velocityConfidence` comes back `low` and the number is a lower bound rather than a wrong answer. It fires only when it should: Organic Basics reads 1.76/day from 250 products against 1.77/day from its full 1,816. *** ### Which apps the store pays for The catalog says what a store sells. The storefront says how it **operates**. ``` deathwishcoffee.com 8 apps Klaviyo · Postscript · Rebuy · Smile.io · Afterpay Dawn rothys.com 2 apps Intelligems · Yotpo DNA ``` Over 50 apps recognised across email/SMS, reviews, support, subscriptions, CRO, loyalty, search, analytics, post-purchase, payments and affiliate. Anything third-party that cannot be named is still reported in `otherThirdPartyHosts`, so a store running something unusual shows up rather than looking simpler than it is. `themeIsStock` flags the free Shopify themes — a store still on stock Dawn has not invested in design. *** ### What the merchant says is selling `/collections.json` is the store's own merchandising, and it is public. A product in the merchant's `best-sellers` collection is a demand signal **the store is publishing itself**. Reading all of it is wasteful — Rothy's has 196 collections, Allbirds 1,000, mostly colour and size groupings. So collections are **ranked, not matched**: best sellers first, then new arrivals, trending, sale, each capped so one kind cannot crowd out another, with internal plumbing (`Discount Eligible Products Excluding Markdown`, test and hidden collections) dropped. On Allbirds that turns 1,000 collections into 23 useful reads. *** ### Prices you can compare `products.json` carries prices as bare numbers with **no currency in it**. A UK store showing `50` and a US store showing `50` are £50 and $50. Currency is read from the store profile and every price reported twice, in store currency and USD. When no rate is available the USD fields are **null**, never a guess. A trap worth naming: `/products.json` gives decimal strings while `/products/{handle}.js` gives integer **cents**. Mixing them divides enriched prices by a hundred. Each source is normalised on the way in. *** ### Quick start ```json { "storeUrls": ["rothys.com", "deathwishcoffee.com"] } ``` Domains, full URLs and product links all work. `www.` is stripped, and `www.x.com` and `x.com` merge so you are never billed twice for one store. *** ### Recipes **Scan 100 competitors fast — one row each, no per-product cost** ```json { "storeUrls": ["..."], "output": "store" } ``` **Build a Google Merchant feed** ```json { "storeUrls": ["rothys.com"], "output": "products", "requireBarcode": true } ``` `feedReady` says whether a row has everything Merchant Center needs — id, title, link, image, brand, price, availability and GTIN. **Every subscription product and what subscribing saves** ```json { "storeUrls": ["deathwishcoffee.com"], "requireSubscription": true } ``` **Everything discounted under $50 across rivals** ```json { "storeUrls": ["a.com", "b.com"], "onlyOnSale": true, "maxPrice": 50 } ``` *** ### Every option | Option | Default | What it does | |---|---|---| | **`storeUrls`** | — | **Required.** Domains, URLs or product links | | **`output`** | `both` | `both`, `products`, or `store` for a fast one-row scan | | **`enrich`** | `true` | **Barcodes, subscriptions, wholesale.** One request per product | | `rowsPer` | `variant` | `variant` for feeds and price monitoring, `product` for analysis | | `detectApps` | `true` | Apps and theme on the store record. One request per store | | `maxProductsPerStore` | `1000` | Read depth — the main lever on time and cost | | `collectionsMode` | `key` | `key` reads signal collections only, `all`, or `none` | | **`requireBarcode`** | `false` | Keep only rows with a GTIN | | `onlyInStock` / `onlyOnSale` | `false` | Stock and discount filters | | `requireSubscription` | `false` | Subscription products only | | `minPrice` / `maxPrice` | — | Compared in USD, so consistent across stores | | `vendors` / `productTypes` / `tags` | — | Filter by brand, type or tag | | `includeDescription` | `false` | Adds full descriptions. Large | | `maxConcurrency` | `5` | Parallel requests. Lower it if a store returns 429 | *** ### What it covers, honestly **About five stores in six work.** Of 18 real stores tested, 15 served the catalog. The rest return `not_a_shopify_store`. A miss is always reported. **Barcode coverage is the merchant's choice.** Six of eight stores returned 97–100%. Rothy's returned 81% — its older products were never given one — and Death Wish Coffee 26%, because they have not entered them. A null barcode means the merchant left it empty; if the store throttled the run instead, the log says so and `RUN_SUMMARY` counts the rows affected. **`catalogSize` is exact only when `catalogComplete` is true.** Otherwise it is null rather than a number that would be wrong. **App detection reads what the storefront loads.** A retailer that self-hosts its scripts shows fewer apps than it runs — Gymshark detects only one — so a low count on a big brand means "not visible", not "not installed". **Currency is what the store serves *you*.** Shopify switches storefront by geography. **Direct requests first; Apify Proxy only for what a store refuses.** Shopify throttles the platform's shared addresses by address, not by rate — Allbirds enriched 30% of its catalog directly from the platform and 96% once the refused products were retried through Apify Proxy, while the same requests pass 100% from an ordinary home connection. So every product is tried directly, and only the ones the store turned down go through the proxy, one fresh session each. It is included in the price; the log says how many it recovered, and anything still missing is reported as missing rather than passed off as "no barcode". **Prices and stock are a snapshot**, true at the moment of the run. *** ### Is this legal? Everything read is a public endpoint Shopify serves by design, requiring no authentication and permitted by robots.txt: `/products.json`, `/products/{handle}.js`, `/collections.json` and `/meta.json`. **What this Actor deliberately does not do:** `/cart/add.js` reveals a variant's exact remaining stock if you request an absurd quantity, and tools advertising "real inventory counts" appear to use it. Shopify's default robots.txt **disallows `/cart`** on every store checked, and `/search` with it. Neither is touched here. Exact stock counts are not worth ignoring a merchant's robots.txt for. Everything returned is catalog data. No customer data, no personal information. *** ### When a store can't be read | `error` | Meaning | |---|---| | `not_a_shopify_store` | No public catalog at `/products.json` | | `catalog_blocked` | The store returned 403 or 429 | | `catalog_empty` | Catalog exists but has no products | | `all_products_filtered_out` | Read fine, but your filters matched nothing | | `invalid_input` | Couldn't read a domain from that value | A breakdown is saved to the key-value store as `RUN_SUMMARY`. *** ### FAQ **How do I scrape a Shopify store?** Pass the domain. It reads the public catalog every Shopify store serves and returns the products plus a summary of the store itself. **Is this a Shopify store scraper or a Shopify products scraper?** Both, from one read. As a Shopify store scraper it returns one summary row per store — launch velocity, pricing, discounting, brand model and installed apps. As a Shopify products scraper it returns every product and variant with its GTIN barcode, price, stock and collections. Set `output` to get either on its own. **How do I get GTIN or UPC codes from Shopify?** Leave `enrich` on. Barcodes come from `/products/{handle}.js`, which `/products.json` does not include. **How is this different from other Shopify scrapers?** Most read one endpoint and return titles, prices and images. This adds barcodes, subscription plans with real subscriber prices, wholesale breaks, whether stock is actually tracked, the merchant's own best-seller lists, the apps the store runs, and a correct launch-velocity measurement. **Can I scrape every Shopify store at once?** No. You supply the store list. It reads the stores you name, thoroughly. **Will it get me blocked?** Plain HTTPS requests at a concurrency you control. Lower `maxConcurrency` if a store returns 429. *** ### How to use it 1. Open the Actor and paste store domains into **Shopify stores**. 2. Choose **What to return** — `both`, `products` for a catalog extract, or `store` for a fast one-row-per-store scan. 3. Click **Start**. Two stores with 800 products take about 45 seconds. 4. Open the **Stores** view for the business read, or **Merchant feed** / **Pricing** / **Subscriptions** for the products. Filter on `recordType` when exporting. 5. **Schedule** it to track competitors over time, or call it through the API and integrations. *** ### How much does it cost? Pay per event. Two things are charged: **one store summary** per store, and **one charge per product** — a product with 20 variants is 20 rows and one charge. | Plan | store summary | product, enriched | product, catalog only | |---|---|---|---| | Free | $0.02 | $0.002 | $0.0005 | | Bronze | $0.018 | $0.0018 | $0.00045 | | Silver | $0.016 | $0.0016 | $0.0004 | | Gold | $0.014 | $0.0014 | $0.00035 | Actor start is $0.002. Stores that could not be read cost nothing. If a store throttles the per-product requests, the products that could not be enriched are billed at the catalog-only price, not the enriched one — you pay for barcodes only when you get them. **Worked examples** - `output: "store"` on **100 competitors** → 100 × $0.02 = **$2.00**, no per-product cost at all - **3 stores, 2,000 products, both records** → 3 × $0.02 + 2,000 × $0.002 = **$4.06** - Rothy's + Death Wish at the defaults (**812 products**, 2 summaries) → **$1.66** - The **free plan's $5 credit** covers about **2,400 enriched products** *** ### More Shopify tools from spool - **[Shopify Stores API — Velocity, Pricing & Brand Model](https://apify.com/spool/shopify-stores-api)** — one row per store: launch velocity measured properly, price bands in USD, discount depth, brand model, and the apps the storefront runs - **[Shopify Product Scraper — GTIN & Collections](https://apify.com/spool/shopify-product-feed)** — every product and variant with GTIN barcodes, subscription plans, wholesale breaks and best-seller membership All three read the same public endpoints, respect robots.txt, and report a miss rather than guessing. *** ### Support Open an issue on the **Issues** tab and you will get a reply, usually the same day. Requests for extra fields are welcome. # Actor input Schema ## `storeUrls` (type: `array`): Domains, full URLs or product links — mixed freely. `www.` is stripped, and `www.x.com` and `x.com` are merged so one store is never read twice. About five stores in six serve a public catalog. A store that does not is reported as `not_a_shopify_store` rather than silently skipped. ## `output` (type: `string`): **Both** returns a store summary row plus every product row — the store record tells you how fast it ships, where it prices and what apps it runs; the product rows carry barcodes, variants and stock. Use **products only** for a pure catalog extract, or **store only** for a fast one-row-per-store scan that skips the per-product requests entirely. ## `enrich` (type: `boolean`): **This is the reason to use this Actor.** `/products.json`, which every other Shopify scraper reads, does not contain the GTIN/UPC barcode, subscription plans, wholesale price breaks or whether stock is even tracked. Those live on a second endpoint, one request per product. Measured across eight stores, barcodes came back on **97-100% of variants for six of them**, 81% on Rothy's and 26% on Death Wish Coffee. Turn this off for a fast, cheap catalog dump with no barcodes. ## `rowsPer` (type: `string`): **Variant** gives one row per buyable SKU, each with its own barcode, price and stock. This is the shape a price monitor or a Google Merchant feed needs. **Product** gives one row per product with the variants nested, which is easier for catalog analysis. ## `maxProductsPerStore` (type: `integer`): Shopify serves 250 products per request, newest-published first. With enrichment on, each product costs one extra request, so this is also the main lever on run time and price. ## `collectionsMode` (type: `string`): Which of the merchant's own collections to map each product against. A product sitting in the store's `best-sellers` collection is a demand signal the merchant is publishing themselves. **Signal collections** reads only the ones that mean something — best sellers, sale, clearance, new arrivals, trending. Rothy's publishes 196 collections; reading all of them would cost hundreds of requests to learn mostly colour and size groupings. ## `detectApps` (type: `boolean`): Read the storefront for the apps the store pays for — Klaviyo, Recharge, Gorgias, Yotpo, Okendo and 50 others — plus its theme. One extra request per store. Included on the store record. ## `requireBarcode` (type: `boolean`): Keep only rows carrying a barcode. Use this when you are building a Google Merchant Center feed or matching against Amazon, where a row without a GTIN is unusable. Needs enrichment on. ## `onlyInStock` (type: `boolean`): Keep only available variants. Note that `available` is only meaningful when the merchant tracks inventory — `inventoryTracked` in the output says whether they do. ## `onlyOnSale` (type: `boolean`): Keep only variants whose price is below their compare-at price. `discountPercent` carries the depth. ## `requireSubscription` (type: `boolean`): Keep only products offering a subscription or subscribe-and-save plan, with the real subscriber price and discount. Needs enrichment on. ## `minPrice` (type: `integer`): Compared against the USD price, so it means the same thing on a GBP store as on a USD one. ## `maxPrice` (type: `integer`): Compared against the USD price. ## `vendors` (type: `array`): Brand names as they appear in the store. Useful on multi-brand retailers — Kith stocks 138 vendors. ## `productTypes` (type: `array`): The store's own product types, such as `Socks` or `Men's Jackets`. ## `tags` (type: `array`): Keeps a product carrying any one of these tags. ## `includeDescription` (type: `boolean`): Adds the full description to every row. Off by default because it is large and most uses do not need it. ## `maxConcurrency` (type: `integer`): How many per-product requests run at once. Shopify throttles by address rather than by rate — the built-in Apify Proxy fallback handles that — so this mostly trades speed for politeness. ## Actor input object example ```json { "storeUrls": [ "allbirds.com", "https://gymshark.com" ], "output": "both", "enrich": true, "rowsPer": "variant", "maxProductsPerStore": 1000, "collectionsMode": "key", "detectApps": true, "requireBarcode": false, "onlyInStock": false, "onlyOnSale": false, "requireSubscription": false, "includeDescription": false, "maxConcurrency": 5 } ``` # Actor output Schema ## `products` (type: `string`): Every product or variant returned, with all fields. ## `merchantFeed` (type: `string`): Only the rows carrying everything Google Merchant Center needs, including a GTIN. ## `subscriptions` (type: `string`): Products sold on a subscription plan, with the real subscriber price. ## `runSummary` (type: `string`): Rows returned, products enriched, and any stores that could not be read. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "storeUrls": [ "deathwishcoffee.com", "allbirds.com" ] }; // Run the Actor and wait for it to finish const run = await client.actor("spool/shopify-store-products-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "storeUrls": [ "deathwishcoffee.com", "allbirds.com", ] } # Run the Actor and wait for it to finish run = client.actor("spool/shopify-store-products-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "storeUrls": [ "deathwishcoffee.com", "allbirds.com" ] }' | apify call spool/shopify-store-products-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,spool/shopify-store-products-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/eglf1XIhhHDT7MMBc/builds/FGK0HtRIt0lL5Mkhl/openapi.json