Shopify Store Scraper: Product Catalogue Data

Shopify Store Scraper — Store Profiles, Collections and Vendor Data

Shopify Store Scraper: Product Catalogue Data profiles any public Shopify storefront and returns a verified, store-level snapshot — a walked product count checked against the store's own reported count, the vendor and product-type mix, a defensible price distribution, and every collection the merchant curates for merchandising. Every result comes back as typed, normalized JSON: one row per store, one row per curated collection, no HTML and no selectors to parse. Point it at a list of storefronts and get catalogue profiles you can compare, filter, and monitor side by side.

🧭 What is Shopify Store Scraper: Product Catalogue Data?

Shopify Store Scraper: Product Catalogue Data is an Apify Actor that reads the public product feed, collection feed, and store metadata that every open Shopify storefront already serves, and turns them into a single store-level profile instead of thousands of product rows. It returns three things for every storefront you queue: a verified catalogue count, a catalogue-composition summary (vendors, product types, price distribution), and a classification of which collections the merchant is actively merchandising. No Shopify account, app install, or API key is required — the Actor only reads addresses you already have, the same way a browser would.

• 🏪 Profile the store: verified vs. store-reported product count, with the gap surfaced, not hidden
• 🏷️ Break down catalogue composition: vendor mix, product-type mix, and a price distribution built only from buyable variants
• ⭐ Detect and export curated collections: best sellers, new arrivals, sale, staff picks, featured, gifts — with the exact wording that justified each label
• ⬇️ Export as JSON, CSV, Excel, or any other format Apify datasets support
• 🌐 No proxy management required — Apify Proxy is optional, off by default

📊 What data does Shopify Store Scraper: Product Catalogue Data collect?

Every run returns three kinds of data on the same storefront: a verified store profile, a catalogue-composition breakdown, and curated-collection classification.

Data Type	Key Fields	JSON Field Names
Store catalogue profile	Verified vs. store-reported product count, storefront status, store identity	verifiedProductCount, metaReportedProductCount, productCountDiscrepancy, storefrontStatus, store_domain, shop_name
Catalogue composition	Vendor mix, product-type mix, price distribution	vendorCount, topVendors, productTypeCount, topProductTypes, priceMin, priceMedian, priceMax, priceOutlierCount
Curated collections	Merchandising label, matched wording, collection size	label, labelEvidence, curatedCollectionCount, totalCollectionCount, curationRatioPct, productsCount

🔗 Need more Shopify data?

If you need per-product rows instead of a store-level summary, Shopify Products Scraper By Collection & Keyword exports individual products by collection or search term. For merchant contact details and lead scoring, Shopify Merchant Scraper: Verified Contacts & Lead Rating is built for outreach lists. For ongoing stock, image, and product-update monitoring on a single store, see Shopify Scraper: Stock, Images & Product Updates.

⚙️ How does this differ from Shopify's official APIs?

Shopify's Admin API returns whatever a merchant's own app is authorized to see; Shopify Store Scraper: Product Catalogue Data returns a verified summary of whatever any public storefront chooses to publish, without needing that merchant's permission.

Feature	Shopify's official APIs	Shopify Store Scraper: Product Catalogue Data
Access requirement	Admin API needs a store owner's own app installed and OAuth-approved, with an access token scoped to that one store (shopify.dev, checked 2026-08-15)	Any public storefront address — no merchant permission, app install, or token
Scope	Only stores you own or have been granted app access to	Any public Shopify storefront you point it at
Query format	Storefront API is GraphQL-only — one query per store, maintained per store (shopify.dev, checked 2026-08-15)	Plain HTTP GET against the storefront's own public feed — add a URL to a list
Rate limits	Admin REST API is leaky-bucket limited: 2 req/s on Standard, 4 on Advanced, 20 on Plus, 40 on Enterprise (shopify.dev, checked 2026-08-15)	Governed by the Actor's own retry and backoff behavior, not a merchant-granted quota
Catalogue verification	Returns whatever the store's own records say, with no cross-check against a separately walked count	Walks the full product feed and reports both the walked and store-reported counts side by side
Merchandising classification	Not provided — collections return as raw records	Every collection classified against a 6-label merchandising taxonomy, with the matched wording as evidence

The Admin API is the right tool when you own the store and need to read or write data behind the scenes. Shopify Store Scraper: Product Catalogue Data is the right tool when you need a comparable, verified catalogue profile across storefronts you do not operate.

💡 Why do developers and teams scrape Shopify stores?

🤖 For AI engineers and agent builders

A retail-intelligence agent that needs to answer "how big is this store, and what does it actually sell" without ingesting a full product export can call this Actor per storefront and get vendorCount, topProductTypes, and priceMedian back as ready-to-index fields. Because the output is one row per store rather than one row per SKU, it fits directly into an agent's tool-call budget — a RAG pipeline can index thousands of storefronts without needing to chunk a product catalogue first.

📈 For competitor and pricing analysts

Point the Actor at a list of competing storefronts and compare priceMin/priceMedian/priceMax, priceOutlierCount, and curationLabelsPresent across all of them in one dataset. A competitor running heavy sale and gift_guide labelled collections during a promo window shows up in curationRatioPct and curationLabelsPresent without opening a single collection page by hand.

🔬 For market researchers and category analysts

Sizing a category across dozens or hundreds of Shopify stores means comparing catalogue depth, vendor concentration, and product-type breadth without pulling every SKU. vendorCount and topVendors distinguish a single-vendor DTC brand from a multi-vendor retailer in one field; verifiedProductCount and productCountDiscrepancy flag stores whose published numbers don't match reality.

🛠️ For developers building data products

Building a lead-scoring or store-monitoring product on top of Shopify data means a stable, typed schema you can re-run on a schedule. scraped_at and scrape_duration_sec on every row make it straightforward to track catalogue drift — verifiedProductCount, priceMedian, and curatedCollectionCount over time — without writing a parser against Shopify's HTML.

🚀 How to scrape Shopify stores with this Actor (step by step)

1. Open Shopify Store Scraper: Product Catalogue Data on its Apify Store listing, or find it from your Apify Console if you've already used it.
2. Add the storefront addresses to profile in 🛍️ Stores to profile (catalogTargets) — a bare domain or a full URL both work.
3. Configure the catalogue-composition and curation options you want (or leave the defaults, which build the full summary and detect all six curation labels).
4. Start the run from the Console, or trigger it programmatically with apify_client.
5. Download results from the dataset as JSON, CSV, Excel, or any other Apify export format — curated-collection rows also land in a separate <runId>-curated-collections dataset for the run.

🔧 What to do when Shopify changes its storefront structure

The Actor reads Shopify's standard public storefront routes (/products.json, /collections.json, /meta.json), which are common to Shopify's own theme architecture rather than any one store's custom template. The Actor is maintained and its output schema is kept stable, so integrations built against these field names keep working without changes on your end.

⬇️ Input

Required: startUrls. In practice, fill in catalogTargets instead — it takes priority whenever it has entries, and startUrls exists only so an input file built for a compatible URL-list field still runs unchanged.

Parameter	Required	Type	Description	Example Value
catalogTargets	No	array of strings	Storefront addresses to profile, one per line. A bare domain works as well as a full address. Leave empty to fall back to startUrls.	["allbirds.com", "https://www.allbirds.com"]
startUrls	Yes	array of strings	The original URL list, kept so an existing input file still runs unchanged. Used only when catalogTargets is empty. Default: ["https://www.allbirds.com"].	["https://www.allbirds.com"]
computeCatalogueSummary	No	boolean	Walks the store's whole product feed and summarizes it: verified product count, vendor mix, product-type mix, price distribution. Off, and vendorCount, topVendors, productTypeCount, topProductTypes, and every price column come back empty. Default: true.	true
topVendorsLimit	No	integer	How many entries topVendors contains. 0 lists every vendor the store carries; vendorCount is always complete regardless. Min 0, max 500. Default: 10. Requires computeCatalogueSummary on.	10
topProductTypesLimit	No	integer	How many entries topProductTypes contains, on the same terms as topVendorsLimit. Min 0, max 500. Default: 10. Requires computeCatalogueSummary on.	10
excludeUnavailableFromPricing	No	boolean	Computes the price range from variants a shopper can actually buy today. Excluded/implausible entries are counted in priceOutlierCount, never dropped silently. Default: true.	true
detectCuratedCollections	No	boolean	Classifies every collection against a merchandising taxonomy (best sellers, new arrivals, sale, staff picks, featured, gifts) using the merchant's own wording. Default: true.	true
curationLabels	No	array of enums	Restricts the taxonomy to the merchandising intents you care about: best_sellers, new_arrivals, sale, staff_picks, featured, gifts. Only selected labels can appear in curationLabelsPresent or on a curated-collection row. All six selected by default.	["sale"]
emitCuratedCollectionRows	No	boolean	Adds one labelled child row per matched collection, carrying title, handle, label, matched wording, product count, and URL. Off, and each store returns its store row only. Default: true.	true
proxyConfiguration	No	object	Optional connection settings for the run. Default: { "useApifyProxy": false }.	{ "useApifyProxy": false }

Common pitfall: leaving curationLabels empty does not disable curation — the Actor treats an empty selection as "all six," the same as the default. If you only want, say, sale collections labelled, you must select sale explicitly and leave the rest unchecked.

Example input

{
  "catalogTargets": [
    "https://www.allbirds.com",
    "blendjet.com",
    "https://nuts.com"
  ],
  "startUrls": ["https://www.allbirds.com"],
  "computeCatalogueSummary": true,
  "topVendorsLimit": 10,
  "topProductTypesLimit": 10,
  "excludeUnavailableFromPricing": true,
  "detectCuratedCollections": true,
  "curationLabels": ["best_sellers", "new_arrivals", "sale", "staff_picks", "featured", "gifts"],
  "emitCuratedCollectionRows": true,
  "proxyConfiguration": { "useApifyProxy": false }
}

⬆️ Output

Every run returns typed, normalized JSON rows to the default dataset, exportable as JSON, CSV, Excel, or any other Apify dataset format. The Console's default table view shows 29 of the fields below for readability; every field documented here is present on the underlying row regardless of which view you're looking at. Curated-collection rows are also mirrored to a separate <runId>-curated-collections dataset for the run — if that dataset can't be opened, every curated row still lands in the main dataset.

🏪 Store catalogue profile row (type: "store", isChild: false)

{
  "type": "store",
  "isChild": false,
  "store_domain": "www.allbirds.com",
  "store_url": "https://www.allbirds.com",
  "shop_name": "weareallbirds.myshopify.com",
  "store_name": "Allbirds",
  "storefrontStatus": "open",
  "country": "US",
  "currency": "USD",
  "locale": "en-US",
  "theme_name": "Symmetry",
  "theme_id": 132649861230,
  "theme_version": "9.2",
  "verifiedProductCount": 291,
  "metaReportedProductCount": 291,
  "productCountDiscrepancy": 0,
  "pagesWalked": 2,
  "vendorCount": 1,
  "topVendors": [{ "vendor": "Allbirds", "productCount": 291 }],
  "productTypeCount": 4,
  "topProductTypes": [
    { "productType": "Shoes", "productCount": 218 },
    { "productType": "Socks", "productCount": 56 },
    { "productType": "Apparel", "productCount": 16 },
    { "productType": "Underwear", "productCount": 1 }
  ],
  "priceMin": 14,
  "priceMax": 160,
  "priceMedian": 99,
  "priceP25": 75,
  "priceP75": 105,
  "priceOutlierCount": 0,
  "pricedVariantCount": 706,
  "totalCollectionCount": 1345,
  "curatedCollectionCount": 115,
  "curationRatioPct": 8.6,
  "curationLabelsPresent": ["best_sellers", "featured", "gifts", "new_arrivals", "sale", "staff_picks"],
  "collectionsPagesWalked": 6,
  "total_products": 291,
  "total_variants": 2829,
  "total_variants_available": 706,
  "inventory_availability_pct": 25.0,
  "collections": ["Best Sellers", "New Arrivals", "..."],
  "collections_count": 1345,
  "categories": ["Apparel", "Shoes", "Socks", "Underwear"],
  "vendors": ["Allbirds"],
  "price_min": 14,
  "price_avg": 91.86,
  "price_max": 160,
  "products_on_sale": 155,
  "discount_pct_avg": 44.3,
  "max_discount_pct": 75,
  "oldest_product_date": "2018-11-12",
  "newest_product_date": "2026-02-13",
  "products_with_images": 291,
  "avg_images_per_product": 4.3,
  "products_with_description": 288,
  "top_products": [
    { "title": "Women's Tree Runners", "url": "https://www.allbirds.com/products/womens-tree-runners", "price": 98.0, "createdAt": "2026-02-13T09:00:00-08:00" }
  ],
  "scraped_at": "2026-08-08T10:36:52+00:00",
  "scrape_duration_sec": 4.72
}

collections, categories, and vendors are complete deduplicated lists for the store (shown truncated above) — collections has one entry per collection title, categories/vendors are the sorted, deduplicated product_type/vendor strings across the whole walked catalogue. top_products is a fixed ten-item sample of the most recently added products; it is not controlled by any input and is not a per-product export.

⭐ Curated collection row (type: "curatedCollection", isChild: true)

{
  "type": "curatedCollection",
  "isChild": true,
  "store_domain": "www.partakefoods.com",
  "store_url": "https://www.partakefoods.com",
  "collectionId": 263407337570,
  "title": "Best Sellers",
  "handle": "best-sellers",
  "label": "best_sellers",
  "labelEvidence": "Best Seller",
  "labelSource": "title",
  "productsCount": 9,
  "collectionUrl": "https://www.partakefoods.com/collections/best-sellers",
  "publishedAt": "2022-01-12T17:15:13-05:00",
  "updatedAt": "2026-05-13T20:47:52-04:00",
  "currency": "USD",
  "scraped_at": "2026-08-08T10:38:06+00:00"
}

labelSource tells you whether the collection's title or handle matched the taxonomy; labelEvidence is the literal text fragment that justified the label, so every row is self-auditable.

Charging: the row_result event is charged once per store — for the store row itself — regardless of whether the storefront turned out open, password-protected, or unreachable, because the status is billable information either way. Curated-collection child rows never carry their own charge; they ride free on the store row that produced them. The only rows pushed without a charge are type: "status" rows, emitted only when no storefront address was supplied at all, or when an unexpected error interrupted a single store's request. To isolate billed rows in the dataset, filter on type == "store".

🎯 How does this compare to other Shopify scrapers?

Feature	Shopify Store Scraper: Product Catalogue Data	Generic alternative
Product count	Walks the full feed and reports both the walked and store-reported counts side by side	Usually trusts a single page or the store's self-reported count only
Price range	Computed from buyable variants by default, with excluded entries counted in priceOutlierCount, not silently dropped	Often mixes sold-out or placeholder prices into the range with no way to tell
Collection curation	Classifies every collection against a 6-label merchandising taxonomy, with the matched wording as evidence	Typically returns the raw collection list only, unlabelled
Output granularity	One row per store, one row per curated collection — a catalogue profile, not a product export	Usually one row per product, so answering a store-level question means exporting the whole catalogue
Closed storefronts	Reports storefrontStatus (open / password_protected / catalogue_unavailable / unreachable) and a null verifiedProductCount rather than guessing	Password-protected or blocked stores often come back as a misleading 0

If you're building an AI agent or RAG pipeline, the output-granularity row is the decision-maker — indexing one profile row per store, instead of chunking a full product export just to answer "how big is this store," is what keeps an agent's tool call cheap.

📏 How many stores and products can you scrape?

There is no hard cap on how many storefront addresses you can queue in catalogTargets/startUrls — runtime scales with the number of stores and each store's catalogue size, since stores are profiled one at a time, in the order supplied. ⚠️ Per store, the product-feed walk stops as a safety measure at 400 pages of 250 products (100,000 products) and the collection walk stops at 60 pages of 250 (15,000 collections); if a single store's catalogue is genuinely larger than that, verifiedProductCount/totalCollectionCount reflect what was walked up to the stop, not a confirmed total — pagesWalked reading 400 is the visible signal to check. topVendorsLimit and topProductTypesLimit independently cap how many entries land in topVendors/topProductTypes (max 500 each, or 0 for all), without affecting vendorCount/productTypeCount, which are always complete.

🔌 Integrate and automate your workflow

Shopify Store Scraper: Product Catalogue Data works with any language or tool that can send an HTTP request, through the Apify API.

🐍 REST API integration

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_API_TOKEN>")
run = client.actor("YOUR_USERNAME/shopify-store-scraper-product-catalogue-data").call(
    run_input={
        "catalogTargets": ["https://www.allbirds.com", "blendjet.com"],
        "computeCatalogueSummary": True,
        "detectCuratedCollections": True,
    }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["store_domain"], item["storefrontStatus"], item.get("verifiedProductCount"))

Works in Python, Node.js, Go, Ruby, and cURL — any client of the Apify API.

🔁 Automation platforms (n8n, Make)

This Actor runs like any other Apify Actor through Apify's official n8n integration (@apify/n8n-nodes-apify, documented at docs.apify.com/platform/integrations/n8n): add the "Run an Actor" operation, select this Actor by ID, and pass your catalogTargets list as the run input. Apify's Make integration works the same way — add the Actor to a Make scenario by ID to profile a fresh list of storefronts on a schedule.

⚖️ Is it legal to scrape Shopify stores?

Yes — Shopify Store Scraper: Product Catalogue Data reads only publicly available storefront data: product feeds, collection feeds, and store metadata that a Shopify store already serves to any visitor's browser. This is business and product data — store names, product counts, vendor names, prices, and collection titles — not personal data about identifiable individuals, so GDPR's personal-data provisions do not apply to what this Actor returns. What does apply is the storefront's own terms of service and any applicable database-rights protections over the compiled catalogue, which is why the Actor never authenticates, never accesses account-gated data, and reports closed or password-protected storefronts as such rather than working around them. Consult legal counsel for commercial use cases involving bulk data collection at scale.

❓ Frequently asked questions

Does Shopify Store Scraper: Product Catalogue Data work without a Shopify account?

Yes. It reads the same public /products.json, /collections.json, and /meta.json routes a storefront already serves to any visitor's browser — no Shopify account, app install, or API key is required for any store you profile.

How often is the scraped data updated?

Every run performs a live fetch against the storefront's current product and collection feeds — nothing is cached or replayed from a previous run. scraped_at on every row is the timestamp of that fetch.

What happens if a storefront is closed, unreachable, or has no verifiable catalogue?

The row still comes back, with storefrontStatus set to password_protected, unreachable, or catalogue_unavailable and verifiedProductCount: null — never a misleading 0. A null verified count means the count is unknown, not zero.

Can I scrape private or password-protected Shopify content?

No. The Actor only reads publicly accessible storefront routes. A password-protected storefront is detected and reported with storefrontStatus: "password_protected" rather than bypassed.

Is there a limit to how many storefronts I can profile in one run?

No hard cap on the list itself — add as many catalogTargets as you need. Runtime scales linearly with the number of stores and each store's catalogue size, since stores are processed one at a time.

Does Shopify Store Scraper: Product Catalogue Data work for AI agent workflows and LLM pipelines?

Yes. It's callable as an HTTP endpoint through the Apify API by any agent framework, and every response is typed JSON — no HTML parsing step before passing a store profile into an LLM context window or agent tool call.

How does it handle Shopify's anti-bot defences?

It sends browser-representative request headers on every fetch, retries up to 5 times with an increasing backoff ladder on throttled responses (HTTP 429/430/503), honors the storefront's Retry-After header when present, and can route requests through Apify Proxy if you enable it in proxyConfiguration. It reads the storefront's public JSON feeds directly, so no browser rendering is used or needed.

How does it compare to other Shopify scrapers?

It returns a store-level profile — verified catalogue counts, composition, and curated-collection classification — rather than a per-product export, and it reports closed or blocked storefronts explicitly instead of returning a misleading zero. See the comparison table above for the full breakdown.

Does it return data in a format LLMs can use directly?

Yes. Typed, normalized JSON with stable field names on every row — pass it directly into an LLM context window, index it into a vector store, or route it through an agent tool without a parsing step.

Can I use it without managing proxies?

Yes. Apify Proxy is optional and off by default (useApifyProxy: false) — the Actor runs against storefronts directly unless you turn proxy routing on in proxyConfiguration.

What happens when Shopify changes its structure or blocks the scraper?

The Actor is maintained and its output schema stays stable — field names and types don't change on your end. No numeric turnaround time is published for fixes.

💬 Your feedback

Found a bug, or a field that doesn't match what's documented here? Open an issue from this Actor's Issues tab in Apify Console, or message the API-Empire team directly through the Actor page — reports of a missing or mismatched field are what keep this README accurate.