Shopify Scraper — Store Products, Prices & Catalog Intelligence

Scrape any Shopify store's full product catalog straight from its own public products.json endpoint. Get every product with variants, SKUs, prices, real discounts and stock status — plus a store-intelligence rollup that tells you how a competitor is priced, how much of the range is actually on sale, how much is out of stock, and how fast they ship new product.

No API key. No login. No browser. Works on any Shopify store that leaves its storefront JSON public.

Keywords: shopify scraper, shopify store products, shopify product scraper, competitor price tracking, ecommerce price monitoring, shopify catalog export, product feed, dropshipping research.

---

Three things this actor gets right

1. It only counts discounts that are real. Shopify merchants routinely leave compare_at_price populated on items that are not on sale. In the catalogs this actor was built against, one store had 603 variants with compare_at_price set and not a single one higher than the actual price — 570 were lower. A scraper that treats "compare-at is present" as a discount reports that store as heavily discounted while nothing is on sale. This actor only counts a discount when compare-at is strictly greater than price.

2. It tells you when it only saw part of the catalog. Every rollup carries catalogComplete and pagesFetched. If a store rate-limits mid-walk, or you capped the run, the percentages describe a slice — and the row says so instead of quietly presenting slice statistics as store statistics. filtersApplied likewise flags a rollup computed over filtered rows rather than the whole store.

3. It reads each store's robots.txt first, and gets the parsing right. Most Shopify robots.txt files contain a literal Disallow: / — scoped to User-agent: Nutch. A naive check would skip nearly every store on the platform; a missing check would crawl stores that genuinely opt out. This actor resolves the correct user-agent group and applies longest-match precedence, then skips stores that disallow the endpoint (shop.bulletproof.com is a real example).

---

Input

Everything is optional — running with {} scrapes a demo store.

Field	Type	Default	Description
storeDomains	array	["allbirds.com"]	Store domains. Scheme/path stripped automatically.
outputMode	enum	products	products, storeSummary, or both.
maxProductsPerStore	int	20	Cap on product rows per store.
keyword	string	""	Match against title, description and tags.
productType	string	""	Exact match on Shopify product_type.
vendor	string	""	Exact match on Shopify vendor.
minPrice / maxPrice	int	0	Price band in the store's own currency. 0 = unbounded.
onlyDiscounted	bool	false	Keep only genuinely discounted products.
onlyInStock	bool	false	Drop products with no available variant.
publishedAfter	string	""	YYYY-MM-DD; find new arrivals.
respectRobots	bool	true	Honour each store's robots.txt.
requestDelayMs	int	500	Delay between catalog pages.
useApifyProxy	bool	false	Rotating residential IPs (rarely needed).

{
  "storeDomains": ["www.deathwishcoffee.com"],
  "outputMode": "both",
  "onlyDiscounted": true,
  "maxProductsPerStore": 50
}

Output

Product row — flat, stable keys, safe for agents and spreadsheets:

{
  "storeDomain": "www.deathwishcoffee.com",
  "storeName": "Death Wish Coffee",
  "productId": 7218414092343,
  "handle": "rebellious-by-nature-trucker",
  "title": "Rebellious By Nature Trucker",
  "vendor": "Sportiqe",
  "productType": "Accessories",
  "tags": ["Accessories", "Apparel", "Gifting", "Merch", "Website Exclusive Sale"],
  "currency": "USD",
  "minPrice": 17.5, "maxPrice": 17.5,
  "maxDiscountPercent": 30, "isDiscounted": true,
  "availability": "in_stock",
  "variantCount": 1, "inStockVariantCount": 1,
  "skus": ["UHCLBD25"],
  "imageCount": 5,
  "createdAt": "2025-01-23",
  "publishedAt": "2026-06-15", "daysSincePublished": 45,
  "productUrl": "https://www.deathwishcoffee.com/products/rebellious-by-nature-trucker"
}

Store summary row — one per store:

{
  "storeDomain": "www.deathwishcoffee.com",
  "storeName": "Death Wish Coffee", "country": "US", "currency": "USD",
  "myshopifyDomain": "deathwishcoffee.myshopify.com",
  "catalogComplete": true, "filtersApplied": false, "pagesFetched": 1,
  "productCount": 141, "variantCount": 404,
  "vendorCount": 15, "productTypeCount": 10,
  "medianPrice": 29.98,
  "discountedProductPercent": 15.6,
  "averageDiscountPercent": 34.5,
  "outOfStockVariantPercent": 11.6,
  "newProducts30d": 18,
  "topVendors": [
    { "name": "Death Wish Coffee Company", "count": 36 },
    { "name": "Sportiqe", "count": 27 }
  ],
  "topProductTypes": [
    { "name": "Apparel", "count": 39 },
    { "name": "Coffee", "count": 27 }
  ]
}

(Both examples are verbatim from live runs on 2026-07-31, trimmed for length.)

What you can do with it

• Competitor price monitoring — schedule daily, watch medianPrice and discountedProductPercent move.
• Promo detection — onlyDiscounted plus a schedule tells you the moment a rival runs a sale, and how deep.
• New-arrival tracking — publishedAfter or newProducts30d shows how fast a brand ships product.
• Stock-out intelligence — outOfStockVariantPercent is a proxy for demand and supply trouble.
• Catalog export / migration — full product and variant dump with SKUs.

Data coverage and limits — read this

• Only stores that leave products.json public. Most do; some disable it or sit behind a bot wall. Those return a clear error on the summary row (HTTP 403, HTTP 404, or not a Shopify store) rather than failing the run.
• Not every field exists. products.json exposes no inventory quantity (only available yes/no), no cost, and no per-product sales data. Currency and store identity come from /meta.json; if a store blocks that, currency is null.
• Prices are the storefront's default-market prices. Shopify Markets stores may show different prices to shoppers in other countries.
• Rate limits are real. The default 500 ms delay keeps large catalog walks healthy; raise requestDelayMs for stores that still return 429.
• Company-level data only. Product catalogs are business data. This actor requests no customer, reviewer or account data and never touches checkout, cart or account endpoints.

Legal

Reads only robots.txt, meta.json and products.json — the same public storefront endpoints a Shopify theme calls, shipped by default on every Shopify store. The actor fetches and honours each store's robots.txt before reading anything, identifies itself honestly in its User-Agent, and skips any store that disallows the endpoint or blanket-bans AI crawlers by name.