Shopify Store Products Scraper (products.json, prices, monitor)

Point it at any public Shopify store — allbirds.com, brand.myshopify.com, a collection URL — and get the whole catalogue as flat rows: every product with its variants, prices, compare-at prices, stock flags, images, options, tags and vendor. The data comes from the storefront feed the shop publishes for its own theme, so it is the shop's own catalogue, not a rendering of it. No app to install, no Shopify API key, no login.

What you get

One row per product: id (<host>:<product id>), productId, url, storeUrl, storeHost, title, handle, vendor, productType, tags, description (plain text, up to 1 000 characters), price (cheapest variant), priceMax, compareAtPrice, currency, available, variantCount, variants (id, title, sku, price, compareAtPrice, available, grams), imageUrl, imageCount, options, createdAt, updatedAt, publishedAt, collectionHandle, query, scrapedAt.

Example output

A real row (storeUrls: ["https://www.allbirds.com"]), shortened:

{
  "id": "www.allbirds.com:7218356060240",
  "productId": "7218356060240",
  "url": "https://www.allbirds.com/products/mens-strider-explore",
  "storeUrl": "https://www.allbirds.com",
  "storeHost": "www.allbirds.com",
  "title": "Men's Strider Explore - Natural Black (Dark Grey Sole)",
  "handle": "mens-strider-explore",
  "vendor": "Allbirds",
  "productType": "Shoes",
  "description": "From exploring urban landscapes to tackling the trails (or both), this active shoe is here for it all...",
  "price": 130.0,
  "priceMax": 130.0,
  "compareAtPrice": null,
  "currency": "USD",
  "available": false,
  "variantCount": 13,
  "variants": [
    {"id": "41334293889104", "title": "8", "sku": "A11768M080", "price": 130.0, "compareAtPrice": null,
     "available": false, "grams": 919}
  ],
  "imageUrl": "https://cdn.shopify.com/s/files/1/1104/4168/files/...",
  "imageCount": 5,
  "options": ["Size"],
  "publishedAt": "2026-08-26T10:22:14-07:00",
  "updatedAt": "2026-09-16T15:53:47-07:00"
}

Use cases

• Competitor catalogue and price tracking — pull a rival's full assortment with per-variant prices and compare-at prices, then run it on a schedule in monitor mode and get told the moment something changes.
• Assortment research — which vendors, product types and tags a store carries, how deep each product's variant tree goes, how much of the catalogue is actually in stock.
• Dropshipping and reselling — build a product list with SKUs, weights and images from the supplier's own store instead of a stale spreadsheet.
• Stock alerts — availableOnly plus monitor mode tells you when a sold-out product comes back.
• Collection snapshots — scrape only new-arrivals or sale with collectionHandles.

How it works

1. Each store's robots.txt is read first, at runtime, before any catalogue request. If the store's own User-agent: * group forbids /, /products or /products.json, the store is skipped and nothing is scraped or charged for it. A collection path a store forbids separately is skipped on its own — some large stores publish Disallow: /collections/*/products* while leaving the whole-catalogue feed open.
2. The store URL is normalised: example.com, https://example.com/, brand.myshopify.com and a collection URL such as https://example.com/collections/sale all resolve to the same store (the collection handle is picked up from the URL).
3. The catalogue is read page by page (?limit=250&page=N). Paging stops on a short page or as soon as a page brings no product id that was not already seen — never on "the page was empty", because a store that redirects past the end of its catalogue serves the last page again instead.
4. The store home page is read once per store for Shopify.currency, so rows carry the shop's own currency. A theme that does not publish it leaves currency as null — the actor does not guess a currency from the domain or the exit country.
5. availableOnly, minPrice and maxPrice are applied before a row is pushed, so you are not charged for rows you filtered out.

Input

Field	Meaning	Default
storeUrls	Stores to scrape, one per line. A bare domain, a full URL or a collection URL	required
collectionHandles	Scrape only these collections, applied to every store	empty (whole catalogue)
maxProductsPerStore	Stop after this many products per store	250
availableOnly	Keep only products with at least one variant in stock	false
includeVariants	Include the per-variant array (variantCount and prices are kept either way)	true
minPrice / maxPrice	Keep only products whose cheapest variant is inside this range	empty
tier	Proxy tier: auto climbs none → datacenter → residential only when a store refuses	auto
proxyCountry	Exit country for the proxied tiers	US
maxItems	Stop after this many rows in total	500
mode	scrape or monitor (only new/changed since the last run)	scrape
monitorStateId, webhookUrl, telegramBotToken, telegramChatId	Monitor-mode state key and alert targets	empty

Monitor mode

mode: monitor compares this run against the previous run of the same task and returns only what is new or changed. A product counts as changed when its price, stock flag, variant count or updatedAt moves, so you get price drops, restocks, sell-outs and new products — and nothing else. Alerts can go to a webhook and to Telegram.

Pricing

Event	Price
result	$0.0005 per product ($0.50 per 1,000)
monitor-check	$0.005 per monitor run
change	$0.0005 per new or changed product

You are charged only for rows actually pushed. Most stores need no proxy at all, so platform usage stays small.

Limits

• Public storefronts only. A store whose front end is behind the Shopify password page answers with HTTP 401; it is reported as password_protected and produces no rows. There is no way in.
• The store decides. A store whose robots.txt forbids the feed is skipped. This is checked per store at runtime and cannot be turned off.
• Some stores block the endpoint at their CDN. Gymshark, for example, answers /products.json with a CloudFront 403 on every proxy tier. Such a run is reported as blocked, never as "no results".
• Not every host is a Shopify store. A host with no product feed is reported as not_shopify rather than silently returning nothing.
• currency is null when the store's theme does not publish Shopify.currency on its home page. Prices are always the numbers the store's own feed returns.
• The feed carries the shop's published catalogue: products hidden from the online-store sales channel, draft products and per-customer prices are not in it, and neither are inventory counts (Shopify publishes a boolean available per variant, not a quantity).
• description is the product description as plain text, capped at 1 000 characters.
• Shopify caps a page at 250 products; larger catalogues are paged automatically.

What is NOT collected

No personal data of any kind. The storefront feed is a product catalogue — it contains no customer, order, review-author or contact object, and the output schema has no field for one. No e-mail addresses, no names, no buyer data. Stores are businesses, and only business/catalogue facts are returned.

FAQ

Does it need a Shopify API key or a private app? No. Every request is a plain public web request to the store's own storefront feed.

How do I know a URL is a Shopify store? Just give it — a host that serves no product feed comes back as not_shopify in the run summary, and costs one request.

Can I scrape only one collection? Yes: put its handle in collectionHandles, or paste the collection URL straight into storeUrls.

What happens when a run returns nothing? No rows are pushed and no result events are charged. The RUN_SUMMARY record in the run's key-value store carries emptyReason, and ADAPTER_STATS carries a per-store report (ok, robots_disallowed, not_shopify, password_protected, blocked) so you can tell an empty catalogue from a refusal.

Changelog

• 0.1: initial release — whole-catalogue and per-collection scraping, variants with SKU and compare-at prices, currency from the store's own page, per-store runtime robots.txt check, price and availability filters, monitor mode.

---

If this actor saved you time, a short review on its Store page genuinely helps other people find it. Found a bug or need a field that is missing? Open a ticket on the Issues tab.