Shopify Product Feed & Price Scraper 🛍️📈

Scrape public Shopify product feeds from compatible stores and export clean product, variant, price, availability, tag, image, and discount rows.

This Actor reads Shopify’s public /products.json catalog endpoint — no browser, login, proxy, or Shopify admin access required — and turns it into structured data you can export to JSON, CSV, Excel, Google Sheets, or consume through the Apify API.

It works especially well for competitor product catalog snapshots, Shopify price monitoring, discount tracking, new product drop discovery, and variant-level ecommerce research. Beauty, skincare, and hair-care stores are the default examples, but the Actor can scan any Shopify store that exposes the public product feed.

What you get

• Shopify product feed scraping — product title, vendor, product type, tags, product URL, image URL, and publish date.
• Variant-level rows — capture size, shade, bundle, SKU-like variant title, price, compare-at price, and availability at the variant level.
• Price and discount intelligence — detect real Shopify compare_at_price markdowns and compute discount percentage.
• New product and catalog snapshots — sort by newest to identify recently published products.
• Keyword filtering — scan only matching products such as serum, spf, bundle, dress, supplement, or any category terms.
• Opportunity scoring — rank in-stock markdowns and sale opportunities with a bounded dropOrSaleScore.
• Structured diagnostics — invalid, blocked, disabled, or no-match feeds return diagnostic rows instead of disappearing.
• Automation-ready output — schedule repeated runs and diff datasets for restocks, price changes, new drops, and assortment changes.

Best use cases

• Scrape a Shopify competitor’s public catalog into a spreadsheet.
• Monitor competitor prices, compare-at prices, and markdowns.
• Track new product drops by sorting products by publish date.
• Compare product assortment, tags, product types, vendors, and price bands across stores.
• Build daily or weekly Shopify catalog snapshots for alerts and BI workflows.
• Research beauty, skincare, hair-care, fashion, supplement, home goods, or other Shopify-powered ecommerce niches.

Buyer-ready offer

Send me 3-5 Shopify competitor stores. I’ll return a clean product/variant/price/discount spreadsheet with top markdowns and new-product signals. First sample is free.

Best buyers: Shopify agencies, DTC brands, ecommerce operators, retention/CRO teams, and analysts who need a repeatable competitor catalog feed.

Input example

{
  "storeUrls": [
    "https://mielleorganics.com",
    "https://baskandlatherco.com",
    "https://kaleidoscopehairproducts.com",
    "https://patternbeauty.com",
    "https://themanemagic.com"
  ],
  "keywords": ["shampoo", "conditioner", "oil", "serum", "leave-in", "curl", "edge", "mask"],
  "maxProductsPerStore": 100,
  "minDiscountPercent": 0,
  "includeVariants": true,
  "sortBy": "opportunity",
  "includeDiagnostics": true
}

Field	What it controls
storeUrls required	Shopify store URLs to scan. Bare domains are accepted. Paths are normalized to the store origin.
keywords	Filter products by title, vendor, product type, and tags. Empty = keep all products.
maxProductsPerStore	Product cap per store before variant expansion.
minDiscountPercent	Minimum markdown percent to return. Use 0 for all matches.
includeVariants	true = one row per variant; false = one row per product using its best-deal variant.
sortBy	opportunity, discount, price, or newest.
includeDiagnostics	Include invalid, blocked, failed, or no-match rows for auditability.

Sample output

Real top row from the sample input:

{
  "rowType": "product",
  "storeUrl": "https://mielleorganics.com",
  "productId": 6752188989553,
  "title": "Mango & Tulsi Nourishing Conditioner",
  "vendor": "MIELLE",
  "productType": "Conditioners",
  "productUrl": "https://mielleorganics.com/products/mango-tulsi-nourishing-conditioner",
  "imageUrl": "https://cdn.shopify.com/s/files/1/0763/8199/files/250513-Mielle_SHA-Mielle-MT-Conditioner-3.jpg?v=1747750810",
  "publishedAt": "2022-09-02T17:31:07-05:00",
  "variantId": 39844163518577,
  "variantTitle": "Default Title",
  "price": 5,
  "compareAtPrice": 14.99,
  "discountPercent": 67,
  "available": true,
  "matchedKeywords": ["conditioner"],
  "dropOrSaleScore": 72,
  "scrapedAt": "2026-07-04T02:50:20.087Z"
}

Example run result from the sample input:

Store	Result
mielleorganics.com	53 rows
baskandlatherco.com	38 rows
kaleidoscopehairproducts.com	41 rows
patternbeauty.com	59 rows
themanemagic.com	1 diagnostic row
Total	192 dataset rows

The top finding was a 67% off in-stock Mielle conditioner with a dropOrSaleScore of 72.

Output fields

Field	Meaning
rowType	product or diagnostic.
storeUrl	Normalized Shopify store origin.
productId, title, vendor, productType, tags	Shopify product metadata.
productUrl, imageUrl	Product page and primary image.
publishedAt	Shopify publish timestamp.
variantId, variantTitle	Specific variant scanned.
price, compareAtPrice	Current and compare-at prices parsed to numbers.
discountPercent	Rounded markdown percent when compareAtPrice > price.
available	Shopify variant availability boolean.
matchedKeywords	Which configured keywords matched.
dropOrSaleScore	0–100 opportunity score; deeper, in-stock markdowns rank higher.
scrapedAt	Run timestamp.

Diagnostic rows

When includeDiagnostics is enabled, stores that fail or return no usable feed produce rows like:

{
  "rowType": "diagnostic",
  "storeUrl": "https://themanemagic.com",
  "statusCode": 200,
  "errorType": "parse_error",
  "message": "Could not parse products.json: Unexpected token '<' ... is not valid JSON",
  "scrapedAt": "2026-07-04T02:50:20.087Z"
}

Possible errorType values: invalid_url, http_error, network_error, parse_error, invalid_response, no_matching_rows.

How it works

This Actor queries each store’s public Shopify product feed at:

/products.json

It paginates the feed, normalizes products and variants, applies optional keyword and discount filters, sorts rows, and writes product or diagnostic rows to the default dataset.

It does not log in, scrape private data, bypass paywalls, or access hidden Shopify admin data.

Pricing

This Actor is configured for Pay per event pricing:

• apify-actor-start: $0.00005 per run start.
• product-row: $0.001 per real product / variant row.
• Diagnostic rows are emitted for transparency but are not charged as value rows.

Displayed price: from $1.00 / 1,000 product or variant rows.

Example: a 1,263-row competitor scan costs about $1.263 in Actor event charges, plus/minus Apify platform usage depending on the user’s plan/settings.

Limitations

• Works only with stores exposing Shopify’s public /products.json endpoint.
• Some stores disable, block, or alter the public product feed.
• Detects markdowns from compare_at_price; coupon/cart-only discounts are not visible.
• Inventory is a boolean (available), not exact stock count.
• Region-specific pricing and hidden/draft products are not included.
• A run is a snapshot; schedule repeated runs and compare datasets for true change tracking.
• Not affiliated with Shopify or any listed brand. Use responsibly and respect each site’s terms.

Ecommerce Data Toolkit

This Actor is part of a focused ecommerce / retail-intelligence toolkit:

Need	Actor
Shopify product feeds, variants, prices, discounts	Shopify Product Feed & Price Scraper
Brand retail footprint and where-to-buy / store-locator rows	Brand Store Locator & Stockist Scraper
Salon, stylist, and certified-pro locator rows	Beauty Salon & Pro Locator Scraper

Use the three together to map what brands sell, where products are stocked, and which salons/pros carry or certify around professional lines.

FAQ

Does this work on every Shopify store?

No. It works when the store exposes the public /products.json endpoint. Some stores disable or alter that endpoint.

Can I monitor changes over time?

Yes. Schedule this Actor daily or hourly, export datasets, and compare new runs against previous runs.

Does it require proxies?

No proxy is required for the default public-feed approach.

Can I export to CSV or Excel?

Yes. Apify datasets can be exported to JSON, CSV, Excel, XML, RSS, or consumed through API integrations.

Quality / verification

This project now ships with a reusable sample input and standard local checks:

npm run check
cp examples/sample-input.json storage/key_value_stores/default/INPUT.json
CRAWLEE_STORAGE_DIR=./storage APIFY_LOCAL_STORAGE_DIR=./storage node src/main.js

The buyer demo package lives at:

/opt/data/apify-projects/outreach/shopify-product-feed-demo.zip

API usage

Run from the Apify CLI:

$apify call fierce_quoll/shopify-beauty-radar --input-file examples/live-input.json

Or use the API tab on the Actor page for language-specific snippets.

Local development

npm install
npm test
/opt/data/bin/apify validate-schema .actor/input_schema.json
mkdir -p storage/key_value_stores/default
cp examples/live-input.json storage/key_value_stores/default/INPUT.json
CRAWLEE_STORAGE_DIR=./storage APIFY_LOCAL_STORAGE_DIR=./storage node src/main.js