Shopify Store Scraper

Shopify Store Scraper

The Shopify Store Scraper is a production-ready Apify actor that extracts structured data from publicly accessible Shopify stores — including products, variants, prices, collections, themes, installed apps, and optional Yotpo reviews — so you can scrape Shopify store products at scale with clean, export-ready data. It solves the pain of manual catalog research by automating Shopify product data extraction for marketers, developers, data analysts, and researchers. With smart proxy fallback, concurrent fetching, and robust error handling, this Shopify store crawler powers market research, competitor tracking, and catalog exports with confidence.

What data / output can you get?

Below are examples of real output fields the actor saves to the Apify dataset (CSV/JSON exports supported):

Data field	Description	Example value
store_domain	Store domain name without protocol	"www.allbirds.com"
store_url	Full store URL	"https://www.allbirds.com"
shop_name	Shopify shop domain inferred from meta	"shop11044168.myshopify.com"
currency	Store currency code from meta	"USD"
locale	Store locale (from meta or HTML)	"en-US"
theme_name	Active theme name parsed from HTML	"rc-cm-2025-12-26_20-35 Update"
theme_id	Active theme ID (if detected)	128596836432
apps_installed	Detected apps from homepage HTML	["Elevar Conversion Tracking","JsEncrypt"]
total_products	Count of products fetched via /products.json	681
total_variants	Count of variants across all products	7276
price_min	Minimum variant price	2.0
price_avg	Average variant price (rounded)	79.34
price_max	Maximum variant price	500.0
total_variants_available	Variants in stock (available=true)	5647
inventory_availability_pct	% of variants in stock	77.6
collections_count	Number of collections	250
top_products	Newest 10 products with basic info	[{"title":"Allbirds Laces - Warm Red - Round","url":"https://www.allbirds.com/products/allbirds-laces-warm-red-round","price":8.0,"reviews":0,"rating":0.0}]

Bonus fields include apps_by_category, categories, vendors, products_on_sale, products_with_images, avg_images_per_product, products_with_description, estimated_launch_date, oldest_product_date, newest_product_date, scrape_duration_sec, api_blocked, and (when enabled) detailed_reviews with Yotpo review details. Data is saved to the Apify dataset for easy export to JSON or CSV.

Key features

• 🔄 Smart proxy fallback
  • Automatically attempts no proxy → datacenter proxy → residential proxy with up to 3 retries on 403/429/503, and sticks to residential once selected for reliability on tougher stores.
• ⚡ Concurrent, async fetching
  • Async/await architecture with polite delays accelerates catalog traversal while keeping requests respectful.
• 📊 Comprehensive Shopify product extractor
  • Gathers products, variants, prices, images, descriptions, availability, collections, vendors, categories, and more — a complete Shopify product scraper for structured catalog intelligence.
• 🧩 Theme & app detection
  • Extracts theme_name, theme_id, theme_version, and detects installed apps (e.g., Elevar Conversion Tracking, JsEncrypt) with apps_by_category and apps_count.
• ⭐ Optional Yotpo review extraction
  • When extractReviews is true, fetches rating aggregates and review details for the top 10 newest products using Yotpo’s public widget API.
• 💾 Live dataset saving
  • Writes each store’s results to the Apify dataset as it completes, so you never lose progress mid-run.
• 🛡️ Robust error handling
  • Continues processing remaining stores and records error entries with context, keeping your pipeline resilient.
• 🧑‍💻 Developer friendly (Python & Apify SDK)
  • Built with the apify Python SDK and aiohttp for easy customization and pipeline integration — ideal for Shopify scraper Python workflows.

How to use Shopify Store Scraper - step by step

1. Create or log in to your Apify account and open the Apify Console.
2. Find the “shopify-store-scraper” actor and open it.
3. Add input:
   • startUrls: Provide one or more Shopify store URLs (string list like "https://www.allbirds.com"). The actor also accepts items as objects with a url field.
   • extractReviews: Toggle to enable Yotpo review extraction for the top 10 newest products (default: true).
   • proxyConfiguration: Optionally enable Apify Proxy. If enabled and blocks occur, the actor falls back to residential automatically.
4. Start the run and monitor logs:
   • You’ll see progress logs as products and collections are fetched, plus any proxy fallback events.
5. Wait for completion:
   • Each store’s JSON record is saved to the run’s default dataset as soon as it’s finished.
6. Export your data:
   • Go to the OUTPUT tab and download results from the dataset as JSON or CSV, or access via the Apify API.
7. Pro Tip: Chain this Shopify store scraper tool into automations and analytics pipelines using the Apify API for scheduled runs and downstream processing.

Use cases

Use case name	Description
Market research + catalog intelligence	Analyze categories, vendors, and product volumes across many stores to size markets and spot trends using a Shopify store data extractor.
Competitor monitoring + price tracking	Track price_min/avg/max and inventory_availability_pct to benchmark pricing and stock with a Shopify price scraper.
Product sourcing + catalog building	Export structured products and variants to build enrichment-ready catalogs and download products from Shopify store data.
Theme and app landscape analysis	Gather theme_name/theme_version and apps_installed/apps_by_category to map technology usage across markets.
Review insights for product validation	Enable extractReviews to aggregate Yotpo ratings and review details on top/new products.
Developer pipelines (API/ETL)	Integrate the Shopify product extractor into Python/SDK pipelines for automated data collection and enrichment workflows.

Why choose Shopify Store Scraper?

This actor is engineered for precision, automation, and reliability — a purpose-built Shopify store scraper tool that outperforms brittle, browser-based alternatives.

• 🎯 Accurate, structured output: Parses Shopify public JSON endpoints and HTML for clean, schema-consistent results.
• 🚀 Scales across multiple stores: Processes a list of startUrls sequentially with per-store isolation and live dataset writes.
• 🧠 Smart proxy management: No proxy → datacenter → residential with retries; sticks to residential for persistent blocks.
• 🛠️ Built with Python & Apify SDK: Ideal for developers integrating a Shopify scraper Python workflow.
• 🔒 Public data only: Targets Shopify’s public JSON APIs (/products.json, /collections.json, /meta.json) and Yotpo’s widget API for reviews.
• 💸 Export-ready data: Results land in the Apify dataset for easy JSON/CSV export and downstream analytics.
• 🧰 Reliable and production-ready: Robust error handling ensures partial failures don’t stop your end-to-end job.

In short, it’s a stable Shopify product scraper you can trust for ongoing research, monitoring, and enrichment — not a fragile extension.

Is it legal / ethical to use Shopify Store Scraper?

Yes — when done responsibly. This actor collects data from publicly available Shopify endpoints (/products.json, /collections.json, /meta.json) and the public Yotpo widget API. It does not access private or authenticated content.

Guidelines for compliant use:

• Only scrape publicly available data; avoid password-protected areas.
• Ensure your use complies with applicable laws and regulations (e.g., GDPR, CCPA).
• Review the target site’s terms of service and respect rate limits.
• Consult your legal team for edge cases or specific jurisdictions.

Input parameters & output format

Example JSON input

{
  "startUrls": [
    "https://www.allbirds.com",
    "https://example-store.myshopify.com"
  ],
  "extractReviews": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Parameters

• startUrls (array, required): List one or more Shopify store URLs (e.g., https://www.allbirds.com). Editor: stringList. Required: Yes. Default: not specified.
• extractReviews (boolean, optional): Enable to extract detailed product reviews from Yotpo (slower but more comprehensive). Required: No. Default: true.
• proxyConfiguration (object, optional): Choose which proxies to use. If Shopify rejects the proxy, a residential proxy will be used as a fallback. Required: No. Default: not specified (prefill uses { "useApifyProxy": false }).

Example JSON output (successful store item)

{
  "store_domain": "www.allbirds.com",
  "store_url": "https://www.allbirds.com",
  "shop_name": "shop11044168.myshopify.com",
  "country": "US",
  "currency": "USD",
  "locale": "en-US",
  "theme_name": "rc-cm-2025-12-26_20-35 Update",
  "theme_id": 128596836432,
  "theme_version": "1.131.2",
  "apps_installed": ["Elevar Conversion Tracking", "JsEncrypt"],
  "apps_by_category": {
    "Analytics": ["Elevar Conversion Tracking"],
    "Other": ["JsEncrypt"]
  },
  "apps_count": 2,
  "total_products": 681,
  "total_variants": 7276,
  "collections": ["Accessories", "Shoes", "Apparel"],
  "collections_count": 250,
  "categories": ["Accessories", "Apparel", "Shoes", "Socks"],
  "vendors": ["Allbirds"],
  "price_min": 2.0,
  "price_avg": 79.34,
  "price_max": 500.0,
  "products_on_sale": 681,
  "discount_pct_avg": null,
  "products_with_reviews": 0,
  "review_coverage_pct": 0,
  "avg_reviews_per_product": null,
  "avg_rating": null,
  "total_estimated_sales": null,
  "avg_sales_per_product": null,
  "total_variants_available": 5647,
  "inventory_availability_pct": 77.6,
  "estimated_launch_date": "2018-11-12",
  "oldest_product_date": "2018-11-12",
  "newest_product_date": "2025-11-06",
  "products_with_images": 680,
  "avg_images_per_product": 4.5,
  "products_with_description": 681,
  "top_products": [
    {
      "title": "Allbirds Laces - Warm Red - Round",
      "url": "https://www.allbirds.com/products/allbirds-laces-warm-red-round",
      "price": 8.0,
      "reviews": 0,
      "rating": 0.0
    }
  ],
  "detailed_reviews": [
    {
      "product_url": "https://www.allbirds.com/products/allbirds-laces-warm-red-round",
      "product_id": "7242343809104",
      "reviews_count": 0,
      "avg_rating": 0.0,
      "reviews": []
    }
  ],
  "scraped_at": "2025-12-29T07:34:23.715157Z",
  "scrape_duration_sec": 15.71,
  "api_blocked": false
}

Error outputs (when applicable)

• If no valid startUrls are provided:

{
  "error": "No valid URLs found in startUrls. Please provide at least one Shopify store URL in the input.",
  "input_received": {
    "startUrls": []
  },
  "scraped_at": "2025-12-29T07:34:23.715157Z"
}

• If a specific store fails to scrape:

{
  "store_url": "https://example-store.com",
  "error": "Failed to fetch homepage from https://example-store.com. All request attempts failed.",
  "scraped_at": "2025-12-29T07:34:23.715157Z"
}

Notes

• Fields like theme_id, price_min/avg/max, and estimated dates may be null if not detected or if the store has no products.
• detailed_reviews appears only when extractReviews is true and review data is successfully fetched.

FAQ

Do I need a login or cookies to use this?

No. The actor targets publicly available Shopify endpoints (/products.json, /collections.json, /meta.json) and the public Yotpo widget API, so no login is required.

Can it scrape multiple stores in one run?

Yes. Provide multiple URLs in startUrls. The actor processes them sequentially, saving each store’s result to the dataset as it completes.

How does the proxy fallback work?

It first tries direct requests. On 403/429/503 and when a proxy is configured, it switches to a datacenter proxy, then to a residential proxy with up to 3 retries. Once residential is selected, it sticks with it for the rest of the run.

Which products get reviews extracted?

When extractReviews is true, the actor takes the 10 newest products and retrieves Yotpo review aggregates and details. It requests up to 3 pages of reviews with 10 reviews per page.

What formats can I export to?

Results are stored in the Apify dataset, which supports export to JSON and CSV via the console or Apify API.

What happens if a store blocks my requests?

The actor will automatically attempt proxy fallback. If all attempts fail for a store, it records an error entry with store_url, error, and scraped_at so you have full visibility.

Is this a Shopify store scraper Chrome extension?

No. This is a server-side Apify actor built with Python and aiohttp. It’s designed for reliability and scale compared to browser extensions.

Does this work for price and inventory monitoring?

Yes. It calculates price_min, price_avg, price_max and inventory_availability_pct, making it suitable as a Shopify price scraper and inventory scraper for ongoing monitoring.

Closing CTA / Final thoughts

Shopify Store Scraper is built for structured, reliable Shopify product data extraction at scale. With smart proxy fallback, comprehensive catalog coverage, and optional Yotpo review enrichment, it helps marketers, analysts, researchers, and developers export products from Shopify stores with confidence. Run it in Apify Console, export JSON/CSV from the dataset, or integrate via the Apify API to automate your workflows. Start extracting smarter, cleaner Shopify store data today.