Shopify Store Scraper

Shopify Store Scraper

The Shopify Store Scraper is a fast, reliable Shopify web scraping tool that extracts structured store-level and product-level data from publicly accessible Shopify stores. It solves the tedious work of gathering catalogs, prices, variants, inventory, collections, app usage, and optional Yotpo reviews into clean JSON/CSV for analysis. Built for marketers, developers, data analysts, and researchers, this Shopify product scraper and Shopify store data extractor enables scalable competitive analysis, price monitoring, and SEO intelligence across any number of stores. 🚀

What data / output can you get?

Below are examples of structured fields the scraper saves to the Apify dataset. Exports are available as JSON or CSV.

Data field	Description	Example value
store_domain	Store domain without protocol	www.allbirds.com
store_url	Full store URL	https://www.allbirds.com
shop_name	Shopify shop name	shop11044168.myshopify.com
country	Store country code (from meta.json)	US
currency	Store currency code (from meta.json)	USD
locale	Store locale (from meta.json or HTML)	en-US
theme_name	Active theme name parsed from HTML	rc-cm-2025-12-26_20-35 Update
theme_id	Active theme ID parsed from HTML	128596836432
apps_installed	Detected apps from HTML (e.g., Elevar, JsEncrypt)	["Elevar Conversion Tracking","JsEncrypt"]
total_products	Count of all products via /products.json pagination	681
total_variants	Count of all variants across products	7276
price_min / price_avg / price_max	Min/avg/max prices across variants	2.0 / 79.34 / 500.0
inventory_availability_pct	Percentage of variants in stock	77.6
collections_count	Number of collections via /collections.json	250
top_products	Top 10 newest products with basic info (+ optional reviews)	[{"title":"Allbirds Laces - Warm Red - Round","url":"https://www.allbirds.com/products/allbirds-laces-warm-red-round","price":8.0,"reviews":0,"rating":0.0}]

Bonus output: When review extraction is enabled, a detailed_reviews array is included with per-product review stats and items from Yotpo. You can download results from the Apify dataset as JSON or CSV and integrate via the Apify API.

Key features

• 🔄 Smart proxy fallback & retries: Automatically moves from direct → datacenter → residential proxies with retry logic to keep scraping moving even when facing 403/429/503 blocks.
• 📊 Comprehensive data extraction: Collects store metadata, collections, full product catalogs, variants, prices, images, and availability to power Shopify competitor analysis scraper workflows.
• ⭐ Optional Yotpo review scraping: Toggle extractReviews to enrich top/newest products with review counts, average ratings, and review details for deeper product insights.
• 📈 Pricing & inventory analytics: Outputs min/avg/max prices, total/available variants, and inventory availability percentage for Shopify price monitoring scraper and inventory tracking use cases.
• 🧩 App detection: Identifies installed apps like Elevar Conversion Tracking and JsEncrypt from the store HTML for app discovery and tech-stack research.
• 💾 Live dataset saving: Pushes results as they’re collected, so partial data is preserved even if a run is interrupted.
• 🛡️ Robust error handling: Continues to the next store if one fails and logs an error entry to the dataset for full observability.
• 👨‍💻 Developer-friendly: Built on Apify SDK for Python with clean JSON output for ETL/BI pipelines, APIs, and scripts.

How to use Shopify Store Scraper - step by step

1. Sign in to the Apify Console and open the Shopify Store Scraper actor.
2. Add input in the INPUT tab:
   • Paste one or more Shopify store URLs into startUrls.
   • Set extractReviews to true or false depending on whether you want Yotpo reviews.
   • Optionally configure proxyConfiguration (e.g., useApifyProxy) for better resilience.
3. Start the run by clicking Start.
4. Monitor progress in the LOG tab. You’ll see notes about product page counts and any proxy fallback events.
5. Wait for completion. The actor processes stores sequentially and saves output items as they are scraped.
6. View results in the OUTPUT tab. Each store (or error) is a separate dataset item.
7. Export your data as JSON or CSV, or access it programmatically via the Apify API.

Pro tip: Chain this Shopify store crawler with your ETL/BI or CRM using the Apify API to download products from Shopify store catalogs at scale.

Use cases

Use case	Description
Competitive analysis & benchmarking	Track catalogs, prices, and variants across competitors to extract Shopify product details and compare assortments.
Market research & category insights	Analyze categories, vendors, and collections to discover gaps and opportunities with a Shopify collections scraper.
Price monitoring & alerts	Monitor min/avg/max pricing across Shopify stores to power pricing dashboards and alerts.
Inventory availability tracking	Measure inventory availability percentages and variant counts with a Shopify inventory scraper.
Theme & app intelligence	Identify theme IDs/versions and detect apps used by stores for technology research.
Review analytics (Yotpo)	Enrich top products with review counts and ratings for product quality and sentiment tracking using a Shopify product reviews scraper strategy.
Data pipelines (API)	Automate exports to JSON/CSV via the Apify API for data warehousing, BI, or enrichment workflows.

Why choose Shopify Store Scraper?

A production-ready, reliable Shopify web scraping tool built for precision and automation.

• ✅ Accurate public data: Leverages Shopify’s public JSON endpoints (/products.json, /collections.json, /meta.json) and HTML parsing for theme/app signals.
• ⚡ Fast and resilient: Async/await architecture plus proxy fallback ensures successful runs at scale.
• 🧠 Optional review depth: Toggle Yotpo review extraction for richer product intelligence when needed.
• 🔌 Easy integrations: Export JSON/CSV from Apify datasets or connect directly via API to your systems.
• 🔒 No login required: Works with publicly accessible Shopify stores without cookies or sessions.
• 🧱 Built for reliability: Live saving to dataset, robust error handling, and clear logging beat flaky browser extensions and one-off scripts.

In short: a stable Shopify product data scraping engine that combines coverage, resilience, and clean outputs.

Is it legal / ethical to use Shopify Store Scraper?

Yes — when used responsibly. The scraper collects data from publicly available sources (Shopify’s public JSON APIs and public pages). It does not access private or password-protected content.

Guidelines for compliant use:

• Only collect public data and respect each site’s terms of service.
• Ensure compliance with relevant regulations (e.g., GDPR, CCPA).
• Use scraped data responsibly; avoid spam and misuse.
• Respect rate limits and avoid abusive traffic patterns. For edge cases, verify your use with your legal team.

Input parameters & output format

Example JSON input

{
  "startUrls": [
    "https://www.allbirds.com",
    "https://example-store.myshopify.com"
  ],
  "extractReviews": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Input fields

• startUrls (array, required): List one or more Shopify store URLs (e.g., https://www.allbirds.com). Default: not specified.
• extractReviews (boolean, optional): Enable to extract detailed product reviews from Yotpo (slower but more comprehensive). Default: true.
• proxyConfiguration (object, optional): Choose which proxies to use. If Shopify rejects the proxy, a residential proxy will be used as a fallback. Default: not specified.

Example JSON output (successful store item)

{
  "store_domain": "www.allbirds.com",
  "store_url": "https://www.allbirds.com",
  "shop_name": "shop11044168.myshopify.com",
  "country": "US",
  "currency": "USD",
  "locale": "en-US",
  "theme_name": "rc-cm-2025-12-26_20-35 Update",
  "theme_id": 128596836432,
  "theme_version": "1.131.2",
  "apps_installed": ["Elevar Conversion Tracking", "JsEncrypt"],
  "apps_by_category": {
    "Analytics": ["Elevar Conversion Tracking"],
    "Other": ["JsEncrypt"]
  },
  "apps_count": 2,
  "total_products": 681,
  "total_variants": 7276,
  "collections": ["Accessories", "Shoes", "Apparel"],
  "collections_count": 250,
  "categories": ["Accessories", "Apparel", "Shoes", "Socks"],
  "vendors": ["Allbirds"],
  "price_min": 2.0,
  "price_avg": 79.34,
  "price_max": 500.0,
  "products_on_sale": 681,
  "discount_pct_avg": null,
  "products_with_reviews": 0,
  "review_coverage_pct": 0,
  "avg_reviews_per_product": null,
  "avg_rating": null,
  "total_estimated_sales": null,
  "avg_sales_per_product": null,
  "total_variants_available": 5647,
  "inventory_availability_pct": 77.6,
  "estimated_launch_date": "2018-11-12",
  "oldest_product_date": "2018-11-12",
  "newest_product_date": "2025-11-06",
  "products_with_images": 680,
  "avg_images_per_product": 4.5,
  "products_with_description": 681,
  "top_products": [
    {
      "title": "Allbirds Laces - Warm Red - Round",
      "url": "https://www.allbirds.com/products/allbirds-laces-warm-red-round",
      "price": 8.0,
      "reviews": 0,
      "rating": 0.0
    }
  ],
  "detailed_reviews": [
    {
      "product_url": "https://www.allbirds.com/products/allbirds-laces-warm-red-round",
      "product_id": "7242343809104",
      "reviews_count": 0,
      "avg_rating": 0.0,
      "reviews": []
    }
  ],
  "scraped_at": "2025-12-29T07:34:23.715157Z",
  "scrape_duration_sec": 15.71,
  "api_blocked": false
}

Example JSON output (error item when a store fails)

{
  "store_url": "https://unreachable-store.com",
  "error": "Failed to fetch homepage from https://unreachable-store.com. All request attempts failed.",
  "scraped_at": "2025-12-29T07:34:23.715157Z"
}

Notes

• Fields like theme_id, price_min/avg/max, and review metrics may be null when not available.
• When no valid startUrls are provided, the dataset will contain an error item with error, input_received, and scraped_at for debugging.

FAQ

How long does it take to scrape a store?

Small stores typically finish in seconds, while larger catalogs take longer. If extractReviews is enabled, additional time is spent fetching Yotpo data for the newest products.

What happens if a store blocks my requests?

The scraper implements intelligent proxy fallback. It starts without a proxy, switches to a datacenter proxy if blocked, then to a residential proxy with retries if needed. Once residential is used, it sticks with it for subsequent requests.

Can I scrape multiple stores at once?

Yes. Add multiple URLs to the startUrls array. The actor processes them sequentially and writes each result as a separate dataset item.

Do I need to enable a proxy?

No. Proxy is optional. However, enabling proxyConfiguration (e.g., useApifyProxy) can improve reliability for large-scale Shopify store crawler runs.

Can it extract product reviews?

Yes. Set extractReviews to true to fetch Yotpo review counts, average ratings, and review details for the top newest products. If review extraction fails for a product, the run continues.

What data formats can I export?

Results are stored in the Apify dataset and can be exported as JSON or CSV. You can also access data via the Apify API for downstream pipelines.

Does it work with every Shopify store?

It works with publicly accessible Shopify stores using endpoints like /products.json, /collections.json, and /meta.json. If a store heavily restricts access, proxy fallback increases the likelihood of success.

What happens if I provide no URLs?

If startUrls is empty or invalid, the actor will save an error item to the dataset describing the issue (including input_received for troubleshooting) and end the run.

Closing CTA / Final thoughts

Shopify Store Scraper is built to extract structured Shopify store intelligence at scale. With robust proxy fallback, detailed product and collection coverage, and optional Yotpo reviews, it’s ideal for marketers, developers, analysts, and researchers who need reliable Shopify product data scraping.

Export clean JSON/CSV from Apify datasets, or connect via the Apify API to automate a Shopify product CSV extractor pipeline end-to-end. Start extracting smarter insights from any Shopify store today. If you need custom features or priority support, email dev.scraperengine@gmail.com.