Shopify Store Scraper

A powerful Apify Actor that extracts comprehensive data from Shopify stores, including products, collections, themes, installed apps, and optional product reviews from Yotpo. Perfect for competitive analysis, market research, product monitoring, and e-commerce intelligence gathering.

Why Choose Us?

• 🔄 Smart Proxy Fallback: Automatically switches between no proxy, datacenter, and residential proxies with intelligent retry logic to ensure successful scraping
• 📊 Comprehensive Data Extraction: Captures everything from store metadata to product details, pricing, inventory, and customer reviews
• ⚡ High Performance: Async/await architecture ensures fast concurrent data fetching
• 🛡️ Robust Error Handling: Gracefully handles failures and continues processing remaining stores
• 💾 Live Data Saving: Results are saved in real-time to Apify dataset, so you don't lose data if the actor stops
• 📝 Detailed Logging: Comprehensive logging keeps you informed throughout the scraping process

Key Features

• Store Metadata: Extracts store domain, shop name, country, currency, locale, and theme information
• Product Data: Fetches all products with variants, prices, images, descriptions, and availability
• Collections: Retrieves all product collections and their organization
• Theme Information: Identifies theme name, ID, and version
• Installed Apps: Detects Shopify apps installed on the store (e.g., Elevar, JsEncrypt)
• Pricing Analytics: Calculates min, max, and average prices across all products
• Inventory Stats: Provides inventory availability percentages and variant counts
• Product Reviews: Optional extraction of Yotpo reviews for top products (author, rating, content, dates)
• Top Products: Identifies newest products with optional review data
• Smart Proxy Management: Intelligent fallback system ensures requests succeed even when blocked

Input

The actor accepts the following input parameters:

Input Schema

{
  "startUrls": [
    "https://www.allbirds.com",
    "https://example-store.myshopify.com"
  ],
  "extractReviews": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Input Fields

Field	Type	Required	Description
startUrls	array	✅ Yes	List of Shopify store URLs to scrape. Supports bulk input with multiple store URLs. Example: ["https://www.allbirds.com"]
extractReviews	boolean	❌ No	Enable to extract detailed product reviews from Yotpo for top 10 newest products. Default: true. Note: Enabling this will slow down the scraping process.
proxyConfiguration	object	❌ No	Apify proxy configuration. If not provided, the actor will attempt direct requests first. If blocked, it automatically falls back to datacenter proxy, then residential proxy with 3 retries.

Input Examples

Basic Usage (No Proxy)

{
  "startUrls": ["https://www.allbirds.com"],
  "extractReviews": true
}

With Proxy Enabled

{
  "startUrls": ["https://www.allbirds.com"],
  "extractReviews": true,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

Multiple Stores

{
  "startUrls": [
    "https://www.allbirds.com",
    "https://example-store.myshopify.com",
    "https://another-store.com"
  ],
  "extractReviews": false
}

Output

The actor outputs structured JSON data to the Apify dataset. Each store is saved as a separate dataset item.

Output Schema

{
  "store_domain": "www.allbirds.com",
  "store_url": "https://www.allbirds.com",
  "shop_name": "shop11044168.myshopify.com",
  "country": "US",
  "currency": "USD",
  "locale": "en-US",
  "theme_name": "rc-cm-2025-12-26_20-35 Update",
  "theme_id": 128596836432,
  "theme_version": "1.131.2",
  "apps_installed": ["Elevar Conversion Tracking", "JsEncrypt"],
  "apps_by_category": {
    "Analytics": ["Elevar Conversion Tracking"],
    "Other": ["JsEncrypt"]
  },
  "apps_count": 2,
  "total_products": 681,
  "total_variants": 7276,
  "collections": ["Accessories", "Shoes", "Apparel", ...],
  "collections_count": 250,
  "categories": ["Accessories", "Apparel", "Shoes", "Socks"],
  "vendors": ["Allbirds"],
  "price_min": 2.0,
  "price_avg": 79.34,
  "price_max": 500.0,
  "products_on_sale": 681,
  "total_variants_available": 5647,
  "inventory_availability_pct": 77.6,
  "estimated_launch_date": "2018-11-12",
  "oldest_product_date": "2018-11-12",
  "newest_product_date": "2025-11-06",
  "products_with_images": 680,
  "avg_images_per_product": 4.5,
  "products_with_description": 681,
  "top_products": [
    {
      "title": "Allbirds Laces - Warm Red - Round",
      "url": "https://www.allbirds.com/products/allbirds-laces-warm-red-round",
      "price": 8.0,
      "reviews": 0,
      "rating": 0.0
    }
  ],
  "detailed_reviews": [
    {
      "product_url": "https://www.allbirds.com/products/...",
      "product_id": "7242343809104",
      "reviews_count": 0,
      "avg_rating": 0.0,
      "reviews": []
    }
  ],
  "scraped_at": "2025-12-29T07:34:23.715157Z",
  "scrape_duration_sec": 15.71,
  "api_blocked": false
}

Output Fields

Field	Type	Description
store_domain	string	Store domain name (without protocol)
store_url	string	Full store URL
shop_name	string	Shopify shop name (e.g., shop12345.myshopify.com)
country	string	Store country code
currency	string	Store currency code
locale	string	Store locale (e.g., en-US)
theme_name	string	Active theme name
theme_id	integer	Shopify theme ID
theme_version	string	Theme version number
apps_installed	array	List of detected Shopify apps
apps_by_category	object	Apps grouped by category
apps_count	integer	Total number of installed apps
total_products	integer	Total number of products in store
total_variants	integer	Total number of product variants
collections	array	List of all collection titles
collections_count	integer	Total number of collections
categories	array	List of product categories
vendors	array	List of product vendors
price_min	float	Minimum product price
price_avg	float	Average product price
price_max	float	Maximum product price
total_variants_available	integer	Number of available variants
inventory_availability_pct	float	Percentage of variants in stock
estimated_launch_date	string	Date of oldest product (store launch estimate)
oldest_product_date	string	Creation date of oldest product
newest_product_date	string	Creation date of newest product
products_with_images	integer	Number of products with images
avg_images_per_product	float	Average number of images per product
products_with_description	integer	Number of products with descriptions
top_products	array	Top 10 newest products with basic info and optional reviews
detailed_reviews	array	Detailed review data (only if extractReviews is enabled)
scraped_at	string	ISO timestamp of when scraping completed
scrape_duration_sec	float	Time taken to scrape the store in seconds
api_blocked	boolean	Whether the Shopify API was blocked

🚀 How to Use the Actor (via Apify Console)

1. Log in to Apify Console and navigate to Actors
2. Find the shopify-store-scraper actor and click on it
3. Configure Inputs:
   • Add one or more Shopify store URLs in the startUrls field
   • Toggle extractReviews to enable/disable review extraction (default: enabled)
   • Optionally configure proxy settings in proxyConfiguration
4. Run the Actor: Click the Start button to begin scraping
5. Monitor Progress: Watch real-time logs to track scraping progress and proxy fallback events
6. Access Results: Once complete, go to the OUTPUT tab to view scraped data
7. Export Data: Download results as JSON or CSV, or access via Apify API

Best Use Cases

• 🔍 Competitive Analysis: Monitor competitor stores, pricing strategies, and product catalogs
• 📈 Market Research: Analyze market trends, product categories, and vendor distribution
• 💰 Price Monitoring: Track pricing changes across multiple Shopify stores
• 📦 Inventory Tracking: Monitor product availability and inventory levels
• 🎨 Theme Research: Identify popular Shopify themes and their usage
• 🔌 App Discovery: Find which Shopify apps are commonly used by stores
• ⭐ Review Analysis: Extract and analyze customer reviews for product insights
• 📊 E-commerce Intelligence: Build comprehensive databases of Shopify store information

Frequently Asked Questions

How long does it take to scrape a store?

Scraping time depends on the store size:

• Small stores (< 100 products): ~5-10 seconds
• Medium stores (100-500 products): ~10-30 seconds
• Large stores (500+ products): ~30-60+ seconds
• With review extraction enabled: Add ~2-5 seconds per product reviewed

What happens if a store blocks my requests?

The actor implements intelligent proxy fallback:

1. First attempts without proxy
2. If blocked, switches to datacenter proxy
3. If still blocked, switches to residential proxy
4. Retries up to 3 times with residential proxy
5. Once residential proxy is used, it sticks with it for all remaining requests

Can I scrape multiple stores at once?

Yes! Simply add multiple URLs to the startUrls array. The actor processes them sequentially to avoid overwhelming servers.

Do I need to enable proxy?

No, proxy is optional. The actor works without proxy, but enabling it can help avoid rate limits and blocks, especially for large-scale scraping.

What if review extraction fails for a product?

The actor continues processing other products even if review extraction fails for some. Failed products will have reviews: 0 and rating: 0.0 in the output.

Can I customize which products get reviews extracted?

Currently, the actor extracts reviews for the top 10 newest products. This is a fixed limit to balance speed and comprehensiveness.

What data format is the output?

Output is in JSON format, saved to Apify dataset. You can export it as JSON, CSV, or access it via Apify API.

Does the actor work with all Shopify stores?

Yes, the actor works with any publicly accessible Shopify store. It uses Shopify's public JSON APIs (/products.json, /collections.json, /meta.json) which are available on all Shopify stores.

Support and Feedback

💬 For custom solutions or feature requests, contact us at dev.scraperengine@gmail.com

We're always looking to improve! If you encounter any issues or have suggestions, please reach out.

Cautions

⚠️ Important Legal and Ethical Considerations:

• Data is collected only from publicly available sources (Shopify's public JSON APIs)
• No data is taken from private accounts or password-protected content
• The end user is responsible for ensuring legal compliance with:
  • Local data protection laws (GDPR, CCPA, etc.)
  • Terms of service of the target websites
  • Copyright and intellectual property laws
  • Spam and privacy regulations
• Respect rate limits: The actor includes delays between requests to be respectful
• Use responsibly: Only scrape data you have permission to access or that is publicly available
• Review terms of service: Always check the target store's terms of service before scraping

---

Built with ❤️ using Apify Platform