Shopify Store Scraper

Scrape Shopify store products and collections using the official Shopify JSON API. No headless browser required - extremely fast and reliable.

Features

• Official JSON API: Uses Shopify's built-in /products.json and /collections.json endpoints
• No Browser Required: Pure HTTP requests - fast and resource-efficient
• Comprehensive Data: Products, variants, images, pricing, inventory, collections
• Multi-Store Support: Scrape multiple Shopify stores in one run
• Automatic Pagination: Handles large product catalogs
• Rate Limiting: Built-in delays to respect Shopify's limits
• Error Handling: Detects password-protected stores, invalid URLs, rate limits

Input Parameters

Parameter	Type	Default	Description
storeUrls	array	-	List of Shopify store URLs
storeUrl	string	-	Single Shopify store URL
collectionsOnly	boolean	false	Only scrape collection data
includeVariants	boolean	true	Include all product variants
includeImages	boolean	true	Include all product images
maxProductsPerStore	number	1000	Max products per store
maxResults	number	5000	Total max results

Example Input

{
  "storeUrls": [
    "https://www.allbirds.com",
    "https://shop.gymshark.com"
  ],
  "includeVariants": true,
  "includeImages": true,
  "maxProductsPerStore": 500
}

Output Schema

Product Record

{
  "productId": 123456789,
  "title": "Merino Wool Runner",
  "handle": "merino-wool-runner",
  "description": "The world's most comfortable shoe...",
  "vendor": "Allbirds",
  "productType": "Shoes",
  "tags": ["Sustainable", "Merino Wool"],
  "createdAt": "2023-01-15T10:00:00Z",
  "updatedAt": "2024-02-13T15:30:00Z",
  "publishedAt": "2023-01-20T09:00:00Z",
  "productUrl": "https://www.allbirds.com/products/merino-wool-runner",
  "storeUrl": "https://www.allbirds.com",
  "storeName": "Allbirds",
  "price": 98.00,
  "currency": "USD",
  "compareAtPrice": 120.00,
  "available": true,
  "imageUrl": "https://cdn.shopify.com/...",
  "variantCount": 12,
  "variants": [
    {
      "variantId": 987654321,
      "variantTitle": "Black / US 10",
      "price": 98.00,
      "compareAtPrice": 120.00,
      "currency": "USD",
      "sku": "WR-BLK-10",
      "available": true,
      "weight": 450,
      "weightUnit": "g",
      "inventoryQuantity": 25
    }
  ],
  "images": [
    "https://cdn.shopify.com/image1.jpg",
    "https://cdn.shopify.com/image2.jpg"
  ]
}

Collection Record (when collectionsOnly: true)

{
  "collectionId": 123456789,
  "handle": "mens-shoes",
  "title": "Men's Shoes",
  "description": "Comfortable shoes for men",
  "updatedAt": "2024-02-13T15:30:00Z",
  "publishedAt": "2023-01-01T00:00:00Z",
  "sortOrder": "best-selling",
  "collectionUrl": "https://www.allbirds.com/collections/mens-shoes",
  "storeUrl": "https://www.allbirds.com",
  "storeName": "Allbirds"
}

Shopify JSON API Endpoints Used

• {store_url}/meta.json - Store metadata
• {store_url}/products.json?limit=250&page={n} - Paginated products
• {store_url}/collections.json - All collections

Error Handling

Status Code	Meaning	Behavior
404	Not a Shopify store	Skip and log error
401/403	Password protected	Skip and log warning
429/430	Rate limited	Retry with exponential backoff

Rate Limiting

• 1-2 seconds delay between requests per store
• Automatic retry on rate limit errors
• Respects Shopify's generous API limits

Use Cases

• Competitor Analysis: Monitor competitor pricing and inventory
• Market Research: Analyze product trends across multiple stores
• Price Monitoring: Track price changes and sales
• Product Discovery: Find new products and collections
• Inventory Tracking: Monitor stock levels
• Data Integration: Feed product data into your systems

Example Stores to Test

• https://www.allbirds.com
• https://shop.gymshark.com
• https://www.fashionnova.com
• https://www.mvmtwatches.com
• https://www.colourpop.com

Notes

• All Shopify stores have these JSON endpoints enabled by default
• No authentication required for public stores
• Password-protected stores will return 401/403
• Currency detection is basic (defaults to USD) - Shopify doesn't expose currency in products.json
• Some stores may have custom rate limits

Performance

• Scrapes ~250 products per request (Shopify's max)
• Typical store (1000 products): 2-3 minutes
• No browser overhead = minimal memory usage
• Highly reliable (uses official API)

License

MIT