Shopify Products Scraper

Shopify Products Scraper

The Shopify Products Scraper is a fast, reliable Shopify product scraper that discovers product pages and extracts structured product data from public Shopify stores. It solves the repetitive, manual work of collecting titles, prices, variants, images, and metadata by automating discovery and product JSON retrieval at scale. Built for marketers, developers, data analysts, and researchers, this Shopify product data scraper enables bulk operations like catalog building, competitor monitoring, and price tracking with robust anti-blocking controls.

What data / output can you get?

Below are the exact fields saved to the Apify dataset for each product. Values are derived from the Shopify product .json response and discovery workflow.

• Data type | Description | Example value
• store_url | The Shopify store base URL | https://lootcrate.com
• product_url | Product page URL discovered on the store | https://lootcrate.com/products/loot-crate
• json_url | Product JSON endpoint (path with .json) | https://lootcrate.com/products/loot-crate.json
• product_id | Unique product ID from Shopify | 5083963261059
• title | Product title from Shopify | Loot Crate
• vendor | Product vendor | Loot Crate Core
• product_type | Product type/category | Subscription Box
• price | Current price from the first variant (if present) | 29.99
• compare_at_price | Compare-at price from the first variant (if present) | 24.99
• tags | Comma-separated product tags | Subscription, Collectibles, Pop Culture
• total_found | Count of products discovered for the store in this run | 5
• successful | Number of product JSONs successfully extracted for the store so far | 5

Bonus: Each dataset item also includes full_data, which preserves the complete raw Shopify product JSON for maximum flexibility (variants, images, timestamps, etc.). You can export your dataset as JSON or CSV directly from Apify.

Key features

• 🔍 Automatic product discovery
  Scans store HTML to find links containing “/products/” and builds a clean list of product URLs — no manual listing required for this Shopify products scraper tool.

• 📦 Complete product JSON capture
  Saves high-value fields (id, title, vendor, type, price, compare_at_price, tags) and the full_data object containing the raw Shopify product JSON for deep analysis.

• 🔄 Smart anti-blocking with proxy fallback
  Starts with no proxy, automatically falls back to datacenter, then residential proxies on 403/429 responses — with sticky residential use for the remaining requests.

• ⚡ Concurrent, scalable scraping
  Asynchronous fetching with a max concurrency of 20 product JSON requests per store enables fast bulk runs across multiple Shopify stores.

• 💾 Live data saving
  Pushes each product to the Apify dataset as soon as it’s extracted, so you can monitor progress and never lose results mid-run.

• 🧪 Reliable error handling
  Built-in retries, graceful handling of failed requests, and detailed logging make this Shopify product scraping tool production-ready.

• 🧰 Developer- and workflow-friendly
  Run on Apify, integrate results into pipelines (export JSON/CSV), and use the “OUTPUT” key-value record for grouped, per-store summaries.

How to use Shopify Products Scraper - step by step

1. Create or log in to your Apify account.
2. Open the “shopify-products-scraper” actor in Apify Console.
3. In the Input tab, paste one or more Shopify store URLs into the startUrls array (e.g., https://lootcrate.com).
4. (Optional) Configure proxyConfiguration. By default, the actor starts with no proxy and automatically escalates to datacenter → residential proxies if blocked.
5. Click Start to run. The actor will detect whether the site is Shopify; if so, it paginates /products.json, otherwise it discovers product links from the HTML.
6. Watch the run logs for discovery counts, proxy switches, and success/failure messages.
7. Open the Dataset to view results in a table and export data to JSON or CSV.
8. Pro tip: For automation, read the grouped per-store summary from the default key-value store item named OUTPUT and feed it into downstream systems or a Shopify product extractor workflow.

Use cases

• Use case name | Description
• E-commerce intelligence | Track competitor catalogs and pricing to benchmark offers and spot assortment gaps across multiple Shopify stores.
• Price monitoring | Monitor price and compare-at price changes over time to detect promotions and margin opportunities.
• Catalog building | Extract product data for listings, merchandising, or content pipelines with a dependable Shopify product data extraction workflow.
• Market research | Analyze vendor, product types, and tags to understand category trends and inventory strategies.
• API/data pipelines | Feed full_data into analytics warehouses or processing jobs for variant- and image-level insights at scale.
• Academic research | Collect public product datasets for studies on pricing, assortment, and online retail dynamics.
• Product images analysis | Use the preserved full_data to build a Shopify product images scraper pipeline for media processing tasks.

Why choose Shopify Products Scraper?

This tool is built for precision, scale, and reliability — a Shopify store scraper for products that goes beyond simple page scraping.

• ✅ Accurate product JSON extraction: Captures authoritative Shopify product fields and the entire raw JSON.
• ⚡ Built for scale: Concurrent requests and automatic product discovery for bulk stores.
• 🔗 Smart proxy fallback: Direct → datacenter → residential with retries to minimize blocks.
• 💾 Resilient by design: Live dataset writes and robust error handling reduce data loss.
• 🧩 Integration-ready: Export results as JSON/CSV or consume grouped OUTPUT for automation.
• 🛡️ Public data only: No login or private access required — works from public endpoints.
• 💻 Developer friendly: Operates as an Apify actor suitable for pipelines, scripts, and repeatable runs.

Bottom line: A best-in-class Shopify product scraper alternative to brittle extensions — stable infrastructure, clear outputs, and production-ready behavior.

Is it legal / ethical to use Shopify Products Scraper?

Yes — when used responsibly. This actor collects data from publicly accessible Shopify product pages and endpoints only. It does not access private accounts or authenticated areas.

Guidelines for compliant use:

• Scrape only public data and respect website terms.
• Be mindful of rate limits and avoid abusive behavior.
• Ensure compliance with applicable laws (e.g., GDPR, CCPA).
• Do not use the tool for harassment, unauthorized data collection, or illegal activities.
• Consult your legal team for edge cases and jurisdiction-specific requirements.

Input parameters & output format

Example JSON input

{
  "startUrls": [
    "https://lootcrate.com",
    "https://www.decathlon.com"
  ],
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Input fields

• Field: startUrls
  • Type: array
  • Description: List one or more Shopify store URLs (e.g., https://lootcrate.com, https://www.decathlon.com). Supports bulk input.
  • Default (prefill): ["https://lootcrate.com"]
  • Required: Yes
• Field: proxyConfiguration
  • Type: object
  • Description: Choose which proxies to use. By default, no proxy is used. If the platform rejects or blocks the request, it will automatically fallback to datacenter proxy, then residential proxy with 3 retries.
  • Default (prefill): {"useApifyProxy": false}
  • Required: No

Example JSON output (dataset item)

{
  "store_url": "https://lootcrate.com",
  "product_url": "https://lootcrate.com/products/loot-crate",
  "json_url": "https://lootcrate.com/products/loot-crate.json",
  "product_id": 5083963261059,
  "title": "Loot Crate",
  "vendor": "Loot Crate Core",
  "product_type": "Subscription Box",
  "price": "29.99",
  "compare_at_price": "24.99",
  "tags": "Subscription, Collectibles, Pop Culture",
  "total_found": 5,
  "successful": 5,
  "full_data": {
    "product": {
      "id": 5083963261059,
      "title": "Loot Crate",
      "vendor": "Loot Crate Core",
      "product_type": "Subscription Box",
      "handle": "loot-crate",
      "tags": "Subscription, Collectibles, Pop Culture",
      "variants": [
        {
          "id": 34197535719555,
          "price": "29.99",
          "compare_at_price": "24.99",
          "sku": "1010126US"
        }
      ],
      "images": [
        {
          "id": 123456789,
          "src": "https://cdn.shopify.com/..."
        }
      ]
    }
  }
}

Note:

• price and compare_at_price are taken from the first variant when available and may be null if a product has no variants.
• Additionally, the actor writes a grouped, per-store summary to the default key-value store under the key OUTPUT:

{
  "https://lootcrate.com": {
    "method": "shopify_api",
    "total_found": 5,
    "successful": 5,
    "products": [
      {
        "url": "https://lootcrate.com/products/loot-crate",
        "json": { "product": { "id": 5083963261059, "handle": "loot-crate" } }
      }
    ]
  }
}

FAQ

Can this Shopify product scraper extract variants, images, and full product details?

Yes. Each dataset item includes full_data, which preserves the complete Shopify product JSON. That JSON contains variants, images, tags, timestamps, and more, making it a robust Shopify product data scraper for analysis.

How does the scraper discover products on a store?

It scans the store’s HTML for links containing “/products/” and builds product URLs automatically. If Shopify is detected, it also paginates the /products.json endpoint to extract product handles efficiently.

What happens if a store blocks my requests?

The scraper starts with no proxy and automatically falls back to datacenter and then residential proxies on 403/429 responses. Once residential is activated, it stays “sticky” for subsequent requests to improve reliability.

Can I scrape multiple Shopify stores at once?

Yes. Add multiple URLs to startUrls. The scraper processes them and writes each product to the dataset, functioning as a scalable Shopify store scraper for products.

Do I need to log in or use a Shopify API key?

No. This Shopify product extractor works with public product pages and JSON endpoints; no authentication or cookies are required.

In what formats can I download results?

You can export the Apify dataset to JSON or CSV. This makes it easy to download Shopify products CSV or integrate with analytics pipelines.

Will it work on non-Shopify websites?

If a site isn’t detected as Shopify, the actor falls back to HTML discovery for links that include “/products/”. However, the best results come from Shopify stores, as the product .json structure is standardized there.

How many products can it scrape per store?

It discovers products via HTML and/or the /products.json pagination and fetches product JSON concurrently (max concurrency 20). Throughput depends on store size, network conditions, and blocking behavior.

Closing CTA / Final thoughts

Shopify Products Scraper is built to extract structured product data from Shopify stores at scale. With automatic discovery, robust proxy fallback, live dataset writes, and full JSON preservation, it’s ideal for marketers, developers, analysts, and researchers.

Run it on Apify to scrape Shopify products, export results as JSON/CSV, and plug outputs into your workflows. Developers can consume the dataset and the grouped OUTPUT record to power automation pipelines and integrations. Start extracting smarter, more reliable Shopify product data today.