Shopify Scraper: Related Products & Price Comparison

Shopify Scraper — Products, Related Products & Price Comparisons

Scrape any public Shopify store — homepage, collection, or single product URL — and get back typed product records plus, for every product, the storefront's own recommended related products with a computed price comparison: raw delta, percentage difference, a cheaper/pricier/same classification, and a currency-consistency check against the store's declared currency. No Shopify account, no Admin API key, and no browser required — the actor reads the storefront's own public JSON endpoints. Unlike scraping frameworks that hand you raw HTML, this actor returns typed JSON — ready for your pipeline, database, or LLM context window without any parsing. This guide covers every input and output field, and how teams actually run it for pricing audits, catalogue monitoring, and AI-training datasets.

🧭 What Does This Shopify Scraper Do?

It takes one or more Shopify URLs — a store homepage, a /collections/<handle> page, or a single /products/<handle> page — and pulls every product it finds through the storefront's own public JSON endpoints (/products.json, /collections/<handle>/products.json). For each product, it optionally calls the storefront's own recommendations endpoint (/recommendations/products.json) to fetch related products, then computes a price comparison between the parent product and each related item. No Shopify login or Admin API credentials are required — only the storefront needs to be publicly reachable.

• Scrapes products from a store homepage, a specific collection, or a single product URL
• Returns full variant detail per product: SKU, price, compare-at price, stock status, options
• Fetches each product's own platform-recommended related products (0–10 per product)
• Computes price delta, percentage difference, and a cheaper/pricier/same classification for every related product
• Flags a currency mismatch between a product's own declared currency and its store's declared currency
• Detects password-protected storefronts instead of silently returning zero products
• Falls back through direct → Apify datacenter proxy → Apify residential proxy automatically when a storefront blocks requests

⚡ Features & Capabilities

The actor covers two things: the product scrape itself, and the related-product price-comparison layer built on top of it.

Core features

• Three input shapes: store homepage, collection URL, or single product URL — all in the same startUrls list
• Full variant data: id, title, sku, options, and price (current, previous, stockStatus) for every variant
• Media: every product image, normalized to an {id, type, url, variantIds, alt} shape
• Related-product fetch: reads the storefront's public /recommendations/products.json?product_id=…&limit=… endpoint per product
• Price-comparison analytics: priceDelta, priceDeltaPct, priceComparisonClass per related product, plus per-parent aggregates (relatedCheaperCount, relatedPricierCount, relatedSameCount, avgRelatedPriceDeltaPct)
• Currency-consistency guard: currencyMismatch compares a product's own currency to its store's declared currency, and returns null — never a fabricated false — when either side can't be determined
• Automatic collection discovery on a store homepage: tries /collections/all first, then parses the homepage for other collection links if that page is empty

How this compares to building it yourself

No competitor Actor data was reviewed for this comparison (none was available at the time of writing), so this compares against the two realistic DIY alternatives instead of naming other Store listings.

Approach	Output	Related products	Setup
This actor	Typed JSON, one row per product/related-product	✅ Built in, with price comparison	Paste URLs, run
Shopify's own APIs (self-built)	Typed JSON, but you write the integration	Only via the Storefront GraphQL productRecommendations query, which requires a Storefront API access token per store — recommendations aren't exposed by the Admin REST API at all (shopify.dev, checked 2026-08-15)	Requires a token/app per store
Raw HTML scraping	Unstructured HTML you parse yourself	Only if the theme happens to render it in markup	Breaks on theme changes

If your use case is feeding structured data to an LLM or a pricing dashboard, the output-format row is the decision-maker — parsing HTML or standing up an Admin/Storefront API integration per store is a reliability failure mode, not a feature.

When another tool might suit you better

If you only need the related-product list itself — without the price-delta analytics — the lighter Shopify Products Scraper: Media, Currency & Related Products variant already covers that fetch on its own. And if your goal is merchant contact discovery rather than product pricing, Shopify Store Scraper: Emails, Phones & Social Links returns the store's published contact and social data instead of product rows. Pick this actor specifically when the price-comparison math — delta, percentage, cheaper/pricier/same, currency check — is what you need.

This scraper within the Scrapio data stack

This actor covers products and their related-product price comparisons. For a store's contact details, socials, and policy inventory, use Shopify Store Scraper: Emails, Phones & Social Links. For structured description and tag-taxonomy extraction on the catalogue, use Shopify Products Scraper: Descriptions, Specs & Tags. For merchant lead scoring with per-country currency and shipping simulation, use Shopify Merchant Scraper: Market Currency & Shipping Data.

📊 Why Do Developers and Data Teams Scrape Shopify?

🏢 Pricing and merchandising teams

Pull a competitor's catalogue by collection URL and get, per product, the parent's parentHeadlinePrice alongside every related product's relatedPrice, priceDelta, and priceComparisonClass. That gives you a ready audit of where your own SKUs sit against the storefront's own cross-sell recommendations — no manual page-by-page comparison. Feed the rows straight into a pricing sheet or BI tool keyed on parentProductId / relatedProductId.

📊 AI training data and RAG indexing

The title, description (raw body_html), brand, categories, and tags fields on every product row are the high-information text fields for RAG enrichment of a product catalogue. For training data, variants[].price, variants[].sku, and the priceComparisonClass labels give you consistently structured, typed fields across every record — no HTML parsing or price-string normalization needed before indexing or fine-tuning.

📱 Competitive and market intelligence

Run the same collection URL on a schedule and track parentHeadlinePrice and avgRelatedPriceDeltaPct over time to see whether a competitor is repositioning a product against its own recommended alternatives, or whether currencyMismatch starts flipping true after a market-expansion change.

🔬 Research and academic use

Catalogue-pricing and product-recommendation-graph research can use relatedCount, priceComparisonClass, and the variant price fields across a sample of public storefronts. Data is limited to what each storefront publishes publicly — no private or account-gated data is collected.

🎥 Product and SaaS development

Build a price-monitoring or catalogue-enrichment product on top of this actor's output: the type field distinguishes product rows from related-product rows in the same dataset, so a downstream consumer can key its own schema off that value directly.

🍚 Input Parameters

Parameter	Required	Type	Constraints	Description
startUrls	Yes	array	editor: stringList	Paste Shopify store URLs, one per line. Examples: store homepage https://store.com/, collection https://store.com/collections/shoes, single product https://store.com/products/blue-sneakers.
productQuery	No	string	editor: textfield, prefill ""	Accepted for compatibility with the base Shopify Scraper's "query" input. This variant does not use it to filter or search results — it is stored on the run but has no effect on the scraped rows. Reserved for future use.
query	No	string	editor: textfield, nullable	Legacy alias of productQuery, kept so an input JSON built for the base Shopify Scraper still runs unchanged on this variant. Not used to filter results.
maxResults	No	integer	minimum: 0, default 100	Maximum products to scrape per start URL. Set to 0 for no limit.
fetchRelatedProducts	No	boolean	default true	Fetch each product's own platform-recommended related products and compute a price comparison against the parent listing.
maxRecommendationsPerProduct	No	integer	minimum: 0, maximum: 10, prefill 5	How many related/recommended products to fetch per product (0–10). Set 0 to disable the fetch entirely. The schema has no declared default — if the input is omitted, the run treats it as 5.
computePriceComparison	No	boolean	default true	For each related product, compute the price delta, percentage difference, and a cheaper/pricier/same classification against the parent product's own headline price.
flagCurrencyMismatch	No	boolean	default true	Compare each product's own declared currency against its store's declared currency and flag a mismatch. Emits null (not a fabricated false) when either currency cannot be determined.
proxyConfiguration	No	object	editor: proxy, prefill {"useApifyProxy": false}	Optional Apify proxy configuration. Note: the first request to any store is always made direct — this input only takes effect once a storefront blocks a direct request, first via an Apify datacenter proxy (group SHADER unless you specify apifyProxyGroups), then via a residential proxy if blocking continues.

Only startUrls is required.

{
  "startUrls": ["https://boat-lifestyle.com/", "https://store.com/collections/shoes"],
  "maxResults": 50,
  "fetchRelatedProducts": true,
  "maxRecommendationsPerProduct": 5,
  "computePriceComparison": true,
  "flagCurrencyMismatch": true,
  "proxyConfiguration": { "useApifyProxy": true }
}

Supported URL types and input formats

startUrls accepts one URL per line via the list editor. Three formats are recognized by path:

• Store homepage — https://store.com/ — the actor scrapes /collections/all first, and if that returns nothing, discovers and scrapes the store's other collections from links found on the homepage.
• Collection page — https://store.com/collections/shoes — paginates through /collections/shoes/products.json?limit=250&page=N until maxResults is reached or the collection runs out of products.
• Single product page — https://store.com/products/blue-sneakers — scrapes exactly that one product.

A bare domain without a scheme (store.com) is accepted and normalized to https://store.com. productQuery and query are accepted on the input but are not used to filter or search — only the URLs you supply determine what gets scraped.

📦 Output Format

Every run pushes typed JSON rows to the default dataset in real time — one row per product, followed by one row per related product found for it. type distinguishes the two row shapes in the same dataset. Related-product rows are additionally mirrored to a separate dataset named <runId>-related-products when a run ID is available (i.e., when running on the Apify platform) — this is a best-effort mirror, not a required delivery path.

Output for a product record

{
  "type": "product",
  "isChild": false,
  "handle": "blue-sneakers",
  "source": {
    "id": "7891234567",
    "canonicalUrl": "https://boat-lifestyle.com/products/blue-sneakers",
    "retailer": "boat-lifestyle.com",
    "language": "en",
    "currency": "USD",
    "createdUTC": "2023-02-11T09:15:00-05:00",
    "updatedUTC": "2026-07-30T14:02:11-04:00",
    "publishedUTC": "2023-02-11T10:00:00-05:00"
  },
  "title": "Blue Canvas Sneakers",
  "description": "<p>Lightweight canvas sneakers with a rubber sole.</p>",
  "brand": "Boat Lifestyle",
  "categories": ["Footwear", "Casual"],
  "tags": ["Footwear", "Casual"],
  "variants": [
    {
      "id": "43219876543",
      "title": "US 9 / Blue",
      "sku": "BL-SNK-09-BLU",
      "options": ["US 9", "Blue"],
      "price": { "current": 4999, "previous": 5999, "stockStatus": "InStock" }
    }
  ],
  "medias": [
    {
      "id": "0",
      "type": "Image",
      "url": {
        "id": 33221100,
        "product_id": 7891234567,
        "position": 1,
        "created_at": "2023-02-11T09:15:00-05:00",
        "updated_at": "2023-02-11T09:15:00-05:00",
        "alt": "Blue canvas sneaker, side view",
        "width": 1200,
        "height": 1200,
        "src": "https://cdn.shopify.com/s/files/.../blue-sneakers.jpg",
        "variant_ids": []
      },
      "variantIds": [],
      "alt": ""
    }
  ],
  "options": [{ "type": "Size", "values": [{ "id": "US 9", "name": "US 9" }] }],
  "scrapedAt": "2026-08-15T09:12:03Z",
  "parentHeadlinePriceMinor": 4999,
  "parentHeadlinePrice": 49.99,
  "parentCurrency": "USD",
  "storeCurrency": "USD",
  "currencyMismatch": false,
  "relatedCount": 4,
  "relatedCheaperCount": 1,
  "relatedPricierCount": 2,
  "relatedSameCount": 1,
  "avgRelatedPriceDeltaPct": 6.3
}

price.current and price.previous are integer minor units (cents). parentHeadlinePrice is the parent's own cheapest-variant price, in decimal units, used as the price-comparison baseline; parentHeadlinePriceMinor is the same value in minor units. relatedCount is null (not 0) when the related-product lookup itself failed, and null (not attempted) when fetchRelatedProducts is off or maxRecommendationsPerProduct is 0 — the actor never reports a fabricated zero for "we didn't check."

Output for a related-product record

{
  "type": "relatedProduct",
  "isChild": true,
  "parentProductId": "7891234567",
  "parentHandle": "blue-sneakers",
  "relatedProductId": "7891234999",
  "relatedHandle": "black-canvas-sneakers",
  "relatedTitle": "Black Canvas Sneakers",
  "relatedPrice": 53.99,
  "relatedPriceMinor": 5399,
  "relatedUrl": "https://boat-lifestyle.com/products/black-canvas-sneakers",
  "relatedVendor": "Boat Lifestyle",
  "relatedAvailable": true,
  "rank": 1,
  "scrapedAt": "2026-08-15T09:12:04Z",
  "priceDelta": 4.0,
  "priceDeltaPct": 8.0,
  "priceComparisonClass": "pricier"
}

priceComparisonClass is "cheaper", "pricier", or "same". priceDelta and priceDeltaPct are null when computePriceComparison is off, or when either the parent's or the related item's price could not be resolved to minor units. The related-product surface does not carry a per-item currency field, so the price comparison compares raw numbers only — it does not attempt to convert currencies.

Schema stability and export options

Field names are fixed by this actor's own code, not read live from Shopify's markup, so they stay stable across a store's theme changes — only the underlying values change. Results are stored in a standard Apify dataset and can be exported from the Apify Console or API as JSON, CSV, Excel, XML, RSS, or HTML table, in addition to being consumed programmatically via the Apify API or apify-client.

💡 Shopify Related-Products Strategy Guide

🎯 Strategy 1: Real-time pricing enrichment

Trigger a run per store or collection URL when you need a fresh price snapshot — say, before a pricing-committee review. Read parentHeadlinePrice, relatedCheaperCount/relatedPricierCount, and avgRelatedPriceDeltaPct off each product row and write them back into your pricing sheet or PIM keyed on source.id (Shopify's own product ID). No polling loop needed — the run finishes with all rows already in the dataset.

🎯 Strategy 2: Scheduled monitoring and alerting

Set up an Apify Console schedule to re-run the same startUrls on a cadence you choose. Diff each run's parentHeadlinePrice and currencyMismatch against the previous run's values for the same source.id, and alert when priceComparisonClass flips on a related product you track, or when a currency mismatch starts appearing after a market change.

🎯 Strategy 3: Bulk catalogue dataset build

Pass a large startUrls list of collection or store URLs in a single run, or fan out one run per store URL for parallel collection. Each run's dataset can be exported to CSV directly from the Apify Console, or pulled programmatically and unioned into a warehouse table, keyed on type to separate product rows from related-product rows before loading.

Strategy comparison at a glance

Strategy	Best for	Run pattern	Output format
Real-time enrichment	Pre-review price checks	On-demand single run	Dataset JSON, read immediately
Scheduled monitoring	Ongoing competitor/price tracking	Apify Console schedule	Dataset JSON, diffed run-to-run
Bulk dataset build	Research or warehouse loads	One run, many startUrls	CSV/JSON export from the Console or API

🌴 Related Shopify Scrapers & Tools

Scraper	What it extracts
Shopify Products Scraper: Media, Currency & Related Products	The same related-product recommendations fetch, without the price-comparison analytics — lighter weight if you only need the list
Shopify Products Scraper: Descriptions, Specs & Tags	Structured description text and a tag-derived attribute map per product
Shopify Store Scraper: Emails, Phones & Social Links	Store-level contact details, social links, and policy inventory instead of product rows
Shopify Merchant Scraper: Market Currency & Shipping Data	Merchant lead rows plus per-destination-country currency and shipping-cost simulation

🔌 How to Integrate This Scraper With Your Stack

This actor works with any language or tool that can call the Apify API. The examples below use apify-client, the official SDK.

Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")

run_input = {
    "startUrls": ["https://boat-lifestyle.com/"],
    "maxResults": 50,
    "fetchRelatedProducts": True,
    "maxRecommendationsPerProduct": 5,
    "computePriceComparison": True,
    "flagCurrencyMismatch": True,
}

run = client.actor("scrapio/shopify-scraper-related-products-and-price-comparison").call(run_input=run_input)

rows = []
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    rows.append(item)

products = [r for r in rows if r["type"] == "product"]
related = [r for r in rows if r["type"] == "relatedProduct"]
print(f"{len(products)} products, {len(related)} related-product comparisons")

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });

const run = await client.actor('scrapio/shopify-scraper-related-products-and-price-comparison').call({
    startUrls: ['https://boat-lifestyle.com/'],
    maxResults: 50,
    fetchRelatedProducts: true,
    maxRecommendationsPerProduct: 5,
    computePriceComparison: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
const pricier = items.filter((r) => r.priceComparisonClass === 'pricier');
console.log(`${pricier.length} related products priced above their parent`);

Async and scheduled pipelines

For large startUrls lists, start the run via the API and poll client.run(runId).get() (or the equivalent REST call) for status rather than blocking on .call(). For recurring jobs, use an Apify Console schedule to re-trigger the same input on a cadence, and either poll the new run's dataset when it finishes or configure a webhook on the run's completion event to notify your own system.

🎯 Who Needs This Shopify Scraper? (Use Cases & Industries)

🏢 Pricing and merchandising teams

Audit a competitor's or your own store's pricing against the storefront's own cross-sell recommendations, using parentHeadlinePrice, relatedPrice, and priceComparisonClass across a full collection in one run.

📊 AI and data teams

Index title, description, brand, categories, and tags for RAG over a product catalogue, or use the typed variant and price-comparison fields as clean training features — no HTML parsing step required.

📱 Market intelligence analysts

Track avgRelatedPriceDeltaPct and currencyMismatch across scheduled runs to catch pricing or market-expansion changes on a monitored storefront as they happen.

🔬 Researchers

Study product-recommendation structures and cross-sell pricing patterns across a sample of public Shopify storefronts using relatedCount and priceComparisonClass. Scope is limited to what each storefront publishes publicly.

🎥 Product and SaaS builders

Build a price-monitoring dashboard or catalogue-enrichment API on top of the dataset's type-keyed rows, without building and maintaining your own Shopify scraping and recommendation-fetch logic.

⚖️ Is It Legal to Scrape Shopify?

Scraping publicly accessible Shopify storefront data is generally lawful in most jurisdictions; this actor only reads endpoints (/products.json, /collections/*/products.json, /recommendations/products.json) that any visitor's browser can already request without logging in. That said, scraping may still violate an individual store's Terms of Service, which is a civil contract matter between you and the store, not a criminal one — review the target store's terms before large-scale or commercial use. This actor returns Shopify product catalogue data — titles, prices, variants, media, and recommendation relationships — not personal data about identifiable individuals, so data-protection frameworks like GDPR do not apply to its output; product and pricing data are instead governed by the store's own terms of service and, in some jurisdictions, database-rights law protecting the compiled catalogue. This actor returns only publicly accessible data. What you do with that data is your responsibility — consult legal counsel for commercial applications.

❓ Frequently Asked Questions

Does this Shopify scraper work without a Shopify account?

Yes. It reads a storefront's own public JSON endpoints directly — no Shopify login, Admin API key, or app installation is required for any input.

How does it handle Shopify's anti-scraping measures?

Every request starts direct. On a 403, 429, 503, or a connection/timeout error, the actor escalates automatically: first to an Apify datacenter proxy (group SHADER by default), then to a residential proxy if datacenter still gets blocked, retrying the residential attempt up to 3 times with exponential backoff before giving up on that request. A 429 response alone triggers a short wait-and-retry before any proxy escalation.

Can I run this at scale without getting blocked?

The actor fetches product pages with bounded concurrency per run and escalates through the proxy tiers described above when a storefront pushes back. There is no published uptime or success-rate figure for this behavior — actual outcomes depend on the target storefront's own blocking policy.

How fresh is the data this scraper returns?

Every run performs a live fetch against the target storefront at run time — nothing is cached between runs. scrapedAt on each row records the UTC timestamp of that fetch.

Which fields work best for AI training and RAG indexing?

For RAG, index title, description, brand, categories, and tags — the high-information text fields on each product row. For training data, variants[].price, variants[].sku, and priceComparisonClass are the most consistently structured fields across records. All fields return as typed JSON primitives — no string parsing or normalization needed before use.

Does scraping Shopify raise data-protection concerns?

No — this actor's output is product catalogue data (titles, prices, variants, media), not personal data about identifiable individuals, so GDPR/CCPA-style obligations do not attach to what it returns. Lawful basis for how you store and use the data still sits with you as the data controller for your own downstream use.

Does this scraper work with Claude, ChatGPT, and other AI agent tools?

It is callable as a standard Apify Actor via the Apify API from any agent framework that can make an HTTP request — every response is typed JSON, ready to drop into an LLM context window without a parsing step.

Why does productQuery / query do nothing?

Both are accepted on the input for compatibility with input JSON built for the base Shopify Scraper, but this variant does not use either to filter or search results — only the URLs in startUrls determine what gets scraped. This is disclosed here rather than hidden because the field will silently do nothing if you rely on it.

How does this compare to building the equivalent with Shopify's own APIs?

Shopify's Admin REST API has no dedicated "related products" resource; recommendations are only exposed via the Storefront GraphQL productRecommendations query (which needs a Storefront API access token per store) or the same public AJAX /recommendations/products.json endpoint this actor reads directly (shopify.dev, checked 2026-08-15). Building this yourself means provisioning API access per store and writing your own price-comparison math; this actor does both out of the box for any public storefront.

ℹ️ Disclaimer

This actor extracts only publicly available data from Shopify storefronts. It is intended for lawful use cases only. Users are responsible for complying with each target store's terms of service and applicable data protection laws in their jurisdiction.