# Shopify Product Monitor — MCP, Prices, Drop Alerts (`harvestlab/shopify-scraper`) Actor Scrape any Shopify store via /products.json — 99%+ run success. Feed RAG agents fresh products, prices, variants, inventory. $0.004/product vs Prisync $99-299/mo or Visualping $14/mo (Price2Spy/Pricer24/Wisepricer). MCP+n8n webhooks, AI via 5 LLMs, x402-ready. - **URL**: https://apify.com/harvestlab/shopify-scraper.md - **Developed by:** [Nick](https://apify.com/harvestlab) (community) - **Categories:** E-commerce, AI, MCP servers - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Shopify Store Scraper — Products, Prices, Drop Alerts & AI > 🧩 **Part of the harvestlab MCP suite** — 36 RAG-ready, AI-agent-payment-ready Apify actors covering ecommerce, social, travel, news, jobs, EU B2B, dev-tools, and government data. [See the full suite →](https://apify.com/harvestlab) Scrape complete product catalogs from any Shopify store using the public `/products.json` API. Extract titles, prices, variants, SKUs, inventory, images, discounts, and collection graphs in seconds — no browser automation, no blocked requests, no fragile CSS selectors. Optional price-watch mode tracks price changes across runs and fires Slack/Zapier webhook alerts when products drop. AI-powered catalog analysis delivers competitive positioning and pricing strategy insights on demand. ### What it does This actor pulls the complete product catalog from any Shopify-powered store by calling the store's public `/products.json` endpoint. Every Shopify store exposes this endpoint by default, returning structured JSON with full product, variant, and image data. No headless browser is required, and there are no anti-bot mechanisms to bypass — the request looks identical to a regular storefront visitor fetching product data. Each product comes back with full variant data including SKUs, prices, compare-at prices, inventory quantities, option values (size, color, material), and a derived `discount_percent` per variant. Prices are normalized from Shopify's string format to floats. The actor resolves each store's ISO currency code from `/meta.json`, the `X-Shopify-Shop-Currency` response header, or ccTLD inference, and emits it as a top-level `currency` field on every product. **Price-watch mode** (`watchMode: true`) persists each product's price history across runs in a named Apify key-value store (60-snapshot FIFO, approximately 2 months at daily cadence). On every subsequent run the actor computes `price_delta_pct`, direction (`up/down/flat`), and `first_seen_at` for every product. Add `alertWebhookUrl` and you get automatic fire-and-forget JSON POSTs to Slack, Zapier, n8n, or Make the moment a product drops past your `alertMinDropPct` threshold. This replaces a custom backend that would otherwise require a database, a cron job, and a webhook server. **Collection scraping** walks each store's `/collections.json` to enumerate categories and build a product-to-collection membership graph, complete with per-collection discount heatmaps. **Discounted-only filtering** lets you pull only sale items for arbitrage and reseller research. **AI catalog analysis** — via OpenRouter, Anthropic, Google AI, OpenAI, or Ollama — generates a structured report on pricing strategy, product gaps, and competitive positioning. The actor supports multi-store runs, processing each store sequentially with polite 1-second delays between requests and exponential backoff on rate limits. ### Features - **Public JSON API** — Uses Shopify's native `/products.json` and `/collections.json` endpoints. No browser, no HTML parsing, no fragile selectors that break on theme updates. - **Full product data** — Titles, handles, descriptions (HTML stripped to plain text), vendors, product types, tags, creation and update dates. - **Variant extraction** — Prices (as floats), compare-at prices, SKUs, inventory quantities, weight, option values (size, color, material), plus a derived `discount_percent` per variant. - **Image collection** — All product images with dimensions and alt text, plus featured image identification. - **Derived analytics** — Price ranges (min/max), average price, on-sale detection, variant counts, `has_variants` flag, `available_variant_count` (in-stock variants), `min_weight_g`/`max_weight_g` normalized to grams, `max_discount_percent`, `avg_discount_percent`, `compare_at_min_price`, `compare_at_max_price`. - **Storefront currency detection** — Resolves each store's ISO 4217 currency code from `/meta.json`, the `X-Shopify-Shop-Currency` response header, or ccTLD fallback (28 country codes mapped). - **Collections scraping** — Optional `/collections.json` scrape. Each collection (id, title, handle, description, products_count, image, URL) is emitted as a dataset item with `type="collection"`, ideal for category-navigation analysis. - **Discounted-only filter** — Set `discountedOnly: true` to receive only products with at least one variant on sale. Built for arbitrage, reseller research, and sale-intensity dashboards. Skipped products are not charged. - **Product-to-collection graph + discount heatmap** — Set `fetchCollectionProducts: true` to walk each collection's `/collections/{handle}/products.json` and build a membership graph. Every product gains a `collection_memberships` array and `collection_count`. Every collection gains a `discount_stats` block (product_count, on_sale_pct, avg/max discount percent, price range). No other Apify Shopify scraper exposes this graph. - **Price-watch mode + drop-alert webhooks** — Set `watchMode: true` to persist each product's min/max/avg price across runs. Every product receives a `price_watch` block with `{price_previous, price_now, price_delta_pct, direction, price_drop, first_seen_at, history_sample_count}`. Add `alertWebhookUrl` to POST JSON to Slack/Zapier/n8n/Make when a product drops by ≥ `alertMinDropPct`. No other Apify Shopify scraper ships cross-run price history plus webhooks. - **Multi-store support** — Scrape multiple Shopify stores in a single run. - **AI catalog analysis** — LLM-powered analysis of pricing strategy, product gaps, competitive positioning, and actionable recommendations. When graph mode is on, adds `collection_discount_heatmap` and `merchandising_signals`. - **Multi-LLM support** — Choose between OpenRouter (recommended — 300+ models, cheapest), Anthropic (Claude), Google AI (Gemini), OpenAI (GPT), or Ollama (self-hosted). - **Pay-per-event pricing** — Only pay for products actually scraped. Webhook failures are not charged. ### Use Cases #### Dropshippers and Product Researchers Discover winning products across Shopify stores in your niche. Compare pricing, identify trending items, and find suppliers by scanning vendor data across multiple stores in a single run. Export product catalogs to CSV for side-by-side comparison with your own inventory. #### Competitive Price Monitoring with Alerts Run the actor on a daily schedule with `watchMode: true` and `alertWebhookUrl` pointing to a Slack channel. The first run builds a baseline. From the second run onward, any product that drops by your configured percentage fires an instant Slack notification with the product title, old price, new price, and percentage change. No custom backend required. #### Market Research and Trend Analysis Build datasets of products across an entire market segment. Analyze pricing distributions, popular product types, common tags, and vendor landscapes. Use the AI-powered catalog analysis to identify underserved categories and pricing gaps without reviewing products manually. #### Sale and Arbitrage Research Enable `discountedOnly: true` to pull only products currently on sale. The actor emits full variant-level discount percentages and compare-at prices, making it easy to identify reseller arbitrage opportunities, clearance liquidation, and deep-discount patterns. #### E-commerce Data Aggregation Aggregate product data from multiple Shopify stores into a single dataset for marketplace platforms, comparison shopping engines, or product discovery tools. The structured JSON output integrates directly with databases, APIs, and data pipelines. Currency normalization means all prices are labeled with their ISO code regardless of the store's country. #### Inventory and Availability Tracking Monitor variant-level inventory data to detect stock levels, out-of-stock products, and restocking patterns. Useful for affiliate marketers who need to promote in-stock items and for resellers timing purchasing decisions based on inventory signals. #### Collection Category Analysis Enable `fetchCollectionProducts: true` to build the full product-to-collection membership graph. This exposes which collections are deep-discount clearance zones versus full-price premium ranges — merchandising intelligence that normally requires manual store browsing or a Shopify partner account. ### Input | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `storeUrls` | array | required | List of Shopify store URLs to scrape (e.g. `https://kith.com`) | | `maxProducts` | integer | 100 | Max products per store (1–500) | | `includeVariants` | boolean | true | Include variant data (prices, SKUs, inventory, options) | | `includeImages` | boolean | true | Include product image URLs with dimensions | | `includeCollections` | boolean | false | Also scrape `/collections.json` for each store | | `maxCollections` | integer | 100 | Max collections per store (1–250) | | `discountedOnly` | boolean | false | Only emit products with at least one on-sale variant | | `fetchCollectionProducts` | boolean | false | Build product-to-collection graph + per-collection discount heatmap | | `maxProductsPerCollection` | integer | 250 | Max products walked per collection in graph mode (1–1000) | | `watchMode` | boolean | false | Track price changes across runs using a named KV store | | `alertWebhookUrl` | string | — | HTTPS webhook URL for price-drop alerts (auto-enables `watchMode`) | | `alertMinDropPct` | integer | 5 | Minimum price drop % to trigger a webhook alert | | `enableAiAnalysis` | boolean | false | Run AI-powered catalog analysis | | `llmProvider` | string | openrouter | LLM provider: `openrouter`, `anthropic`, `google`, `openai`, `ollama` | | `llmModel` | string | (auto) | Override the provider's default model | | `openrouterApiKey` | string | — | OpenRouter API key (get one free at openrouter.ai/keys) | | `anthropicApiKey` | string | — | Anthropic API key | | `googleApiKey` | string | — | Google AI (Gemini) API key | | `openaiApiKey` | string | — | OpenAI API key | | `ollamaBaseUrl` | string | http://localhost:11434 | Ollama base URL for self-hosted LLM | | `proxyConfiguration` | object | Apify Proxy | Proxy settings (datacenter is sufficient for most Shopify stores) | **CLI aliases accepted:** `shopUrl`, `storeUrl`, `url` (single URL); `shopUrls` (array); `maxItems` (alias for maxProducts). ### Output Each product is output as a separate dataset item. Collections and AI analysis are emitted as additional items with a `type` field distinguishing them. **Product item:** ```json { "store_url": "https://hydrogen-preview.myshopify.com", "product_id": 6730943955025, "title": "The Collection Snowboard: Hydrogen", "handle": "the-collection-snowboard-hydrogen", "vendor": "Hydrogen Vendor", "product_type": "Snowboard", "tags": ["Premium", "Snow", "Winter"], "description": "A high-performance snowboard designed for experienced riders.", "created_at": "2023-01-01T00:00:00-05:00", "updated_at": "2024-06-15T12:30:00-05:00", "options": [{"name": "Size", "position": 1, "values": ["154cm", "158cm", "162cm"]}], "currency": "USD", "min_price": 599.95, "max_price": 749.95, "average_price": 674.95, "on_sale": true, "max_discount_percent": 14.29, "avg_discount_percent": 14.29, "compare_at_min_price": 699.95, "compare_at_max_price": 749.95, "total_variants": 3, "has_variants": true, "available_variant_count": 2, "min_weight_g": 3200.0, "max_weight_g": 3500.0, "url": "https://hydrogen-preview.myshopify.com/products/the-collection-snowboard-hydrogen", "scraped_at": "2026-04-22T10:30:00+00:00", "variants": [ { "id": 39888274382929, "title": "154cm", "price": 599.95, "compare_at_price": 699.95, "discount_percent": 14.29, "sku": "SNOW-HYD-154", "available": true, "inventory_quantity": 12, "weight": 3200.0, "weight_unit": "g", "option1": "154cm", "option2": null, "option3": null } ], "images": [ { "id": 28505021718609, "src": "https://cdn.shopify.com/s/files/1/0551/4566/0577/products/snowboard.jpg", "width": 1200, "height": 800, "alt": "Hydrogen Snowboard" } ], "featured_image": "https://cdn.shopify.com/s/files/1/0551/4566/0577/products/snowboard.jpg" } ```` **Price-watch block** (added to each product when `watchMode: true`): ```json { "price_watch": { "price_previous": 49.00, "price_now": 29.00, "price_delta_pct": -40.82, "direction": "down", "price_drop": true, "first_seen_at": "2026-04-01T00:00:00+00:00", "history_sample_count": 14 } } ``` **Collection item** (when `includeCollections: true`): ```json { "type": "collection", "store_url": "https://hydrogen-preview.myshopify.com", "collection_id": 387214442552, "title": "Backcountry Collection", "handle": "backcountry", "description": "Go further with our most technologically advanced boards.", "products_count": 26, "url": "https://hydrogen-preview.myshopify.com/collections/backcountry", "scraped_at": "2026-04-22T10:30:00+00:00" } ``` **AI analysis item** (when `enableAiAnalysis: true`): ```json { "type": "ai_analysis", "store_url": "https://hydrogen-preview.myshopify.com", "total_products_analyzed": 50, "analysis": { "store_overview": "A specialty winter sports retailer focused on premium snowboards.", "pricing_strategy": "Premium pricing with selective discounting on older SKUs.", "competitive_positioning": "Premium segment targeting experienced riders.", "product_gaps": "No entry-level boards, limited accessories.", "recommendations": ["Expand into beginner boards", "Add bundle pricing"] }, "llm_provider": "openrouter", "llm_model": "google/gemini-2.0-flash-001", "analyzed_at": "2026-04-22T10:35:00+00:00" } ``` **Pay-per-event pricing:** | Event | Price | Description | |-------|-------|-------------| | `product-scraped` | $0.004 | Per product successfully extracted | | `collection-scraped` | $0.001 | Per collection extracted (when `includeCollections=true`) | | `price-drop-detected` | $0.005 | Per product whose price fell ≥ `alertMinDropPct` (watchMode only; first-run products not charged) | | `alert-dispatched` | $0.002 | Per price-drop webhook successfully delivered (2xx response) | | `ai-analysis-completed` | $0.05 | Per store catalog analyzed by LLM | **vs. commercial alternatives**: Shopify Partner API requires store-owner approval per store; DataForSEO e-commerce endpoints charge $1.50/1,000 requests; Prisync charges $99/mo for competitor price tracking. This actor uses pay-per-event with no subscription — $0.002/product and zero monthly fees. ### Quick Start **Scrape a single store:** ```json { "storeUrls": ["https://hydrogen-preview.myshopify.com"] } ``` **Scrape multiple stores with a product limit:** ```json { "storeUrls": [ "https://hydrogen-preview.myshopify.com", "https://shop.example.com" ], "maxProducts": 200, "includeVariants": true, "includeImages": true } ``` **Sale-hunting mode — only discounted products:** ```json { "storeUrls": ["https://shop.example.com"], "maxProducts": 200, "discountedOnly": true, "includeCollections": true, "maxCollections": 50 } ``` **Price-watch + drop alerts (schedule this daily):** ```json { "storeUrls": ["https://shop.example.com"], "maxProducts": 200, "watchMode": true, "alertWebhookUrl": "https://hooks.slack.com/services/T000/B000/XXX", "alertMinDropPct": 10 } ``` The first run builds the history baseline — every product emits `price_watch.direction: "new"` and no alert fires. From the second run onward, the actor compares each product's current min\_price against the prior snapshot and fires a webhook for every product that drops by at least 10%. Webhook payload shape: ```json { "store_domain": "shop.example.com", "product_id": 1234567890, "title": "Classic Tee", "handle": "classic-tee", "price_previous": 49.00, "price_now": 29.00, "price_delta_pct": -40.82, "direction": "down", "currency": "USD", "on_sale": true, "url": "https://shop.example.com/products/classic-tee", "scraped_at": "2026-04-22T23:42:00+00:00" } ``` Works with Slack Incoming Webhooks, Zapier, Make/Integromat, n8n, or any HTTP endpoint that accepts JSON POSTs. **AI-powered catalog analysis:** ```json { "storeUrls": ["https://hydrogen-preview.myshopify.com"], "maxProducts": 100, "enableAiAnalysis": true, "llmProvider": "openrouter", "openrouterApiKey": "sk-or-v1-your-key-here" } ``` **Collection graph + AI merchandising signals:** ```json { "storeUrls": ["https://hydrogen-preview.myshopify.com"], "maxProducts": 200, "fetchCollectionProducts": true, "maxCollections": 30, "enableAiAnalysis": true, "llmProvider": "openrouter", "openrouterApiKey": "sk-or-v1-your-key-here" } ``` ### Use with AI agents shopify-scraper output is agent-ready: typed product dicts with variants, SKUs, inventory, currency, `price_watch` deltas, and a product-to-collection graph — pulled straight from Shopify's public `/products.json` endpoint, no browser, no anti-bot guesswork. Wrap it as a single tool and your DTC research, brand-protection, or arbitrage agent gets cross-store catalog + drop-alert intelligence at $0.004 per product instead of $99-299/mo Prisync or Visualping seats. RAG-ready JSON pairs cleanly with `alertWebhookUrl` for n8n / Zapier / Slack drop alerts. See Apify's [`actor-templates`](https://github.com/apify/actor-templates) — specifically [`js-langchain`](https://github.com/apify/actor-templates/tree/master/templates/js-langchain) and [`js-langgraph-agent`](https://github.com/apify/actor-templates/tree/master/templates/js-langgraph-agent) — for end-to-end skeletons. #### LangChain tool wrapper ```python from langchain_core.tools import Tool from apify_client import ApifyClient client = ApifyClient("YOUR_APIFY_TOKEN") def _track_shopify_store(args: dict) -> list: run = client.actor("harvestlab/shopify-scraper").call(run_input={ "storeUrls": args["storeUrls"], "watchMode": args.get("watchMode", True), "alertWebhookUrl": args.get("alertWebhookUrl"), "alertMinDropPct": args.get("alertMinDropPct", 5), "fetchCollectionProducts": True, }) return list(client.dataset(run["defaultDatasetId"]).iterate_items()) track_shopify_store = Tool( name="track_shopify_store", description="Scrape Shopify stores via /products.json — variants, inventory, price drops, collection graph. Input: {storeUrls: [str], watchMode: bool, alertWebhookUrl: str, alertMinDropPct: int}.", func=_track_shopify_store, ) ## track_shopify_store.invoke({"storeUrls": ["https://hydrogen-preview.myshopify.com"], "watchMode": True, "alertWebhookUrl": "https://hooks.slack.com/services/T000/B000/XXX"}) ``` #### LangGraph arbitrage / brand-protection node ```python from langgraph.graph import StateGraph, END from typing import TypedDict from apify_client import ApifyClient client = ApifyClient("YOUR_APIFY_TOKEN") class StoreState(TypedDict): storeUrls: list[str] alertMinDropPct: int products: list[dict] drops: list[dict] def track_node(state: StoreState) -> StoreState: run = client.actor("harvestlab/shopify-scraper").call(run_input={ "storeUrls": state["storeUrls"], "watchMode": True, "alertMinDropPct": state["alertMinDropPct"], "discountedOnly": False, }) items = list(client.dataset(run["defaultDatasetId"]).iterate_items()) drops = [i for i in items if (i.get("price_watch") or {}).get("price_drop")] return {**state, "products": items, "drops": drops} graph = StateGraph(StoreState) graph.add_node("track", track_node) graph.set_entry_point("track") graph.add_edge("track", END) ## graph.compile().invoke({"storeUrls": ["https://shop.example.com"], "alertMinDropPct": 10, "products": [], "drops": []}) ``` ### Troubleshooting #### Store URL returns 404 on /products.json The target site is not a Shopify store, or the store owner has disabled the public products endpoint. Verify by opening `https://DOMAIN/products.json` in a browser — a Shopify store returns a JSON object with a `products` array. A 404 or HTML page means the site uses a different platform (WooCommerce, Magento, BigCommerce) or has explicitly restricted the endpoint. The actor logs the error and skips that store without failing the entire run. #### Store is password-protected and returns no products Shopify stores in development mode show a login page at `/password` before any other route. The `/products.json` endpoint returns an empty response or a redirect in this state. There is no workaround — password-protected stores do not expose the public JSON API. Wait until the store goes live or contact the store owner for access. #### Product variants appear incomplete or prices show null Shopify's `/products.json` returns all published variant data by default. If variant prices appear as `null`, the store may be running a headless Shopify setup with a custom storefront that intercepts the request before it reaches Shopify's default API handler. This is rare and cannot be worked around via the public endpoint. Inventory quantities can also be hidden by the store owner via Shopify admin settings. #### maxProducts cap hit but the store has more products Shopify paginates `/products.json` at 250 items per page. The actor's hard cap is 500 products per store per run. If the store has more than 500 products, use `fetchCollectionProducts: true` to walk per-collection endpoints — large catalogs are often split across collections, each with fewer than 500 items. #### Price-watch webhook not firing after a price change Alert webhooks require all four conditions simultaneously: (1) `watchMode: true`, (2) a valid `alertWebhookUrl` returning a 2xx response, (3) the same store URL used across at least 2 runs so a prior baseline exists, and (4) the price delta meets or exceeds `alertMinDropPct`. The first run always seeds the baseline silently — no alert fires. If the webhook URL returns anything other than 2xx, the `alert-dispatched` event is not charged and the failure is logged. #### AI analysis returns an error about missing API key Provide the API key either in the actor input field (e.g. `openrouterApiKey`) or as an environment variable (`OPENROUTER_API_KEY`, `ANTHROPIC_API_KEY`, `GOOGLE_API_KEY`, or `OPENAI_API_KEY`). Set `enableAiAnalysis: false` to skip AI entirely — the product scrape completes normally without it. #### No products scraped from any store Check the run log for HTTP errors. Common causes: (1) the URLs are not Shopify stores, (2) the stores have disabled `/products.json`, (3) aggressive rate limiting — try enabling the Apify proxy in the proxy settings field. Datacenter proxies are sufficient for the vast majority of Shopify stores. ### Legal and Compliance This actor accesses Shopify stores through the publicly available `/products.json` API endpoint. This is a standard, documented Shopify feature that returns product data intended for public consumption — the same data visible on the storefront without authentication. No Terms of Service restrictions apply to reading publicly exposed JSON endpoints. **User Responsibility.** You are responsible for ensuring your use of this actor complies with all applicable laws and regulations, including data protection laws such as GDPR, CCPA, and other privacy regulations. If you collect data that may include personal information (e.g. vendor names that identify individuals), ensure you have a lawful basis for processing it. **Data Handling.** Product data scraped by this actor is stored in your Apify dataset. You control the retention, export, and deletion of this data through the Apify platform. The price-watch KV store (`shopify-price-history`) persists across runs and can be manually deleted from your Apify account storage at any time. **Rate Limiting.** The actor implements polite crawling with 1-second delays between requests and exponential backoff on rate limit responses (HTTP 429/403). This prevents excessive load on store servers. **Third-party review widgets** (Yotpo, Judge.me, Stamped.io, Okendo) are not scraped. These platforms have separate Terms of Service and often require authentication. ### Pair with amazon-scraper Shopify Scraper covers direct-to-consumer Shopify stores; [Amazon Scraper](https://apify.com/harvestlab/amazon-scraper) covers the marketplace side of the same SKUs. Run both on a daily schedule with `watchMode: true` to build a unified cross-channel price-history dataset: same product, two channels, side-by-side delta. Common workflows: - **Brand price-policing** — detect when a Shopify brand's product is undercut on Amazon by a third-party seller. Pipe both datasets into a single Slack channel via `alertWebhookUrl`. - **Reseller arbitrage** — pull `discountedOnly: true` from Shopify, cross-reference Amazon Buy Box price, surface profitable flips. - **MAP enforcement** — manufacturers track Minimum Advertised Price compliance across both channels in one pipeline. Both actors share the canonical `url` and `currency` output fields, so a join on product handle or SKU is one query. ### Related Actors - [Amazon Scraper](https://apify.com/harvestlab/amazon-scraper) — Scrape product data, prices, reviews, and Buy Box across 10+ Amazon domains - [Website Contact Extractor](https://apify.com/harvestlab/contact-extractor) — Extract emails, social profiles, and tech stack from any website - [Trustpilot Scraper](https://apify.com/harvestlab/trustpilot-scraper) — Scrape reviews and ratings from Trustpilot business profiles - [Bol.com Scraper](https://apify.com/harvestlab/bol-com-scraper) — Scrape product data from the largest Dutch e-commerce marketplace - [Grocery Price Monitor](https://apify.com/harvestlab/grocery-price-monitor) — Track grocery prices across major supermarket chains - [Review Analyzer](https://apify.com/harvestlab/review-analyzer) — AI-powered sentiment analysis for product and business reviews # Actor input Schema ## `storeUrls` (type: `array`): List of Shopify store URLs to scrape (e.g. https://kith.com, https://hydrogen-preview.myshopify.com). The actor auto-detects the /products.json endpoint. Accepts aliases: 'shopUrls' (array), 'shopUrl'/'storeUrl'/'url' (single URL). ## `shopUrl` (type: `string`): CLI alias for storeUrls (single URL). Hidden from Console form. ## `storeUrl` (type: `string`): CLI alias for shopUrl/storeUrls (single URL). Hidden from Console form. ## `url` (type: `string`): CLI alias for shopUrl/storeUrls (single URL). Hidden from Console form. ## `shopUrls` (type: `array`): CLI alias for storeUrls (array of URLs). Hidden from Console form. ## `maxProducts` (type: `integer`): Maximum number of products to scrape per store (1-500). Shopify stores can have thousands of products — start small to test. Accepts alias: 'maxItems'. ## `maxItems` (type: `integer`): CLI alias for maxProducts. Hidden from Console form. ## `includeVariants` (type: `boolean`): Include all product variants (sizes, colors, etc.) with individual pricing and inventory data. Recommended for price monitoring. ## `includeImages` (type: `boolean`): Include product image URLs in the output (primary image plus gallery images). ## `includeCollections` (type: `boolean`): Also scrape the store's public /collections.json endpoint. Each collection (title, handle, description, products\_count, image) is emitted as a dataset item with type='collection'. Useful for category-mapping and navigation analysis. ## `maxCollections` (type: `integer`): Maximum number of collections to scrape per store when includeCollections=true (1-250). Shopify paginates at 250 per page. ## `discountedOnly` (type: `boolean`): If true, only emit products that have at least one variant on sale (compare\_at\_price > price). Useful for bargain-hunting, reseller arbitrage, and sale-tracking workflows. ## `fetchCollectionProducts` (type: `boolean`): If true, walk each collection's /collections/{handle}/products.json endpoint to build a product-to-collection membership graph. Every product gets a 'collection\_memberships' array ({collection\_handle, collection\_title, position}) and 'collection\_count'. Every collection gets a 'discount\_stats' block (per-collection discount heatmap). When AI analysis is enabled, adds 'collection\_discount\_heatmap' and 'merchandising\_signals' to the output. Auto-enables includeCollections. Cost: one extra HTTP per collection — no existing Apify Shopify competitor exposes this graph. ## `maxProductsPerCollection` (type: `integer`): When fetchCollectionProducts=true, cap the number of products fetched per individual collection (1-1000). Default 250 covers most mid-size collections in a single API call. ## `watchMode` (type: `boolean`): Persist each product's min/max/avg price across runs in a named Apify key-value store (shopify-price-history). Every product receives a `price_watch` block with {price\_previous, price\_now, price\_delta\_pct, direction (up/down/flat), price\_drop, first\_seen\_at, history\_sample\_count}. 60-snapshot FIFO per product (~2 months at daily cadence). Bills the `price-drop-detected` event ($0.005) when a product's price falls by ≥ alertMinDropPct (default 5%). First-run products emit direction='new' and are not billed. ## `alertWebhookUrl` (type: `string`): HTTPS webhook URL that receives a JSON POST for every product whose price falls ≥ alertMinDropPct. Payload: {store\_domain, product\_id, title, handle, vendor, price\_previous, price\_now, price\_delta\_pct, direction, currency, on\_sale, url}. Works with Slack Incoming Webhooks, Zapier, Make/Integromat, n8n, or any HTTP endpoint. Fire-and-forget with 5s timeout + 1 retry — webhook failures never fail the scraper. Setting this auto-enables watchMode. Bills `alert-dispatched` ($0.002) per successfully delivered webhook. ## `alertMinDropPct` (type: `integer`): Minimum price drop (in percent) required to fire a webhook and bill price-drop-detected. Default 5% — fires when a product's min\_price falls at least 5% vs. its last snapshot. Set higher (e.g. 10) for fewer, bigger alerts, or lower (e.g. 2) to catch smaller moves. Requires watchMode=true. ## `enableAiAnalysis` (type: `boolean`): Run AI-powered analysis on the store's product catalog. Provides pricing strategy insights, product gap analysis, and competitive positioning. Requires an API key. ## `llmProvider` (type: `string`): Choose the LLM provider for AI analysis. OpenRouter is cheapest. ## `llmModel` (type: `string`): Specific model to use. Leave empty for the provider default (google/gemini-2.0-flash-001 for OpenRouter, claude-sonnet-4-20250514 for Anthropic, gemini-2.0-flash for Google AI, gpt-4o-mini for OpenAI, llama3.1 for Ollama). ## `openrouterApiKey` (type: `string`): API key for OpenRouter. Get one free at openrouter.ai/keys. ## `anthropicApiKey` (type: `string`): API key for Anthropic. Get one at console.anthropic.com. ## `googleApiKey` (type: `string`): API key for Google AI (Gemini). Get one at aistudio.google.com/app/apikey. ## `openaiApiKey` (type: `string`): API key from platform.openai.com. Required if using OpenAI provider. ## `ollamaBaseUrl` (type: `string`): Base URL for Ollama API (self-hosted). Default: http://localhost:11434. ## `proxyConfiguration` (type: `object`): Apify proxy configuration. Shopify stores generally work without residential proxies — datacenter proxies are sufficient. ## Actor input object example ```json { "storeUrls": [ "https://hydrogen-preview.myshopify.com" ], "maxProducts": 100, "includeVariants": true, "includeImages": true, "includeCollections": false, "maxCollections": 100, "discountedOnly": false, "fetchCollectionProducts": false, "maxProductsPerCollection": 250, "watchMode": false, "alertMinDropPct": 5, "enableAiAnalysis": false, "llmProvider": "openrouter", "ollamaBaseUrl": "http://localhost:11434", "proxyConfiguration": { "useApifyProxy": true } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "storeUrls": [ "https://hydrogen-preview.myshopify.com" ], "ollamaBaseUrl": "http://localhost:11434", "proxyConfiguration": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("harvestlab/shopify-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "storeUrls": ["https://hydrogen-preview.myshopify.com"], "ollamaBaseUrl": "http://localhost:11434", "proxyConfiguration": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("harvestlab/shopify-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "storeUrls": [ "https://hydrogen-preview.myshopify.com" ], "ollamaBaseUrl": "http://localhost:11434", "proxyConfiguration": { "useApifyProxy": true } }' | apify call harvestlab/shopify-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=harvestlab/shopify-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Shopify Product Monitor — MCP, Prices, Drop Alerts", "description": "Scrape any Shopify store via /products.json — 99%+ run success. Feed RAG agents fresh products, prices, variants, inventory. $0.004/product vs Prisync $99-299/mo or Visualping $14/mo (Price2Spy/Pricer24/Wisepricer). MCP+n8n webhooks, AI via 5 LLMs, x402-ready.", "version": "1.7", "x-build-id": "mESVHNa49Jc7Cg62l" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/harvestlab~shopify-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-harvestlab-shopify-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/harvestlab~shopify-scraper/runs": { "post": { "operationId": "runs-sync-harvestlab-shopify-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/harvestlab~shopify-scraper/run-sync": { "post": { "operationId": "run-sync-harvestlab-shopify-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "storeUrls": { "title": "Shopify Store URLs", "type": "array", "description": "List of Shopify store URLs to scrape (e.g. https://kith.com, https://hydrogen-preview.myshopify.com). The actor auto-detects the /products.json endpoint. Accepts aliases: 'shopUrls' (array), 'shopUrl'/'storeUrl'/'url' (single URL).", "items": { "type": "string" } }, "shopUrl": { "title": "Single Store URL (CLI alias)", "type": "string", "description": "CLI alias for storeUrls (single URL). Hidden from Console form." }, "storeUrl": { "title": "Store URL (CLI alias)", "type": "string", "description": "CLI alias for shopUrl/storeUrls (single URL). Hidden from Console form." }, "url": { "title": "URL (CLI alias)", "type": "string", "description": "CLI alias for shopUrl/storeUrls (single URL). Hidden from Console form." }, "shopUrls": { "title": "Shop URLs (CLI alias)", "type": "array", "description": "CLI alias for storeUrls (array of URLs). Hidden from Console form.", "items": { "type": "string" } }, "maxProducts": { "title": "Max Products Per Store", "minimum": 1, "maximum": 500, "type": "integer", "description": "Maximum number of products to scrape per store (1-500). Shopify stores can have thousands of products — start small to test. Accepts alias: 'maxItems'.", "default": 100 }, "maxItems": { "title": "Max Items (CLI alias)", "minimum": 1, "maximum": 1000, "type": "integer", "description": "CLI alias for maxProducts. Hidden from Console form." }, "includeVariants": { "title": "Include Variants", "type": "boolean", "description": "Include all product variants (sizes, colors, etc.) with individual pricing and inventory data. Recommended for price monitoring.", "default": true }, "includeImages": { "title": "Include Images", "type": "boolean", "description": "Include product image URLs in the output (primary image plus gallery images).", "default": true }, "includeCollections": { "title": "Include Collections", "type": "boolean", "description": "Also scrape the store's public /collections.json endpoint. Each collection (title, handle, description, products_count, image) is emitted as a dataset item with type='collection'. Useful for category-mapping and navigation analysis.", "default": false }, "maxCollections": { "title": "Max Collections Per Store", "minimum": 1, "maximum": 250, "type": "integer", "description": "Maximum number of collections to scrape per store when includeCollections=true (1-250). Shopify paginates at 250 per page.", "default": 100 }, "discountedOnly": { "title": "Discounted Products Only", "type": "boolean", "description": "If true, only emit products that have at least one variant on sale (compare_at_price > price). Useful for bargain-hunting, reseller arbitrage, and sale-tracking workflows.", "default": false }, "fetchCollectionProducts": { "title": "Build Product-to-Collection Graph", "type": "boolean", "description": "If true, walk each collection's /collections/{handle}/products.json endpoint to build a product-to-collection membership graph. Every product gets a 'collection_memberships' array ({collection_handle, collection_title, position}) and 'collection_count'. Every collection gets a 'discount_stats' block (per-collection discount heatmap). When AI analysis is enabled, adds 'collection_discount_heatmap' and 'merchandising_signals' to the output. Auto-enables includeCollections. Cost: one extra HTTP per collection — no existing Apify Shopify competitor exposes this graph.", "default": false }, "maxProductsPerCollection": { "title": "Max Products Per Collection (graph mode)", "minimum": 1, "maximum": 1000, "type": "integer", "description": "When fetchCollectionProducts=true, cap the number of products fetched per individual collection (1-1000). Default 250 covers most mid-size collections in a single API call.", "default": 250 }, "watchMode": { "title": "Price-Watch Mode (track price changes across runs)", "type": "boolean", "description": "Persist each product's min/max/avg price across runs in a named Apify key-value store (shopify-price-history). Every product receives a `price_watch` block with {price_previous, price_now, price_delta_pct, direction (up/down/flat), price_drop, first_seen_at, history_sample_count}. 60-snapshot FIFO per product (~2 months at daily cadence). Bills the `price-drop-detected` event ($0.005) when a product's price falls by ≥ alertMinDropPct (default 5%). First-run products emit direction='new' and are not billed.", "default": false }, "alertWebhookUrl": { "title": "Alert Webhook URL (POST on price drop)", "type": "string", "description": "HTTPS webhook URL that receives a JSON POST for every product whose price falls ≥ alertMinDropPct. Payload: {store_domain, product_id, title, handle, vendor, price_previous, price_now, price_delta_pct, direction, currency, on_sale, url}. Works with Slack Incoming Webhooks, Zapier, Make/Integromat, n8n, or any HTTP endpoint. Fire-and-forget with 5s timeout + 1 retry — webhook failures never fail the scraper. Setting this auto-enables watchMode. Bills `alert-dispatched` ($0.002) per successfully delivered webhook." }, "alertMinDropPct": { "title": "Alert Minimum Drop %", "minimum": 1, "maximum": 90, "type": "integer", "description": "Minimum price drop (in percent) required to fire a webhook and bill price-drop-detected. Default 5% — fires when a product's min_price falls at least 5% vs. its last snapshot. Set higher (e.g. 10) for fewer, bigger alerts, or lower (e.g. 2) to catch smaller moves. Requires watchMode=true.", "default": 5 }, "enableAiAnalysis": { "title": "Enable AI Analysis", "type": "boolean", "description": "Run AI-powered analysis on the store's product catalog. Provides pricing strategy insights, product gap analysis, and competitive positioning. Requires an API key.", "default": false }, "llmProvider": { "title": "LLM Provider", "enum": [ "openrouter", "anthropic", "google", "openai", "ollama" ], "type": "string", "description": "Choose the LLM provider for AI analysis. OpenRouter is cheapest.", "default": "openrouter" }, "llmModel": { "title": "LLM Model Override", "type": "string", "description": "Specific model to use. Leave empty for the provider default (google/gemini-2.0-flash-001 for OpenRouter, claude-sonnet-4-20250514 for Anthropic, gemini-2.0-flash for Google AI, gpt-4o-mini for OpenAI, llama3.1 for Ollama)." }, "openrouterApiKey": { "title": "OpenRouter API Key", "type": "string", "description": "API key for OpenRouter. Get one free at openrouter.ai/keys." }, "anthropicApiKey": { "title": "Anthropic API Key", "type": "string", "description": "API key for Anthropic. Get one at console.anthropic.com." }, "googleApiKey": { "title": "Google AI API Key", "type": "string", "description": "API key for Google AI (Gemini). Get one at aistudio.google.com/app/apikey." }, "openaiApiKey": { "title": "OpenAI API Key", "type": "string", "description": "API key from platform.openai.com. Required if using OpenAI provider." }, "ollamaBaseUrl": { "title": "Ollama Base URL", "type": "string", "description": "Base URL for Ollama API (self-hosted). Default: http://localhost:11434." }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "Apify proxy configuration. Shopify stores generally work without residential proxies — datacenter proxies are sufficient." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```