# Shopify Store Intelligence Scraper - Products, Leads & Apps (`elliotpadfield/shopify-store-intelligence-scraper`) Actor Scrape Shopify stores for product catalogs, prices, variants, collections, contact details, social links, theme/app signals, pages, and store-level ecommerce intelligence. - **URL**: https://apify.com/elliotpadfield/shopify-store-intelligence-scraper.md - **Developed by:** [Elliot Padfield](https://apify.com/elliotpadfield) (community) - **Categories:** E-commerce, Lead generation, Automation - **Stats:** 1 total users, 0 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $2.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Shopify Store Intelligence Scraper - Products, Leads & Apps Scrape Shopify stores for product catalogs, prices, variants, collections, contact details, social links, theme/app signals, useful pages, and store-level ecommerce intelligence. This Actor uses Shopify's public storefront endpoints plus homepage and sitemap parsing, so it is fast, structured, and does not require a Shopify API key. Use it for ecommerce competitor research, DTC lead generation, price monitoring, product catalog exports, brand enrichment, app-stack research, supplier discovery, and automated Shopify store tracking. ### What can this Shopify scraper do? - Scrape one or many Shopify stores by URL or domain - Extract product catalogs from `/products.json` - Extract product prices, compare-at prices, variants, options, availability, SKUs, tags, vendors, types, images, and descriptions - Extract collections from `/collections.json` - Save a store-level summary row with catalog statistics - Detect common Shopify apps and marketing tools from storefront HTML - Detect theme name, theme ID, locale, country, currency, and myshopify.com domain when available - Extract social links, emails, and phone numbers from the homepage and useful pages - Discover contact, about, wholesale, affiliate, privacy, and policy pages from the Shopify sitemap - Export structured ecommerce data to Apify datasets - Run on schedules for store monitoring - Use Apify residential proxies on every run for production reliability ### What data can you extract? | Field | Description | |---|---| | `itemType` | `store`, `product`, `collection`, or `page` | | `storeUrl` | Normalized Shopify store URL | | `storeDomain` | Store domain | | `shopifyDomain` | myshopify.com domain when visible | | `storeName` | Store or site name | | `currency`, `country`, `locale` | Storefront metadata when available | | `themeName`, `themeId`, `themeRole` | Shopify theme signals from storefront HTML | | `detectedApps` | Detected tools such as Klaviyo, Gorgias, Yotpo, Loox, Okendo, Recharge, Attentive, Postscript, Judge.me, Elevar, and more | | `emails`, `phones`, `socialUrls` | Public contact and social links | | `warnings`, `failedSections` | Optional sections that failed while the Actor still saved available data | | `productCount`, `collectionCount`, `totalVariants` | Store-level catalog statistics | | `productId`, `productUrl`, `title`, `handle` | Product identity fields | | `vendor`, `productType`, `tags` | Product classification fields | | `descriptionText`, `descriptionHtml` | Clean or raw product description | | `minPrice`, `maxPrice`, `compareAtMinPrice`, `onSale` | Product pricing fields | | `available`, `availableVariants`, `variantsCount` | Product availability fields | | `imageUrls`, `featuredImageUrl` | Product images | | `options`, `variants` | Product option and variant data | | `collectionUrl`, `productsCount` | Collection fields | | `pageUrl`, `text` | Useful page extraction | | `scrapedAt` | Timestamp when the row was saved | ### How to scrape Shopify stores 1. Add one or more Shopify store URLs or domains. 2. Choose whether to save store summaries, products, collections, and pages. 3. Set `maxProductsPerStore`, `maxCollectionsPerStore`, and `maxPagesPerStore` for test or production runs. 4. Keep `includeVariants` enabled if you need SKU, option, price, and stock-level detail. 5. Run the Actor and export the dataset in JSON, CSV, Excel, XML, RSS, or HTML from Apify. ### Input examples #### Scrape a Shopify store with products and lead signals ```json { "storeUrls": ["https://www.allbirds.com"], "includeStoreSummary": true, "includeProducts": true, "includeCollections": true, "includePages": true, "maxProductsPerStore": 250, "maxCollectionsPerStore": 250, "maxPagesPerStore": 10 } ```` #### Fast store enrichment without product rows ```json { "storeUrls": [ "https://www.allbirds.com", "https://www.gymshark.com" ], "includeStoreSummary": true, "includeProducts": false, "includeCollections": false, "includePages": true } ``` #### Full catalog export with variants ```json { "storeUrls": ["https://www.example-store.com"], "includeProducts": true, "includeVariants": true, "includeHtmlDescription": false, "maxProductsPerStore": 5000 } ``` ### Output example ```json { "itemType": "product", "storeUrl": "https://www.allbirds.com", "storeDomain": "www.allbirds.com", "productId": 6616124981328, "productUrl": "https://www.allbirds.com/products/trino-cozy-crew-heathered-onyx", "title": "Trino Cozy Crew - Heathered Onyx", "handle": "trino-cozy-crew-heathered-onyx", "vendor": "Allbirds", "productType": "Socks", "tags": ["collection:apr26", "shoprunner"], "minPrice": 16, "maxPrice": 16, "currency": "USD", "available": true, "availableVariants": 4, "variantsCount": 4, "featuredImageUrl": "https://cdn.shopify.com/s/files/...", "scrapedAt": "2026-05-28T11:20:00.000Z" } ``` ### Search methods and filters | Capability | Supported | |---|---| | Store URL/domain input | Yes | | Multi-store runs | Yes | | Product catalog extraction | Yes | | Collection extraction | Yes | | Store summary row | Yes | | Contact page discovery | Yes | | Emails and phone numbers | Yes | | Social links | Yes | | Product variants | Yes | | Product availability | Yes | | Compare-at pricing and sale flag | Yes | | Product images | Yes | | Product descriptions | Yes | | Theme detection | Yes | | App/tool detection | Yes | | myshopify.com domain detection | Yes | | Forced Apify Residential Proxy | Yes | ### Pricing This Actor is designed for pay-per-result pricing. Each saved dataset item is one billable result. | Result type | What counts as one result | |---|---| | Store summary | One saved store-level intelligence row | | Product | One saved Shopify product row | | Collection | One saved collection row | | Page | One saved useful page row | For small tests, set `maxProductsPerStore` to 10 or disable product rows and run store-summary enrichment only. The Actor stops saving new rows when the Apify pay-per-result charge limit is reached. The Actor always uses Apify residential proxies. For scheduled monitoring, run daily or weekly with the same store inputs and deduplicate product rows by `storeDomain` plus `productId` or `productUrl` in your downstream workflow. ### Reliability notes The Actor saves best-effort results. If optional sections such as `/products.json`, `/collections.json`, or sitemap page discovery fail for a store, the store summary is still saved with `failedSections` and `warnings` when the homepage was reachable. This keeps lead-enrichment and monitoring runs useful even when one Shopify endpoint is disabled, blocked, or temporarily unavailable. ### Why use this Actor? Most Shopify scrapers focus only on product exports. This Actor is designed for ecommerce intelligence: it combines catalog extraction with lead enrichment, contact discovery, social links, theme/app hints, and store-level metrics. It helps answer questions like: - What products does a Shopify store sell? - Which variants are available and which products are on sale? - What apps and marketing tools does a store appear to use? - Which social profiles, emails, and useful pages are publicly linked? - How large is the catalog, and what is the pricing range? - Which Shopify stores are good leads for ecommerce SaaS, agencies, affiliates, or suppliers? Because it runs on Apify, you also get scheduling, API access, datasets, webhooks, proxy rotation, and integrations without maintaining your own server. # Actor input Schema ## `storeUrls` (type: `array`): Shopify store homepages or domains to scrape. ## `urls` (type: `array`): Alias for Shopify Store URLs. Useful when triggering the Actor from API clients that use a generic urls field. ## `includeStoreSummary` (type: `boolean`): Save one store-level row with metadata, detected apps, contact details, social links, and catalog statistics. ## `includeProducts` (type: `boolean`): Save product rows from Shopify's public products.json endpoint. ## `includeCollections` (type: `boolean`): Save collection rows from Shopify's public collections.json endpoint. ## `includePages` (type: `boolean`): Discover useful pages from the Shopify sitemap and extract page text, emails, and phone numbers. ## `includeVariants` (type: `boolean`): Include full variant objects on product rows. Disable for smaller output files. ## `includeHtmlDescription` (type: `boolean`): Include raw product description HTML. Clean description text is always included. ## `maxProductsPerStore` (type: `integer`): Maximum product rows to save per Shopify store. ## `maxCollectionsPerStore` (type: `integer`): Maximum collection rows to save per Shopify store. ## `maxPagesPerStore` (type: `integer`): Maximum discovered contact, about, wholesale, privacy, and policy pages to save per store. ## `maxConcurrency` (type: `integer`): Number of stores or pages to process in parallel. ## `requestDelayMs` (type: `integer`): Delay between retry attempts in milliseconds. Higher values can help on larger multi-store runs. ## `maxRetries` (type: `integer`): Retry count for transient HTTP failures, proxy failures, and target-side 429/5xx responses. ## Actor input object example ```json { "storeUrls": [ "https://www.allbirds.com" ], "includeStoreSummary": true, "includeProducts": true, "includeCollections": true, "includePages": true, "includeVariants": true, "includeHtmlDescription": false, "maxProductsPerStore": 250, "maxCollectionsPerStore": 250, "maxPagesPerStore": 10, "maxConcurrency": 5, "requestDelayMs": 250, "maxRetries": 3 } ``` # Actor output Schema ## `results` (type: `string`): Store summaries, products, collections, pages, contacts, social links, prices, variants, and theme/app signals stored in the default dataset. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "storeUrls": [ "https://www.allbirds.com" ], "includeStoreSummary": true, "includeProducts": true, "includeCollections": true, "includePages": true, "includeVariants": true, "includeHtmlDescription": false, "maxProductsPerStore": 250, "maxCollectionsPerStore": 250, "maxPagesPerStore": 10, "maxConcurrency": 5, "requestDelayMs": 250, "maxRetries": 3 }; // Run the Actor and wait for it to finish const run = await client.actor("elliotpadfield/shopify-store-intelligence-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "storeUrls": ["https://www.allbirds.com"], "includeStoreSummary": True, "includeProducts": True, "includeCollections": True, "includePages": True, "includeVariants": True, "includeHtmlDescription": False, "maxProductsPerStore": 250, "maxCollectionsPerStore": 250, "maxPagesPerStore": 10, "maxConcurrency": 5, "requestDelayMs": 250, "maxRetries": 3, } # Run the Actor and wait for it to finish run = client.actor("elliotpadfield/shopify-store-intelligence-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "storeUrls": [ "https://www.allbirds.com" ], "includeStoreSummary": true, "includeProducts": true, "includeCollections": true, "includePages": true, "includeVariants": true, "includeHtmlDescription": false, "maxProductsPerStore": 250, "maxCollectionsPerStore": 250, "maxPagesPerStore": 10, "maxConcurrency": 5, "requestDelayMs": 250, "maxRetries": 3 }' | apify call elliotpadfield/shopify-store-intelligence-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=elliotpadfield/shopify-store-intelligence-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Shopify Store Intelligence Scraper - Products, Leads & Apps", "description": "Scrape Shopify stores for product catalogs, prices, variants, collections, contact details, social links, theme/app signals, pages, and store-level ecommerce intelligence.", "version": "1.1", "x-build-id": "an60NHYtDSiztcShG" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/elliotpadfield~shopify-store-intelligence-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-elliotpadfield-shopify-store-intelligence-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/elliotpadfield~shopify-store-intelligence-scraper/runs": { "post": { "operationId": "runs-sync-elliotpadfield-shopify-store-intelligence-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/elliotpadfield~shopify-store-intelligence-scraper/run-sync": { "post": { "operationId": "run-sync-elliotpadfield-shopify-store-intelligence-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "storeUrls": { "title": "Shopify Store URLs", "uniqueItems": true, "type": "array", "description": "Shopify store homepages or domains to scrape.", "items": { "type": "string" }, "default": [ "https://www.allbirds.com" ] }, "urls": { "title": "URLs", "uniqueItems": true, "type": "array", "description": "Alias for Shopify Store URLs. Useful when triggering the Actor from API clients that use a generic urls field.", "items": { "type": "string" } }, "includeStoreSummary": { "title": "Include Store Summary", "type": "boolean", "description": "Save one store-level row with metadata, detected apps, contact details, social links, and catalog statistics.", "default": true }, "includeProducts": { "title": "Include Products", "type": "boolean", "description": "Save product rows from Shopify's public products.json endpoint.", "default": true }, "includeCollections": { "title": "Include Collections", "type": "boolean", "description": "Save collection rows from Shopify's public collections.json endpoint.", "default": true }, "includePages": { "title": "Include Contact and Policy Pages", "type": "boolean", "description": "Discover useful pages from the Shopify sitemap and extract page text, emails, and phone numbers.", "default": true }, "includeVariants": { "title": "Include Variants", "type": "boolean", "description": "Include full variant objects on product rows. Disable for smaller output files.", "default": true }, "includeHtmlDescription": { "title": "Include HTML Descriptions", "type": "boolean", "description": "Include raw product description HTML. Clean description text is always included.", "default": false }, "maxProductsPerStore": { "title": "Maximum Products per Store", "minimum": 0, "maximum": 25000, "type": "integer", "description": "Maximum product rows to save per Shopify store.", "default": 250 }, "maxCollectionsPerStore": { "title": "Maximum Collections per Store", "minimum": 0, "maximum": 5000, "type": "integer", "description": "Maximum collection rows to save per Shopify store.", "default": 250 }, "maxPagesPerStore": { "title": "Maximum Pages per Store", "minimum": 0, "maximum": 100, "type": "integer", "description": "Maximum discovered contact, about, wholesale, privacy, and policy pages to save per store.", "default": 10 }, "maxConcurrency": { "title": "Maximum Concurrency", "minimum": 1, "maximum": 20, "type": "integer", "description": "Number of stores or pages to process in parallel.", "default": 5 }, "requestDelayMs": { "title": "Request Delay", "minimum": 0, "maximum": 10000, "type": "integer", "description": "Delay between retry attempts in milliseconds. Higher values can help on larger multi-store runs.", "default": 250 }, "maxRetries": { "title": "Maximum Retries", "minimum": 0, "maximum": 8, "type": "integer", "description": "Retry count for transient HTTP failures, proxy failures, and target-side 429/5xx responses.", "default": 3 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```