# Shopify Hidden API Spy - Zero-DOM Product & Price Intel (`foxpink/shopify-hidden-api-spy`) Actor

Extract every product, variant, price, image, and tag from any Shopify store via hidden JSON API. Zero DOM, zero browser — never breaks. Includes competitive intelligence: launch velocity, price range, and tag cloud analysis.

- **URL**: https://apify.com/foxpink/shopify-hidden-api-spy.md
- **Developed by:** [Nguyễn Anh Duy](https://apify.com/foxpink) (community)
- **Categories:** AI, Developer tools
- **Stats:** 5 total users, 0 monthly users, 95.5% runs succeeded, 0 bookmarks
- **User rating**: 4.67 out of 5 stars

## Pricing

from $0.01 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## Shopify Hidden API Spy — Zero-DOM Product & Price Intelligence

[![Run on Apify](https://img.shields.io/badge/Run%20on%20Apify-FF7754?style=for-the-badge&logo=apify)](https://console.apify.com/actors/foxpink/shopify-hidden-api-spy)
[![Apify Marketplace](https://img.shields.io/badge/Marketplace-FF7754?style=for-the-badge&logo=apify)](https://apify.com/foxpink/shopify-hidden-api-spy)
[![GitHub Repo](https://img.shields.io/badge/Source%20Code-181717?style=for-the-badge&logo=github)](https://github.com/FoxPink/shopify-hidden-api-spy)
[![Version](https://img.shields.io/badge/v1.3-blue?style=for-the-badge)]()

> Extract every product, variant, price, image, and tag from any Shopify store via hidden JSON API. Zero DOM, zero browser — never breaks. Includes sale detection (compareAtPrice/hasSale/discountPercent), collection filter, password detection, launch velocity, price range, tag cloud, theme detection, SEO metadata, installed apps, and review analytics at **$0.01/1k results**.

---

### Why This Actor

Most Shopify scrapers use Puppeteer or Playwright to render pages, parse HTML classes, and extract data from DOM elements. The moment a store changes its theme or installs a layout app, those selectors break — costing you time and compute budget.

This Actor takes a different approach: it calls Shopify's **internal JSON API** (`products.json`) — the same endpoint Shopify itself uses to power every storefront on the internet. This API never changes, never breaks, and returns clean structured data regardless of theme, plugins, or design updates.

### Features

- **Zero DOM / Zero Browser** — pure `fetch` calls, no Puppeteer, no headless Chrome, no selectors to break
- **Hidden JSON API** — taps into `https://[store]/products.json?limit=250&page=N` for raw product data
- **Password Protection Detection** — checks `x-shopify-stage` header to detect password-protected stores
- **Bulk Processing** — accepts a single URL or an array of stores for batch intelligence runs
- **Launch Velocity** — counts products published in the last 30 days to measure store activity
- **Price Range Analysis** — calculates min, max, and average variant prices across the entire store
- **Tag Cloud** — aggregates and ranks the top 15 SEO tags used across all products
- **Product Type Breakdown** — identifies the top 5 product categories by volume
- **Variant SKU & Inventory** — per-variant SKU, price, inventory quantity, and availability status
- **Full Product Images** — array of all product images plus main image shortcut
- **Product Descriptions** — raw HTML body for AI pipelines and SEO analysis
- **Collection Mapping** — maps each product to its store collections
- **Store Name Extraction** — pulls store name from homepage `<title>` tag
- **Auto-Retry & Timeout** — built-in retry with exponential backoff for transient failures
- **Theme Detection** — identifies the Shopify theme (Dawn, Debut, etc.) via `Shopify.theme` HTML injection
- **SEO Metadata Extraction** — retrieves meta description, og:title, og:description, og:image from homepage
- **Review Analytics** — extracts aggregate rating and total review count from JSON-LD structured data
- **Installed Apps Detection** — scans homepage for 20+ known Shopify app scripts (Yotpo, Klaviyo, Gorgias, etc.)
- **Store Metadata** — fetches shop name, currency, country, locale via `/meta.json`
- **Tech Stack Detection** — identifies CDN provider, payment gateways, analytics tools, live chat, newsletter services, review platforms, and loyalty programs
- **Proxy Fallback** — transparently falls back to Apify residential proxy on 403/429

### Use Cases

| Who | Why |
|-----|-----|
| E-commerce Competitor Analysts | Monitor competitor pricing, new product launches, and catalog size |
| Dropshippers | Find winning products by analyzing top-selling Shopify stores |
| Brand Managers | Track brand presence, product mix, and SEO strategy of competitors |
| Market Researchers | Build product catalogs and price databases at scale |
| AI/ML Pipelines | Feed product data into recommendation engines or pricing models |

### Input

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `shopifyUrl` | string | — | Single target Shopify store URL (e.g. `gymshark.com`) |
| `shopifyUrls` | array | — | Bulk list of store URLs for batch processing |
| `maxProducts` | integer | `500` | Max products to extract per store (250 per API page) |

Either `shopifyUrl` or `shopifyUrls` must be provided.

#### Input Example

```json
{
  "shopifyUrl": "gymshark.com",
  "maxProducts": 200
}
````

Bulk mode:

```json
{
  "shopifyUrls": ["gymshark.com", "allbirds.com", "warbyparker.com"],
  "maxProducts": 500
}
```

### Output

Each product record:

| Field | Type | Example |
|-------|------|---------|
| `id` | integer | `6745123456789` |
| `title` | string | `"Glow Skin Serum V2"` |
| `handle` | string | `"glow-skin-serum-v2"` |
| `vendor` | string | `"Beauty Vendor"` |
| `productType` | string | `"Cosmetics"` |
| `bodyHtml` | string | `"<p>Product description...</p>"` |
| `createdAt` | string | `"2026-05-10T08:30:00Z"` |
| `collections` | array | `["Skincare", "Best Sellers"]` |
| `images` | array | `["https://cdn.shopify.com/img1.jpg", "..."]` |
| `mainImage` | string | `https://cdn.shopify.com/...` |
| `variants` | array | `[{title, sku, price, inventoryQuantity, available}]` |
| `totalVariants` | integer | `3` |
| `minPrice` | number | `29.99` |
| `maxPrice` | number | `39.99` |
| `tags` | array | `["serum", "skincare", "organic"]` |
| `isNewlyLaunched30Days` | boolean | `true` |

A `_summary` entry is appended per store with aggregate intelligence:

| Field | Description |
|-------|-------------|
| `storeName` | Store name extracted from homepage title |
| `totalProducts` | Number of products extracted |
| `totalVariants` | Total variant count across all products |
| `totalCollections` | Number of collections found |
| `globalMinPrice` | Cheapest variant across the store |
| `globalMaxPrice` | Most expensive variant |
| `avgVariantPrice` | Average price across all variants |
| `newlyLaunched30Days` | Products published in the last 30 days |
| `launchVelocity30Days` | Average new products per day (30-day window) |
| `themeName` | Shopify theme name (e.g. Dawn, Debut) |
| `themeRole` | Theme role (main, mobile, etc.) |
| `metaDescription` | Homepage meta description |
| `ogTitle` | Open Graph title tag |
| `ogDescription` | Open Graph description |
| `ogImage` | Open Graph image URL |
| `overallRating` | Aggregate rating from JSON-LD structured data |
| `totalReviews` | Total review count from JSON-LD |
| `appsInstalled` | Detected Shopify apps (Yotpo, Klaviyo, etc.) |
| `currency` | Store currency from `/meta.json` |
| `country` | Store country from `/meta.json` |
| `myshopifyDomain` | MyShopify domain from `/meta.json` |
| `topTags` | Top 15 SEO tags with frequency counts |
| `topProductTypes` | Top 5 product types by volume |
| `techStack` | Detected CDN, payment providers, analytics, newsletter, live chat, review, loyalty tools |
| `proxyUsed` | Whether Apify proxy was needed to bypass rate limiting |

### How It Works

```
Shopify Store → HTTPS test → Detect password protection → products.json pagination → Collection mapping → Product Mapping → Intelligence Summary
```

1. Normalizes the store domain
2. Sends a test request to detect password protection or non-Shopify stores
3. Paginates through `products.json` (250 products per page) with auto-retry until all products or `maxProducts` limit is reached
4. Maps each product to its store collections via `collections.json` and `smart_collections.json`
5. Enriches output with variant details (SKU, inventory, availability), full image array, and HTML body
6. Computes competitive intelligence: launch velocity, price range, tag cloud, and product type stats

### Pricing

**$0.01 per 1,000 results.** One result = one product record. Lightweight fetch-only architecture keeps compute costs near zero. A store with 500 products costs less than a cent to scrape.

### Quick Start

```bash
curl -X POST https://api.apify.com/v2/acts/foxpink~shopify-hidden-api-spy/runs \
  -H "Content-Type: application/json" \
  -d '{
    "shopifyUrl": "gymshark.com",
    "maxProducts": 200
  }' \
  "https://api.apify.com/v2/acts/foxpink~shopify-hidden-api-spy/runs?token=YOUR_API_TOKEN"
```

### Compatibility

- 100% Node.js (18+)
- No browser, no headless, no DOM
- No external scraping libraries — pure `fetch` API
- ESM (ECMAScript Modules)

# Actor input Schema

## `shopifyUrl` (type: `string`):

URL of the target Shopify store (e.g. gymshark.com or https://example.myshopify.com)

## `shopifyUrls` (type: `array`):

Optional: list of multiple store URLs for bulk processing

## `maxProducts` (type: `integer`):

Maximum number of products to extract per store (250 per API page)

## `collectionUrlFilter` (type: `string`):

Optional: filter products by collection handle (e.g. 'winter-sale' or '/collections/featured'). Only products in this collection will be returned.

## `sortBy` (type: `string`):

Sort order for products. 'best-selling' returns products ranked by sales volume (requires more API pages).

## Actor input object example

```json
{
  "shopifyUrl": "gymshark.com",
  "maxProducts": 50,
  "sortBy": "created-desc"
}
```

# Actor output Schema

## `productData` (type: `string`):

Primary output: JSON array of all extracted products. Each record includes id, title, handle, vendor, productType, bodyHtml, collections, images, variants (with SKU, inventory, compareAtPrice), price range, compareAtPrice, hasSale, discountPercent, tags, and launch velocity.

## `summaryByStore` (type: `string`):

Competitive intelligence summary per store: store name, total products, price range, launch velocity, collection count, top tags cloud, and top product types.

## `csvExport` (type: `string`):

Download all products as CSV for spreadsheet analysis.

## `jsonExport` (type: `string`):

Download all products as formatted JSON.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "shopifyUrl": "gymshark.com",
    "maxProducts": 50
};

// Run the Actor and wait for it to finish
const run = await client.actor("foxpink/shopify-hidden-api-spy").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "shopifyUrl": "gymshark.com",
    "maxProducts": 50,
}

# Run the Actor and wait for it to finish
run = client.actor("foxpink/shopify-hidden-api-spy").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "shopifyUrl": "gymshark.com",
  "maxProducts": 50
}' |
apify call foxpink/shopify-hidden-api-spy --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=foxpink/shopify-hidden-api-spy",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Shopify Hidden API Spy - Zero-DOM Product & Price Intel",
        "description": "Extract every product, variant, price, image, and tag from any Shopify store via hidden JSON API. Zero DOM, zero browser — never breaks. Includes competitive intelligence: launch velocity, price range, and tag cloud analysis.",
        "version": "1.3",
        "x-build-id": "DElHkbKsRuu770dIz"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/foxpink~shopify-hidden-api-spy/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-foxpink-shopify-hidden-api-spy",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/foxpink~shopify-hidden-api-spy/runs": {
            "post": {
                "operationId": "runs-sync-foxpink-shopify-hidden-api-spy",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/foxpink~shopify-hidden-api-spy/run-sync": {
            "post": {
                "operationId": "run-sync-foxpink-shopify-hidden-api-spy",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "properties": {
                    "shopifyUrl": {
                        "title": "Shopify Store URL",
                        "type": "string",
                        "description": "URL of the target Shopify store (e.g. gymshark.com or https://example.myshopify.com)",
                        "default": "gymshark.com"
                    },
                    "shopifyUrls": {
                        "title": "Bulk Store URLs",
                        "type": "array",
                        "description": "Optional: list of multiple store URLs for bulk processing",
                        "items": {
                            "type": "string"
                        }
                    },
                    "maxProducts": {
                        "title": "Max Products",
                        "minimum": 1,
                        "maximum": 10000,
                        "type": "integer",
                        "description": "Maximum number of products to extract per store (250 per API page)",
                        "default": 500
                    },
                    "collectionUrlFilter": {
                        "title": "Collection URL Filter",
                        "type": "string",
                        "description": "Optional: filter products by collection handle (e.g. 'winter-sale' or '/collections/featured'). Only products in this collection will be returned."
                    },
                    "sortBy": {
                        "title": "Sort Products By",
                        "enum": [
                            "created-desc",
                            "created-asc",
                            "best-selling",
                            "price-asc",
                            "price-desc",
                            "title-asc",
                            "title-desc"
                        ],
                        "type": "string",
                        "description": "Sort order for products. 'best-selling' returns products ranked by sales volume (requires more API pages).",
                        "default": "created-desc"
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```