# Product Catalog Extractor — Ecommerce Products (AI) (`noniner/product-catalog-extractor`) Actor

Extract a store's full product catalog into clean structured rows — title, price, variants, SKU, availability, images. Uses Shopify's public products.json where available, with an AI fallback for other stores. For price monitoring and competitive catalog intelligence.

- **URL**: https://apify.com/noniner/product-catalog-extractor.md
- **Developed by:** [Fredrik Hansson](https://apify.com/noniner) (community)
- **Categories:** E-commerce, Automation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $100.00 / 1,000 catalog extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Product Catalog Extractor — Ecommerce Products (AI)

**Turn any store URL into its full product catalog — as clean, structured rows.** Point this Actor at a store and it hands back **one row per product**: title, price range, variants, SKU, availability, images and product URL. For Shopify stores it reads the merchant's own **public `products.json`** — fast, complete, exact. For everything else it falls back to an **AI extractor** that reads the products visible on the page.

You bring the store. It hands back the catalog as a spreadsheet.

***

### The flow

```
1. Give a store URL              2. Detect the platform               3. Get structured product rows
   gymshark.com /          →        Shopify?  ── yes ──▶ products.json  →   title · price · variants ·
   a collection URL                           └─ no  ──▶ AI fallback         SKU · availability · image · URL
```

1. **Give a store domain** (e.g. `gymshark.com`) or a **storefront / collection URL**.
2. The Actor **detects Shopify** (via headers, markup and the public products endpoint). Shopify → it paginates `products.json`. Not Shopify → it fetches the page and uses OpenAI to read the listed products.
3. **Every product becomes one dataset row.** The dataset *is* the catalog — export it straight to CSV, Excel, JSON or feed it to your BI.

***

### What you get per product

For every product the Actor returns:

1. **`productTitle`** — the product name.
2. **`priceMin` / `priceMax`** — lowest and highest price across variants (numbers), plus **`currency`**.
3. **`variantsCount`** — how many variants (sizes / colours) the product has.
4. **`sku`** — the first variant's SKU.
5. **`available`** — whether any variant is in stock.
6. **`image`** — the primary product image URL.
7. **`url`** / **`handle`** — the canonical product URL and slug.
8. **`vendor`, `productType`, `tags`** — brand, category and keywords (Shopify).
9. **`extractionMethod`** — `shopify-json` (high fidelity) or `ai-fallback` (page-scoped).

***

### Three ways people use it

- **Competitive catalog monitoring** — pull a competitor's **entire Shopify catalog** on a schedule and diff it over time: new drops, discontinued SKUs, assortment size. **Extract Shopify products** from any store without an API key.
- **Price tracking & repricing** — grab `priceMin`/`priceMax` per product across a list of rival stores and feed a **price monitoring** or dynamic-pricing pipeline. The catalog refreshes as fast as you re-run it.
- **Product data feeds & ecommerce market research** — build a **product data feed** for affiliate sites, marketplaces or comparison engines, or do **ecommerce market research** on a whole niche (brands, price bands, in-stock rates) in one dataset.

***

### Input

| Field | Type | Notes |
|---|---|---|
| `urls` | array | **Required.** Store domains (`gymshark.com`) or storefront / collection URLs. One per line. |
| `maxProducts` | integer | Per-store cap on products extracted. Default `250`. |
| `maxResults` | integer | Max number of stores / URLs to process. Default `50`. One charge per store extracted. |
| `maxConcurrency` | integer | Stores processed in parallel. Default `5`. |
| `model` | string | OpenAI model for the AI fallback only. Default `gpt-4o-mini`. |

#### Minimal input example

```json
{
  "urls": ["gymshark.com", "allbirds.com"],
  "maxProducts": 250
}
```

***

### Output example

```json
[
  {
    "store": "gymshark.com",
    "productTitle": "Vital Seamless 2.0 Leggings",
    "handle": "vital-seamless-2-0-leggings",
    "productType": "Leggings",
    "vendor": "Gymshark",
    "tags": ["Leggings", "Seamless", "Womens"],
    "url": "https://gymshark.com/products/vital-seamless-2-0-leggings",
    "image": "https://cdn.shopify.com/s/files/.../vital.jpg",
    "variantsCount": 20,
    "priceMin": 40,
    "priceMax": 40,
    "currency": "USD",
    "available": true,
    "sku": "B1A1B-BBBB-XS",
    "extractionMethod": "shopify-json"
  },
  {
    "store": "allbirds.com",
    "productTitle": "Men's Wool Runners",
    "handle": "mens-wool-runners",
    "productType": "Shoes",
    "vendor": "Allbirds",
    "tags": ["Wool", "Runners"],
    "url": "https://allbirds.com/products/mens-wool-runners",
    "image": "https://cdn.shopify.com/s/files/.../wool-runner.jpg",
    "variantsCount": 14,
    "priceMin": 110,
    "priceMax": 110,
    "currency": "USD",
    "available": true,
    "sku": "A10001",
    "extractionMethod": "shopify-json"
  }
]
```

A non-Shopify store comes back with `extractionMethod: "ai-fallback"`, a populated `productTitle`, `priceText`/`priceMin` and (when present) `url` — but no variants / SKU, since those aren't exposed on a public page.

A per-store summary (product count, price range, method) is also written to the key-value store under **`SUMMARY`**.

***

### Pay-per-event pricing

This Actor uses **pay-per-event** billing with a single event:

| Event | Charged when | Price |
|---|---|---|
| `catalog-extracted` | Once per **store** whose catalog was extracted into rows — **any number of products**. | **$0.10** |

You are charged **per store, not per product**. A store yielding 5 products and a store yielding 5,000 both cost one `catalog-extracted`. Stores that return no products are not charged. The Actor logs its OpenAI token usage and estimated AI cost at the end of every run.

***

### Setup

- **Shopify stores need no setup** — `products.json` is public.
- The **non-Shopify AI fallback** calls OpenAI. Set the environment variable / secret **`OPENAI_API_KEY`** on the Actor. Without it, Shopify stores still work in full; non-Shopify stores are skipped and marked clearly.

***

### How extraction works

- **Shopify (primary):** the store's own public `products.json` endpoint is paginated (250 per page) until it's empty or `maxProducts` is reached. This is the merchant's real catalog data — titles, variants, prices, SKUs, availability and images — not scraped guesses.
- **AI fallback (other stores):** the given URL is fetched and cleaned, then OpenAI extracts the products **visible on that page**. This is **page-scoped** (one page, no pagination) and **lower-fidelity** than Shopify JSON — no variants, SKUs or stock status — so point it at a collection / category page for the best coverage.

***

### Use it from an AI agent (Apify MCP server)

This Actor is available through the [Apify MCP server](https://mcp.apify.com), so AI agents (Claude, ChatGPT, or anything that speaks MCP) can call it as a tool — hand it a list of store URLs and read back the full product catalog — with no custom integration code.

***

### Run it locally

```bash
npm install
## put your input in storage/key_value_stores/default/INPUT.json
export OPENAI_API_KEY=sk-...   # optional, enables the non-Shopify AI fallback
apify run -p
```

***

### Chain it

**This Actor is a building block in the No Niner ecommerce suite.** Wire it to the next step with no code in the **Integrations** tab (trigger: *Run succeeded* → start the next Actor), or run them in sequence.

- ⬆️ Upstream: [Shopify Store Analyzer](https://apify.com/noniner/shopify-store-analyzer) — find and qualify the stores first.
- ⬇️ Next: [DTC Brand Teardown](https://apify.com/noniner/dtc-brand-teardown) — fold the catalog into a brand teardown.
- 🔁 Pair with: [B2B Competitor Monitor](https://apify.com/noniner/b2b-competitor-monitor) — track catalog and price changes on a schedule.

***

### Related actors

- [Shopify Store Analyzer](https://apify.com/noniner/shopify-store-analyzer) — Analyze any Shopify store's catalog, pricing and assortment at a glance.
- [DTC Brand Teardown](https://apify.com/noniner/dtc-brand-teardown) — Teardown of a direct-to-consumer brand's site, products and positioning.
- [AI Price Intelligence](https://apify.com/noniner/ai-price-intelligence) — Track and compare prices across competing stores for repricing decisions.
- [Etsy Scraper](https://apify.com/noniner/etsy-scraper) — Scrape Etsy search results into structured product listings.

***

Built by the No Niner team.

# Actor input Schema

## `urls` (type: `array`):

Store domains (e.g. gymshark.com) or storefront / collection URLs. One per line. Shopify stores are read from their public products.json; other stores use an AI fallback on the given page. Optional if you supply a datasetId for chaining.

## `maxProducts` (type: `integer`):

Cap on how many products to extract per store.

## `maxResults` (type: `integer`):

Max number of stores / URLs to process. One pay-per-event charge per store whose catalog is extracted.

## `maxConcurrency` (type: `integer`):

How many stores to process in parallel.

## `model` (type: `string`):

Model used only for the non-Shopify AI fallback (default gpt-4o-mini).

## `datasetId` (type: `string`):

Optional. Read the URLs/domains from a previous Actor run's dataset — for no-code chaining via the Integrations tab.

## `datasetField` (type: `string`):

Which dataset field holds the URL/domain. Leave blank to auto-detect (url, website, domain, pageUrl, companyUrl, link).

## Actor input object example

```json
{
  "urls": [
    "gymshark.com",
    "allbirds.com"
  ],
  "maxProducts": 250,
  "maxResults": 50,
  "maxConcurrency": 5
}
```

# Actor output Schema

## `products` (type: `string`):

All extracted products in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "gymshark.com",
        "allbirds.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("noniner/product-catalog-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "gymshark.com",
        "allbirds.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("noniner/product-catalog-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "gymshark.com",
    "allbirds.com"
  ]
}' |
apify call noniner/product-catalog-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,noniner/product-catalog-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/0Tr1jmvvQ0jvG2fwd/builds/a0PJ6r5RLdWfiL238/openapi.json