Product Catalog Extractor — Ecommerce Products (AI)

Turn any store URL into its full product catalog — as clean, structured rows. Point this Actor at a store and it hands back one row per product: title, price range, variants, SKU, availability, images and product URL. For Shopify stores it reads the merchant's own public products.json — fast, complete, exact. For everything else it falls back to an AI extractor that reads the products visible on the page.

You bring the store. It hands back the catalog as a spreadsheet.

---

The flow

1. Give a store URL              2. Detect the platform               3. Get structured product rows
   gymshark.com /          →        Shopify?  ── yes ──▶ products.json  →   title · price · variants ·
   a collection URL                           └─ no  ──▶ AI fallback         SKU · availability · image · URL

1. Give a store domain (e.g. gymshark.com) or a storefront / collection URL.
2. The Actor detects Shopify (via headers, markup and the public products endpoint). Shopify → it paginates products.json. Not Shopify → it fetches the page and uses OpenAI to read the listed products.
3. Every product becomes one dataset row. The dataset is the catalog — export it straight to CSV, Excel, JSON or feed it to your BI.

---

What you get per product

For every product the Actor returns:

1. productTitle — the product name.
2. priceMin / priceMax — lowest and highest price across variants (numbers), plus currency.
3. variantsCount — how many variants (sizes / colours) the product has.
4. sku — the first variant's SKU.
5. available — whether any variant is in stock.
6. image — the primary product image URL.
7. url / handle — the canonical product URL and slug.
8. vendor, productType, tags — brand, category and keywords (Shopify).
9. extractionMethod — shopify-json (high fidelity) or ai-fallback (page-scoped).

---

Three ways people use it

• Competitive catalog monitoring — pull a competitor's entire Shopify catalog on a schedule and diff it over time: new drops, discontinued SKUs, assortment size. Extract Shopify products from any store without an API key.
• Price tracking & repricing — grab priceMin/priceMax per product across a list of rival stores and feed a price monitoring or dynamic-pricing pipeline. The catalog refreshes as fast as you re-run it.
• Product data feeds & ecommerce market research — build a product data feed for affiliate sites, marketplaces or comparison engines, or do ecommerce market research on a whole niche (brands, price bands, in-stock rates) in one dataset.

---

Input

Field	Type	Notes
urls	array	Required. Store domains (gymshark.com) or storefront / collection URLs. One per line.
maxProducts	integer	Per-store cap on products extracted. Default 250.
maxResults	integer	Max number of stores / URLs to process. Default 50. One charge per store extracted.
maxConcurrency	integer	Stores processed in parallel. Default 5.
model	string	OpenAI model for the AI fallback only. Default gpt-4o-mini.

Minimal input example

{
  "urls": ["gymshark.com", "allbirds.com"],
  "maxProducts": 250
}

---

Output example

[
  {
    "store": "gymshark.com",
    "productTitle": "Vital Seamless 2.0 Leggings",
    "handle": "vital-seamless-2-0-leggings",
    "productType": "Leggings",
    "vendor": "Gymshark",
    "tags": ["Leggings", "Seamless", "Womens"],
    "url": "https://gymshark.com/products/vital-seamless-2-0-leggings",
    "image": "https://cdn.shopify.com/s/files/.../vital.jpg",
    "variantsCount": 20,
    "priceMin": 40,
    "priceMax": 40,
    "currency": "USD",
    "available": true,
    "sku": "B1A1B-BBBB-XS",
    "extractionMethod": "shopify-json"
  },
  {
    "store": "allbirds.com",
    "productTitle": "Men's Wool Runners",
    "handle": "mens-wool-runners",
    "productType": "Shoes",
    "vendor": "Allbirds",
    "tags": ["Wool", "Runners"],
    "url": "https://allbirds.com/products/mens-wool-runners",
    "image": "https://cdn.shopify.com/s/files/.../wool-runner.jpg",
    "variantsCount": 14,
    "priceMin": 110,
    "priceMax": 110,
    "currency": "USD",
    "available": true,
    "sku": "A10001",
    "extractionMethod": "shopify-json"
  }
]

A non-Shopify store comes back with extractionMethod: "ai-fallback", a populated productTitle, priceText/priceMin and (when present) url — but no variants / SKU, since those aren't exposed on a public page.

A per-store summary (product count, price range, method) is also written to the key-value store under SUMMARY.

---

Pay-per-event pricing

This Actor uses pay-per-event billing with a single event:

Event	Charged when	Price
catalog-extracted	Once per store whose catalog was extracted into rows — any number of products.	$0.10

You are charged per store, not per product. A store yielding 5 products and a store yielding 5,000 both cost one catalog-extracted. Stores that return no products are not charged. The Actor logs its OpenAI token usage and estimated AI cost at the end of every run.

---

Setup

• Shopify stores need no setup — products.json is public.
• The non-Shopify AI fallback calls OpenAI. Set the environment variable / secret OPENAI_API_KEY on the Actor. Without it, Shopify stores still work in full; non-Shopify stores are skipped and marked clearly.

---

How extraction works

• Shopify (primary): the store's own public products.json endpoint is paginated (250 per page) until it's empty or maxProducts is reached. This is the merchant's real catalog data — titles, variants, prices, SKUs, availability and images — not scraped guesses.
• AI fallback (other stores): the given URL is fetched and cleaned, then OpenAI extracts the products visible on that page. This is page-scoped (one page, no pagination) and lower-fidelity than Shopify JSON — no variants, SKUs or stock status — so point it at a collection / category page for the best coverage.

---

Use it from an AI agent (Apify MCP server)

This Actor is available through the Apify MCP server, so AI agents (Claude, ChatGPT, or anything that speaks MCP) can call it as a tool — hand it a list of store URLs and read back the full product catalog — with no custom integration code.

---

Run it locally

npm install
# put your input in storage/key_value_stores/default/INPUT.json
export OPENAI_API_KEY=sk-...   # optional, enables the non-Shopify AI fallback
apify run -p

---

Chain it

This Actor is a building block in the No Niner ecommerce suite. Wire it to the next step with no code in the Integrations tab (trigger: Run succeeded → start the next Actor), or run them in sequence.

• ⬆️ Upstream: Shopify Store Analyzer — find and qualify the stores first.
• ⬇️ Next: DTC Brand Teardown — fold the catalog into a brand teardown.
• 🔁 Pair with: B2B Competitor Monitor — track catalog and price changes on a schedule.

---

Related actors

• Shopify Store Analyzer — Analyze any Shopify store's catalog, pricing and assortment at a glance.
• DTC Brand Teardown — Teardown of a direct-to-consumer brand's site, products and positioning.
• AI Price Intelligence — Track and compare prices across competing stores for repricing decisions.
• Etsy Scraper — Scrape Etsy search results into structured product listings.

---

Built by the No Niner team.