# Product Data Extractor — Structured JSON from Any Store (`egeusta/product-data-extractor`) Actor

Extract structured product data from almost any online store. Get normalized prices, currency, availability, variants, images, SKU/GTIN, metadata, and a completeness score from JSON-LD, microdata, and Open Graph—ready for APIs, spreadsheets, and AI workflows.

- **URL**: https://apify.com/egeusta/product-data-extractor.md
- **Developed by:** [Ege](https://apify.com/egeusta) (community)
- **Categories:** E-commerce, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$5.00 / 1,000 product extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Product Data Extractor

Turns product pages into one clean row each — parsed prices, normalised stock
status, variants, absolute image URLs — from any storefront, not just Shopify.

Give it product URLs, or give it a store's homepage and let it find them.

```json
{
  "startUrls": [{ "url": "https://www.gymshark.com" }],
  "discoverProducts": true,
  "maxProducts": 500
}
```

```json
{
  "name": "Legacy Fitted T-Shirt",
  "brand": "Gymshark",
  "sku": "GSTS-4471",
  "price": 32,
  "priceCurrency": "USD",
  "availability": "InStock",
  "images": ["https://cdn.shopify.com/s/files/…/legacy-tee.jpg"],
  "variants": [
    { "name": "Legacy Tee — S", "sku": "GSTS-4471-S", "price": 32, "availability": "InStock" },
    { "name": "Legacy Tee — M", "sku": "GSTS-4471-M", "price": 32, "availability": "OutOfStock" }
  ],
  "source": "jsonld",
  "completeness": 1,
  "missingFields": [],
  "ok": true
}
```

### What this does that a generic scraper does not

**Prices come out as numbers, in the right magnitude.** `"1.299,90"` and
`"1,299.90"` are the same amount written by two halves of the world. Returning
the string hands you a problem you discover in row 40,000; guessing wrong makes
it a thousandfold error. Both are parsed, and anything genuinely unreadable —
"call for pricing" — comes back absent rather than as `0`.

**Variants are not lost.** A `ProductGroup` keeps name and brand on the parent
and SKU, price and stock on each child. Read either node alone and you get half
a product. Both are read, and every variant is returned with its own row of
fields.

**Non-English stores stay legible.** Pages served as windows-1254 or ISO-8859-9
are decoded in the encoding the server declared, and HTML entities are resolved
with case respected — `Ç` is Ç, `ç` is ç. No mojibake, no
`ü` in your spreadsheet.

**It works past the Shopify convention.** Product URLs are found from the
store's own sitemap filing rather than guessed from the path, so flat-URL
stores work too — `hisar.com.tr/hisar-16-parca-granit-ceyiz-seti` has no `/products/`
segment anywhere and is found regardless.

**Three sources, in the right order.** JSON-LD first, then microdata, then
OpenGraph — and a fallback only ever fills a gap, never overwrites. Each row
tells you which sources it needed (`source: "jsonld" | "mixed" | …`).

**Every row grades itself.** `completeness` is the share of the seven fields
that matter — name, brand, identifier, price, currency, availability, image —
and `missingFields` names the ones that were absent. Set
**Minimum completeness** to drop thin rows, and you are never charged for them.

**A name that is really a SKU gets fixed.** Some stores publish
`"name": "PCDP253G13IY"`. Where the page offers something better, you get the
real title.

### Input

| Field | What it does |
|---|---|
| `startUrls` | Product URLs, or store homepages when discovery is on |
| `discoverProducts` | Find product pages from robots.txt and the sitemaps it declares |
| `maxProducts` | Hard cap for the run |
| `minCompleteness` | Drop rows below this percentage. Dropped rows are never charged |
| `includeVariants` | Keep the per-variant array, or flatten to one row per product |
| `crawlerIdentity` | Your own User-Agent. Whoever runs the scrape answers for the traffic |

### Output

Two kinds of row, so a run is always auditable:

- **Products** — `ok: true`, plus everything above.
- **Unreachable pages** — `ok: false` with `url`, `httpStatus` and `error`.
  Recorded so you can see them; **never charged.**

### Politeness

Requests to one host are spaced out, bodies are capped, timeouts are hard, and
a `429` is honoured with a single backed-off retry. These are not configurable.
A scrape that gets your IP banned was not a bargain, and the stores being read
did not ask to be.

### Limits, stated plainly

- **No JavaScript rendering.** Products whose data is drawn client-side and
  published in no markup at all will come back thin. `completeness` will say so
  rather than pretending otherwise.
- **Gzipped sitemaps (`.xml.gz`) are skipped**, and reported in the log as
  skipped rather than silently counted as "no products".
- **Discovery depends on the store publishing a sitemap.** Without one, only
  the homepage's own product links are found — the run says so explicitly when
  that happens, because 40 rows against a catalogue of 4,000 is a number you
  need to know the provenance of.
- **At most 15 sitemap documents are read per store**, and no more than 25 are
  requested. A store whose index lists hundreds of dead children will not
  spend your run collecting 404s.
- **A run that reaches your maximum total charge stops there** and says so in
  the log and the run's status. It does not keep fetching pages it cannot
  deliver — raise the budget and run it again for the rest.

***

Built on the engine behind **Scopras**, an audit tool that grades a store's
product markup against what AI shopping agents need. Same extraction rules:
this actor gives you the data, Scopras tells you what is wrong with it.

# Actor input Schema

## `startUrls` (type: `array`):

Product page URLs to extract. To hand over whole stores instead, paste the store homepages and switch on 'Discover product URLs' below.

## `discoverProducts` (type: `boolean`):

Treat the URLs above as stores and find their product pages from robots.txt and the sitemaps it declares. Leave off when you are supplying product URLs directly.

## `maxProducts` (type: `integer`):

Hard cap on how many product pages are fetched in this run.

## `minCompleteness` (type: `integer`):

Drop rows that carry less than this share of the seven scored fields (name, brand, identifier, price, currency, availability, image). Rows dropped here are never charged. 0 keeps everything.

## `includeVariants` (type: `boolean`):

Keep the per-variant array (size and colour rows with their own SKU, price and stock). Turn off for a flat one-row-per-product table.

## `crawlerIdentity` (type: `string`):

How the crawler introduces itself to the stores you scrape. Leave blank for the default. Whoever runs the scrape answers for the traffic, so putting a real contact in here is the courteous choice.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.gymshark.com"
    }
  ],
  "discoverProducts": false,
  "maxProducts": 100,
  "minCompleteness": 0,
  "includeVariants": true
}
```

# Actor output Schema

## `products` (type: `string`):

Structured product records stored in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.gymshark.com"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("egeusta/product-data-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://www.gymshark.com" }] }

# Run the Actor and wait for it to finish
run = client.actor("egeusta/product-data-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.gymshark.com"
    }
  ]
}' |
apify call egeusta/product-data-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,egeusta/product-data-extractor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Sky0jZtaV0wEPFmZD/builds/F0WtedApTNj3gbz2t/openapi.json