Product Data Extractor

Turns product pages into one clean row each — parsed prices, normalised stock status, variants, absolute image URLs — from any storefront, not just Shopify.

Give it product URLs, or give it a store's homepage and let it find them.

{
  "startUrls": [{ "url": "https://www.gymshark.com" }],
  "discoverProducts": true,
  "maxProducts": 500
}

{
  "name": "Legacy Fitted T-Shirt",
  "brand": "Gymshark",
  "sku": "GSTS-4471",
  "price": 32,
  "priceCurrency": "USD",
  "availability": "InStock",
  "images": ["https://cdn.shopify.com/s/files/…/legacy-tee.jpg"],
  "variants": [
    { "name": "Legacy Tee — S", "sku": "GSTS-4471-S", "price": 32, "availability": "InStock" },
    { "name": "Legacy Tee — M", "sku": "GSTS-4471-M", "price": 32, "availability": "OutOfStock" }
  ],
  "source": "jsonld",
  "completeness": 1,
  "missingFields": [],
  "ok": true
}

What this does that a generic scraper does not

Prices come out as numbers, in the right magnitude. "1.299,90" and "1,299.90" are the same amount written by two halves of the world. Returning the string hands you a problem you discover in row 40,000; guessing wrong makes it a thousandfold error. Both are parsed, and anything genuinely unreadable — "call for pricing" — comes back absent rather than as 0.

Variants are not lost. A ProductGroup keeps name and brand on the parent and SKU, price and stock on each child. Read either node alone and you get half a product. Both are read, and every variant is returned with its own row of fields.

Non-English stores stay legible. Pages served as windows-1254 or ISO-8859-9 are decoded in the encoding the server declared, and HTML entities are resolved with case respected — Ç is Ç, ç is ç. No mojibake, no ü in your spreadsheet.

It works past the Shopify convention. Product URLs are found from the store's own sitemap filing rather than guessed from the path, so flat-URL stores work too — hisar.com.tr/hisar-16-parca-granit-ceyiz-seti has no /products/ segment anywhere and is found regardless.

Three sources, in the right order. JSON-LD first, then microdata, then OpenGraph — and a fallback only ever fills a gap, never overwrites. Each row tells you which sources it needed (source: "jsonld" | "mixed" | …).

Every row grades itself. completeness is the share of the seven fields that matter — name, brand, identifier, price, currency, availability, image — and missingFields names the ones that were absent. Set Minimum completeness to drop thin rows, and you are never charged for them.

A name that is really a SKU gets fixed. Some stores publish "name": "PCDP253G13IY". Where the page offers something better, you get the real title.

Input

Field	What it does
startUrls	Product URLs, or store homepages when discovery is on
discoverProducts	Find product pages from robots.txt and the sitemaps it declares
maxProducts	Hard cap for the run
minCompleteness	Drop rows below this percentage. Dropped rows are never charged
includeVariants	Keep the per-variant array, or flatten to one row per product
crawlerIdentity	Your own User-Agent. Whoever runs the scrape answers for the traffic

Output

Two kinds of row, so a run is always auditable:

• Products — ok: true, plus everything above.
• Unreachable pages — ok: false with url, httpStatus and error. Recorded so you can see them; never charged.

Politeness

Requests to one host are spaced out, bodies are capped, timeouts are hard, and a 429 is honoured with a single backed-off retry. These are not configurable. A scrape that gets your IP banned was not a bargain, and the stores being read did not ask to be.

Limits, stated plainly

• No JavaScript rendering. Products whose data is drawn client-side and published in no markup at all will come back thin. completeness will say so rather than pretending otherwise.
• Gzipped sitemaps (.xml.gz) are skipped, and reported in the log as skipped rather than silently counted as "no products".
• Discovery depends on the store publishing a sitemap. Without one, only the homepage's own product links are found — the run says so explicitly when that happens, because 40 rows against a catalogue of 4,000 is a number you need to know the provenance of.
• At most 15 sitemap documents are read per store, and no more than 25 are requested. A store whose index lists hundreds of dead children will not spend your run collecting 404s.
• A run that reaches your maximum total charge stops there and says so in the log and the run's status. It does not keep fetching pages it cannot deliver — raise the budget and run it again for the rest.

---

Built on the engine behind Scopras, an audit tool that grades a store's product markup against what AI shopping agents need. Same extraction rules: this actor gives you the data, Scopras tells you what is wrong with it.