E-commerce Product Scraper - Any Store (Structured Data)

Get clean, consistent product data from online stores you choose: name, brand, SKU, GTIN/barcode, MPN, price, sale price, currency, availability, condition, variants, rating and review count, images, category and breadcrumbs, description and seller. Paste product pages, category pages, a store's home page or a sitemap URL. The scraper finds the products and returns one row per product in the same format for every store.

How it works

Most online stores publish machine-readable product data in their pages so search engines and shopping services can show prices and stock. This tool reads that standard data instead of guessing from page layouts. That makes it work across platforms without per-site setup:

• schema.org Product data in JSON-LD, microdata or RDFa, including offers, aggregate offers, product groups and variants
• Open Graph product tags (product:price:amount, product:availability and similar) as a fallback and to fill gaps
• Shopify fast path: the public products.json feed and product .js endpoints give every variant with its SKU, barcode, price, compare-at price and stock
• WooCommerce fast path: the public WooCommerce Store API gives prices, sale prices, stock, categories and ratings

The platform is detected automatically: Shopify, WooCommerce, BigCommerce, Magento (Adobe Commerce), Salesforce Commerce Cloud, PrestaShop, Shopware, Wix, Squarespace and others. Stores on custom platforms also work when they publish schema.org product data, and most large retailers do.

What you can paste

Input	What happens
Product page URL	One product from that page
Category / collection page URL	Product links on the page are found and each product is scraped. Next-page links are followed up to "Max pages per category"
Store home page	Shopify and WooCommerce stores: the product feed. Other stores: the product sitemap, then links on the home page
Sitemap URL (.xml, .xml.gz, sitemap index)	Product URLs from the sitemap; product sitemaps are preferred

Input

Field	Default	Description
Store URLs	2 sample product pages	Any mix of product, category, home page and sitemap URLs from any stores
Max products per category / store URL	20	Limit for each category, home page or sitemap URL
Max products in total	0 (no limit)	Limit for the whole run
Max pages per category	5	Pagination depth for category pages
Include variants	on	Variant list (options, SKU, GTIN, price, stock) inside each product row
Use Shopify / WooCommerce fast paths	on	Read the platforms' public product feeds
Browser rendering	fallback	fallback renders a page in a headless browser only when plain HTTP finds no product data or product links; always renders every page; never uses plain HTTP only. Browser pages have an extra charge (see Pricing). 2 GB memory or more is recommended for heavy browser use
Track price changes	off	Compare with the previous run and flag changes
Price history store name	product-price-history	Named key-value store that keeps the previous prices
Output only new or changed products	off	With tracking on, skip unchanged products (they are not charged)
Description length	2000	Characters kept from each description (0 = no limit)
Parallel requests	5	Overall concurrency (maximum 10)
Minimum delay per store	0.4 s	Gap between requests to the same store

Output

One row per product:

url, storeDomain, storeName, platform, name, brand, sku, gtin, mpn, productId, price (the current price you would pay), salePrice and regularPrice (filled when the product is discounted), onSale, discountPercent, currency, priceMin / priceMax (range across variants or offers), availability (schema.org value such as InStock, OutOfStock, PreOrder, BackOrder), inStock, condition, variantCount, variants (name, options, sku, gtin, price, regularPrice, availability, url, image), rating, reviewCount, imageUrl, images, category, breadcrumbs, description, seller, tags, dataSources (where the values came from), renderedWithBrowser, scrapedAt.

With price tracking on, each row also has isNew, previousPrice, priceChanged, priceChange, priceChangePercent, previousAvailability, availabilityChanged and previousScrapedAt.

Example (shortened):

{
  "url": "https://www.revzilla.com/motorcycle/arai-regent-x-helmet",
  "storeDomain": "revzilla.com",
  "platform": null,
  "name": "Arai Regent-X Helmet",
  "brand": "Arai Helmets",
  "price": 659.95,
  "currency": "USD",
  "availability": "InStock",
  "variantCount": 42,
  "rating": 4.6,
  "reviewCount": 232,
  "category": "Vehicles & Parts > Vehicle Parts & Accessories > ... > Motorcycle Helmets",
  "breadcrumbs": ["Home", "Helmets", "Full Face"],
  "dataSources": ["json-ld"]
}

Typical uses

• Competitor price monitoring: schedule a daily run on competitors' category pages with price tracking on and "Output only new or changed products", then send changes to Google Sheets, Slack or a webhook.
• Catalog and feed building: collect GTINs, SKUs, brands and images from supplier or brand sites to match products across retailers.
• MAP and reseller checks: brands can watch how their products are priced and stocked on retailer sites.
• Market research: compare assortments, price ranges and ratings across stores in a niche.

Pricing

You pay per product saved, with lower prices on higher Apify plans. Pages that have to be opened in a headless browser have a separate Browser-rendered page charge, added to the product price. A browser costs far more to run than a plain request. The browser charge applies only when:

• Browser rendering is fallback (the default) and plain HTTP found no product data or product links on a page, or
• Browser rendering is always (every page is rendered and charged).

Most stores publish their product data in the plain HTML, so with the default setting most runs have no browser charges. Set Browser rendering to never to rule them out completely. Pages that fail, are blocked or are skipped by robots.txt are not charged as products. A browser-rendered page is charged even when it turns out to have no product. The log lists how many pages were rendered in a browser.

Price tracking

When "Track price changes" is on, the last seen price and availability of every product are stored in a named key-value store in your Apify account. A later run with the same store name compares against it. Run the same input on a schedule to build a change feed. Use different store names for separate watch lists.

Notes and limits

• The scraper follows each store's robots.txt, including crawl delays, identifies itself with a bot user agent and paces requests per store. It does not use proxies to get around blocking, does not log in and does not solve CAPTCHAs. Stores that block automated requests (HTTP 403 or bot challenges) are skipped and listed in the log.
• Shopify limits storefront requests per IP. The scraper keeps a gentle shared pace for all Shopify stores, waits as instructed when rate-limited and falls back to product pages.
• Data quality depends on what each store publishes. Some fields (GTIN, MPN, ratings, condition) are only filled when the store includes them. Prices are what the page shows to a visitor without a login, in the store's default currency and region.
• Pages with too little product data (no price, SKU or availability) are skipped and not charged. A failing page never stops the run.
• Marketplaces whose terms forbid scraping (for example Amazon or eBay) are not a target for this tool. Use it on stores you are allowed to collect public data from.

Plot Point Technologies is not affiliated with Shopify, WooCommerce, BigCommerce, Adobe (Magento) or any store you scrape.