Ecommerce product scraper

Collect source-published product names, complete descriptions, images, SKUs, native prices, stock, physical attributes and tags from public catalog and product pages. The Actor reads supported Product structured data and product markup, including classic and block-based WooCommerce product cards, follows same-origin next-page links, and enriches each selected listing product from its public detail page before saving it.

The default ScrapeMe catalog is a publisher-authored demonstration catalog, not verified real merchandise, sale prices or retailer inventory. Its examples establish extraction of that published demonstration. Coverage of other stores depends on their layout and permitted public access; universal retailer coverage and commercial market completeness are not verified. Private, account, cart, checkout and action links are excluded.

Input

{"startUrl":"https://scrapeme.live/shop/","maxProducts":2}

The preserved defaults are startUrl https://scrapeme.live/shop/, maxProducts 100, maxPages 5 and timeoutMs 30000. maxProducts accepts 0–10000; 0 retains its safety limit of 1000 products. maxPages accepts 1–20. The product limit applies across all listing pages and does not skip detail enrichment for the final selected product. A transparent userAgent is optional.

Direct public requests are the supported route. Requested proxy use fails before any request because this transport does not support proxies. An explicitly direct configuration such as {"useApifyProxy":false} is accepted. Invalid types, private targets and unknown controls fail without fabricated output.

Example product

These native fields were verified against the published demonstration product and a saved Actor run. Runtime descriptions retain their complete published text; the sample shows that full description.

One complete native sentence is “By soaking up the sun’s rays, the seed grows progressively larger.” It appears within the full description below.

{
  "name": "Bulbasaur",
  "sku": "4391",
  "description": "Bulbasaur can be seen napping in bright sunlight. There is a seed on its back. By soaking up the sun's rays, the seed grows progressively larger.",
  "price": 63,
  "currency": "GBP",
  "priceValidThrough": "2027-12-31",
  "valueAddedTaxIncluded": false,
  "availability": "https://schema.org/InStock",
  "inStock": true,
  "stockText": "45 in stock",
  "stockQuantity": 45,
  "weight": "15.2 kg",
  "dimensions": "2 × 2 × 2 cm",
  "tags": ["bulbasaur", "Overgrow", "Seed"],
  "image": "https://scrapeme.live/wp-content/uploads/2018/08/001.png",
  "productUrl": "https://scrapeme.live/shop/Bulbasaur/"
}

Missing brand, rating, review count, currency or other source values are omitted. Missing reviews are not converted to zero. Currency is retained from native codes or explicit symbols; an ambiguous dollar symbol remains a symbol rather than being guessed as USD. Native offer qualifiers and price units are exposed when published without price conversion. Weight and dimensions retain their native units; stockQuantity is present only when visible stock wording states a number. images contains distinct product image assets, not thumbnail size variants. sourceUrl appears only when a separate listing/source page contributed to the product row. Run start and completion timestamps are stored in the run summary.

Results and access

Every dataset row is a genuine source product. OUTPUT and OUTPUT_SUMMARY record acknowledged saved counts, request counts and any failures. Useful records retained after a later failure produce SUCCEEDED with partial=true. Zero useful records produce FAILED and an empty dataset. A configured page or product cap limits coverage; it is not a claim that the whole catalog was collected.

Current publisher robots rules are checked before catalog, detail and redirect requests. Explicit denial or a challenge is not retried or bypassed. Only genuinely transient failures receive one bounded retry. Login, CAPTCHA solving, protection evasion, proxy rotation and alternative sources are not used. Each publisher's terms still govern use of its material. Raspberry Pi's dated terms expressly reserve against this collection; that specific origin is unavailable and does not create a blanket restriction on other publishers.