WooCommerce Products Scraper — Catalog and Price Evidence

Export products from public WooCommerce Store API homepages into a Dataset. Use it for catalog exports, supplier audits and repeated price/inventory analysis. It is not a WooCommerce integration requiring an admin key. It does not modify a shop, create carts or visit account endpoints.

Input

Field	Runtime behavior
domains	Supply a nonempty string array of store root domains/homepages. Bare domains, http(s), www, IDN and trailing slash are accepted. Product/category paths, credentials, unsafe IPs and nonstandard target ports are rejected. Invalid individual stores are reported and skipped. There are no implicit sample stores.
maxProducts	Integer ≥0, default 0, separately for each store. Zero adds no product/page count limit. Positive limits count confirmed unique products, with partial coverage reported.
proxyConfiguration	Default {"useApifyProxy":false}. Explicit Apify pools or HTTP(S) custom proxies are supported. They do not fall back to direct. Apify Proxy needs hosted injected credentials; custom gateways must resolve publicly. Proxy usage/transfer can cost extra.
datasetId	Optional existing caller-owned Apify Dataset ID. Products also stay in the default Dataset. No storage creation, permission changes or foreign writes. Cloud runtime only; ID must be an actual 17-character Apify ID.
runId	Optional previous 17-character Apify run ID. Recorded as associatedRunId in products and reports. It is a supplied provenance link, not independently verified ancestry. No input or Dataset is inherited or fetched.

Supply domains explicitly: missing or empty domains fail before source requests. runId is a provenance link only; no previous input or Dataset is inherited.

Extra controls: pageSize 1–100 (default 100), search (default empty/unfiltered), requestDelayMillis (default500), requestTimeoutSecs (default30), runTimeoutSecs (default900), maxRequests (default0/no count cap), maxResponseBytes (default8 MiB, up to50 MiB), maxRecordBytes (default8 MiB, range1 KiB–8 MiB). Source requests are sequential, paced per origin, increased for robots Crawl-delay, and never automatically retried. Robots, redirects, diagnostics and fake-IP-only DoH count toward the transport limit. The example/prefill explicitly limits to five filtered products; it is not a default of five or proof of a whole-catalog run.

{"domains":["mercantile.wordpress.org"],"maxProducts":5,"proxyConfiguration":{"useApifyProxy":false},"pageSize":5,"search":"WordPress","maxRequests":12,"runTimeoutSecs":180,"maxResponseBytes":2097152}

For an unfiltered catalog, supply domains and leave maxProducts:0, pageSize:100, search:"". Source deadlines, byte bounds, platform timeout, storage acknowledgement and available fees still apply. Zero does not promise an infinite runtime or a transactionally stable catalog.

Product output

Each product includes these 23 catalog fields: featuredImage, name, url, sku, type, currency, shortDescription, description, store, imageCount, reviewCount, variationCount, price, regularPrice, salePrice, lowStockRemaining, averageRating, onSale, available, isOnBackorder, categories, tags, attributes.

Descriptions are HTML-stripped text. Categories, tags and attributes are arrays of names. available requires both native in-stock and purchasable flags. salePrice is null unless a sale is active; averageRating is null with zero reviews. lowStockRemaining is the native low-stock value or null; it does not infer total stock. Variable products retain the native parent/current price and variation-list count; individual authenticated variation prices are not fetched. Missing optional native fields have conservative defaults and appear in mappingWarnings; original structures remain in raw.

Additional fields include productId, rawPrices, currencyMinorUnit, exact decimal strings priceExact/regularPriceExact/salePriceExact, the original raw, hashes, page URL and sourceEvidenceKey. Numeric prices are JSON numbers; exact strings preserve minor-unit arithmetic without binary float loss. Currency/geolocation can change a source price; compare the accompanying currency and raw price evidence.

SOURCE-00000001 and following KVS records bind each page's body hash, URL, response status, received counts and source-declared totals. STORE-00000001 and following records retain each store's result, skipped/duplicate/invalid counts and coverage reason. RUN-REPORT links those numbered records and summarizes scope and SDK observations. Use this audit trail to check which source pages and product IDs were read. API headers and changing live catalogs do not establish a fixed snapshot.

Failures and partial results

The crawler distinguishes unreachable transport, 401/403 access denial, confirmed missing WordPress, WordPress without WooCommerce, and a Woo-enabled site whose products endpoint is404. Ambiguous HTML is reported as unconfirmed rather than proof of a missing platform. Robots rejection/unavailability,429, invalid JSON, oversized responses, pagination inconsistencies, repeated pages and exhausted resources stop the affected store with fixed diagnostics. No proxy or route is switched to evade a rejection. Good stores continue; if every store fails the Actor fails. A genuinely valid empty catalog may succeed with zero products.

Deduplication uses the canonical final store origin plus native product ID on run-local disk. Repeated pages or no new valid products stop with incomplete coverage; overlapping pages retain new IDs. Explicit product caps, spend/request/time limits and malformed products leave an honest partial prefix. These limits do not silently redefine “unlimited” into a fixed count.

A product and each evidence/report record must fit maxRecordBytes of plain UTF-8 JSON (default8 MiB). The limit is checked before storage writes or a product fee. Oversized evidence/report records fail the run. Oversize products are skipped with invalid/partial counts and no custom fee; no content is silently truncated. Page evidence and store reports are separate records, avoiding repeated full-catalog report uploads. Large unlimited catalogs still require enough time, memory, disk, storage and budget.

Delivery, extra Dataset and fees

See the Pricing tab on this Actor’s Store page for current prices. In pay-per-event runs, one product event applies per confirmed unique default product, plus platform apify-actor-start (startup units depend on memory, at least one). The Dataset synthetic event has no additional price. Startup can apply to empty or failed runs. Failed products, duplicates and source errors incur no custom product event; compute, network, storage and proxy costs may still apply.

The adapter admits affordable products in bounded batches, saves the default rows, checks the SDK acknowledgement, then charges the matching number of product events and verifies the actual charged count and amount delta. It never retries a Dataset, KVS or charge mutation. After default rows and their fees are confirmed, an optional extra copy is preceded by a durable DELIVERY-STATUS append intent. An extra copy incurs no second custom product fee. The additional Dataset uses the same caller token through a direct zero-retry, no-redirect SDK client, after metadata verifies its existing ID and caller ownership. An extra ID equal to the default Dataset adds no duplicate copy.

If an extra append fails or has an unknown outcome, default rows and confirmed fees remain; no refund, deletion, replay or automatic restart occurs. The pending intent is retained and the Actor stops. A confirmed extra append is followed by a checkpoint; an unknown checkpoint also stops. Additional copies incur one Dataset item write per product. The adapter submits bounded batches and normally saves a pending and a confirmed DELIVERY-STATUS checkpoint per batch; byte limits may split a page into more batches. The existing-Dataset preflight adds one metadata read. Include storage, retention and read costs in the run budget. A product may be delivered when its fee outcome is unknown: reservations remain and no same-count SDK claim substitutes for a remote receipt.

Hard kills can prevent a final report or leave a pending default/fee/extra write. Run-local ledgers are not crash-resume authorization. Disable automatic restart when reconciling an interrupted paid run, and inspect saved rows and append intent before acting. The Actor does not automatically refund or replay an uncertain fee.

Network boundaries

Targets are public HTTP(S), with public DNS checks on every hop, numeric TCP pinning, original Host/SNI and certificate verification, bounded raw/decoded gzip and no environment proxy. Only all-fake-IP system answers may use the fixed authenticated DoH resolver. The exact SDK-injected hosted proxy endpoint has a narrow internal-gateway capability; target IPs remain public. User-supplied custom proxies get no private-IP exception. Robots are respected,404/410 means unavailable rather than disallowed, and unresolved other robots responses stop that route. Operators must have permission for their chosen stores and uses; public access alone is not a reuse license.

Scope limits

Only public root-homepage Store API catalogs are read. Category/product URL crawling, account/cart endpoints, authenticated individual variation pricing, built-in historical change detection and removal alerts are outside this Actor's scope. You can compare exported snapshots externally or pass an existing additional Dataset ID to combine runs. The source may change while pagination is in progress, and its reported totals do not provide an atomic catalog snapshot.