# WooCommerce Products Scraper — Catalog and Price Evidence (`peerless_columbine/independent-woocommerce-catalog`) Actor

Scrape public WooCommerce product catalogs with exact minor-unit prices, coverage evidence, additional existing Dataset append and sequential multi-store collection. Use a finite run charge limit; support for uncapped runs is being repaired.

- **URL**: https://apify.com/peerless_columbine/independent-woocommerce-catalog.md
- **Developed by:** [tingyou333 zhuang](https://apify.com/peerless_columbine) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 delivered products

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## WooCommerce Products Scraper — Catalog and Price Evidence

Export products from public WooCommerce Store API homepages into a Dataset. Use it for catalog exports, supplier audits and repeated price/inventory analysis. It is not a WooCommerce integration requiring an admin key. It does not modify a shop, create carts or visit account endpoints.

### Input

| Field | Runtime behavior |
| --- | --- |
| `domains` | Supply a nonempty string array of store root domains/homepages. Bare domains, `http(s)`, `www`, IDN and trailing slash are accepted. Product/category paths, credentials, unsafe IPs and nonstandard target ports are rejected. Invalid individual stores are reported and skipped. There are no implicit sample stores. |
| `maxProducts` | Integer ≥0, **default 0**, separately for each store. Zero adds no product/page count limit. Positive limits count confirmed unique products, with partial coverage reported. |
| `proxyConfiguration` | Default `{"useApifyProxy":false}`. Explicit Apify pools or HTTP(S) custom proxies are supported. They do not fall back to direct. Apify Proxy needs hosted injected credentials; custom gateways must resolve publicly. Proxy usage/transfer can cost extra. |
| `datasetId` | Optional existing caller-owned Apify Dataset ID. Products also stay in the default Dataset. No storage creation, permission changes or foreign writes. Cloud runtime only; ID must be an actual 17-character Apify ID. |
| `runId` | Optional previous 17-character Apify run ID. Recorded as `associatedRunId` in products and reports. It is a supplied provenance link, not independently verified ancestry. No input or Dataset is inherited or fetched. |

Supply domains explicitly: missing or empty domains fail before source requests. `runId` is a provenance link only; no previous input or Dataset is inherited.

Extra controls: `pageSize` 1–100 (default **100**), `search` (default empty/unfiltered), `requestDelayMillis` (default500), `requestTimeoutSecs` (default30), `runTimeoutSecs` (default900), `maxRequests` (default0/no count cap), `maxResponseBytes` (default8 MiB, up to50 MiB), `maxRecordBytes` (default8 MiB, range1 KiB–8 MiB). Source requests are sequential, paced per origin, increased for robots Crawl-delay, and never automatically retried. Robots, redirects, diagnostics and fake-IP-only DoH count toward the transport limit. The example/prefill explicitly limits to five filtered products; it is not a default of five or proof of a whole-catalog run.

```json
{"domains":["mercantile.wordpress.org"],"maxProducts":5,"proxyConfiguration":{"useApifyProxy":false},"pageSize":5,"search":"WordPress","maxRequests":12,"runTimeoutSecs":180,"maxResponseBytes":2097152}
```

For an unfiltered catalog, supply domains and leave `maxProducts:0`, `pageSize:100`, `search:""`. Source deadlines, byte bounds, platform timeout, storage acknowledgement and available fees still apply. Zero does not promise an infinite runtime or a transactionally stable catalog.

### Product output

Each product includes these 23 catalog fields: `featuredImage`, `name`, `url`, `sku`, `type`, `currency`, `shortDescription`, `description`, `store`, `imageCount`, `reviewCount`, `variationCount`, `price`, `regularPrice`, `salePrice`, `lowStockRemaining`, `averageRating`, `onSale`, `available`, `isOnBackorder`, `categories`, `tags`, `attributes`.

Descriptions are HTML-stripped text. Categories, tags and attributes are arrays of names. `available` requires both native in-stock and purchasable flags. `salePrice` is null unless a sale is active; `averageRating` is null with zero reviews. `lowStockRemaining` is the native low-stock value or null; it does not infer total stock. Variable products retain the native parent/current price and variation-list count; individual authenticated variation prices are not fetched. Missing optional native fields have conservative defaults and appear in `mappingWarnings`; original structures remain in `raw`.

Additional fields include `productId`, `rawPrices`, `currencyMinorUnit`, exact decimal strings `priceExact`/`regularPriceExact`/`salePriceExact`, the original `raw`, hashes, page URL and `sourceEvidenceKey`. Numeric prices are JSON numbers; exact strings preserve minor-unit arithmetic without binary float loss. Currency/geolocation can change a source price; compare the accompanying currency and raw price evidence.

`SOURCE-00000001` and following KVS records bind each page's body hash, URL, response status, received counts and source-declared totals. `STORE-00000001` and following records retain each store's result, skipped/duplicate/invalid counts and coverage reason. `RUN-REPORT` links those numbered records and summarizes scope and SDK observations. Use this audit trail to check which source pages and product IDs were read. API headers and changing live catalogs do not establish a fixed snapshot.

### Failures and partial results

The crawler distinguishes unreachable transport, 401/403 access denial, confirmed missing WordPress, WordPress without WooCommerce, and a Woo-enabled site whose products endpoint is404. Ambiguous HTML is reported as unconfirmed rather than proof of a missing platform. Robots rejection/unavailability,429, invalid JSON, oversized responses, pagination inconsistencies, repeated pages and exhausted resources stop the affected store with fixed diagnostics. No proxy or route is switched to evade a rejection. Good stores continue; if every store fails the Actor fails. A genuinely valid empty catalog may succeed with zero products.

Deduplication uses the canonical final store origin plus native product ID on run-local disk. Repeated pages or no new valid products stop with incomplete coverage; overlapping pages retain new IDs. Explicit product caps, spend/request/time limits and malformed products leave an honest partial prefix. These limits do not silently redefine “unlimited” into a fixed count.

A product and each evidence/report record must fit `maxRecordBytes` of plain UTF-8 JSON (default8 MiB). The limit is checked before storage writes or a product fee. Oversized evidence/report records fail the run. Oversize products are skipped with invalid/partial counts and no custom fee; no content is silently truncated. Page evidence and store reports are separate records, avoiding repeated full-catalog report uploads. Large unlimited catalogs still require enough time, memory, disk, storage and budget.

### Delivery, extra Dataset and fees

See the **Pricing** tab on this Actor’s Store page for current prices. In pay-per-event runs, one `product` event applies per confirmed unique default product, plus platform `apify-actor-start` (startup units depend on memory, at least one). The Dataset synthetic event has no additional price. Startup can apply to empty or failed runs. Failed products, duplicates and source errors incur no custom product event; compute, network, storage and proxy costs may still apply.

The adapter admits affordable products in bounded batches, saves the default rows, checks the SDK acknowledgement, then charges the matching number of product events and verifies the actual charged count and amount delta. It never retries a Dataset, KVS or charge mutation. After default rows and their fees are confirmed, an optional extra copy is preceded by a durable `DELIVERY-STATUS` append intent. An extra copy incurs no second custom product fee. The additional Dataset uses the same caller token through a direct zero-retry, no-redirect SDK client, after metadata verifies its existing ID and caller ownership. An extra ID equal to the default Dataset adds no duplicate copy.

If an extra append fails or has an unknown outcome, default rows and confirmed fees remain; no refund, deletion, replay or automatic restart occurs. The pending intent is retained and the Actor stops. A confirmed extra append is followed by a checkpoint; an unknown checkpoint also stops. Additional copies incur one Dataset item write per product. The adapter submits bounded batches and normally saves a pending and a confirmed `DELIVERY-STATUS` checkpoint per batch; byte limits may split a page into more batches. The existing-Dataset preflight adds one metadata read. Include storage, retention and read costs in the run budget. A product may be delivered when its fee outcome is unknown: reservations remain and no same-count SDK claim substitutes for a remote receipt.

Hard kills can prevent a final report or leave a pending default/fee/extra write. Run-local ledgers are not crash-resume authorization. Disable automatic restart when reconciling an interrupted paid run, and inspect saved rows and append intent before acting. The Actor does not automatically refund or replay an uncertain fee.

### Network boundaries

Targets are public HTTP(S), with public DNS checks on every hop, numeric TCP pinning, original Host/SNI and certificate verification, bounded raw/decoded gzip and no environment proxy. Only all-fake-IP system answers may use the fixed authenticated DoH resolver. The exact SDK-injected hosted proxy endpoint has a narrow internal-gateway capability; target IPs remain public. User-supplied custom proxies get no private-IP exception. Robots are respected,404/410 means unavailable rather than disallowed, and unresolved other robots responses stop that route. Operators must have permission for their chosen stores and uses; public access alone is not a reuse license.

### Scope limits

Only public root-homepage Store API catalogs are read. Category/product URL crawling, account/cart endpoints, authenticated individual variation pricing, built-in historical change detection and removal alerts are outside this Actor's scope. You can compare exported snapshots externally or pass an existing additional Dataset ID to combine runs. The source may change while pagination is in progress, and its reported totals do not provide an atomic catalog snapshot.

# Actor input Schema

## `domains` (type: `array`):

Public store root domains/homepages only, including bare domains, scheme, www and trailing slash. No product/category URLs. No runtime default: supply a nonempty list.

## `maxProducts` (type: `integer`):

Limit acknowledged unique products separately for each store. Zero sets no product/page count limit. Prefill 5 is a bounded sample, not the runtime default.

## `proxyConfiguration` (type: `object`):

Direct by default. Explicit Apify pool or HTTP(S) custom proxies are supported and never fall back to direct. Hosted Apify proxy needs injected credentials. Custom proxies and all target DNS stay strictly public.

## `datasetId` (type: `string`):

Optional existing caller-owned Apify Dataset ID. Default Dataset is retained; append a copy after the default row and its one custom fee are confirmed. No create, foreign write or permission change. Any unconfirmed append stops without replay.

## `runId` (type: `string`):

Optional previous run identifier recorded as associatedRunId in rows and reports for chaining. Does not fetch or inherit that run input or Dataset.

## `search` (type: `string`):

Extra control: pass an explicit search term to the public Store API. Empty means no search filter. The bounded example uses WordPress.

## `pageSize` (type: `integer`):

Extra control; primary mode requests 100 products/page. The saved real replay/example uses 5.

## `requestDelayMillis` (type: `integer`):

Minimum interval between source attempts on an origin, increased by robots Crawl-delay. No source retries.

## `requestTimeoutSecs` (type: `integer`):

Includes robots, DNS and redirects for a single fetch. Storage/fee operations have separate 15-second acknowledgement deadlines.

## `runTimeoutSecs` (type: `integer`):

Source work time limit; exhaustion reports partial and retains confirmed products. Platform hard kill can interrupt final reports or leave unknown writes.

## `maxRequests` (type: `integer`):

Includes robots, redirects, diagnostics and fake-IP-only DoH. Count zero imposes no transport count cap.

## `maxResponseBytes` (type: `integer`):

Streaming bound on both compressed and decoded response. Oversize stops the affected store without truncation-success. Use maxRecordBytes to separately bound product/report plain JSON delivery bytes.

## `maxRecordBytes` (type: `integer`):

Optional storage control, measured on plain UTF-8 JSON before a write or product fee. Default 8 MiB preserves the original limit. Oversize products are skipped and coverage is partial; no field is truncated. Oversize evidence or reports fail the run.

## Actor input object example

```json
{
  "domains": [
    "mercantile.wordpress.org"
  ],
  "maxProducts": 5,
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "search": "WordPress",
  "pageSize": 5,
  "requestDelayMillis": 500,
  "requestTimeoutSecs": 30,
  "runTimeoutSecs": 180,
  "maxRequests": 12,
  "maxResponseBytes": 2097152,
  "maxRecordBytes": 8388608
}
```

# Actor output Schema

## `products` (type: `string`):

No description

## `report` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "mercantile.wordpress.org"
    ],
    "maxProducts": 5,
    "search": "WordPress",
    "pageSize": 5,
    "runTimeoutSecs": 180,
    "maxRequests": 12,
    "maxResponseBytes": 2097152
};

// Run the Actor and wait for it to finish
const run = await client.actor("peerless_columbine/independent-woocommerce-catalog").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": ["mercantile.wordpress.org"],
    "maxProducts": 5,
    "search": "WordPress",
    "pageSize": 5,
    "runTimeoutSecs": 180,
    "maxRequests": 12,
    "maxResponseBytes": 2097152,
}

# Run the Actor and wait for it to finish
run = client.actor("peerless_columbine/independent-woocommerce-catalog").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "mercantile.wordpress.org"
  ],
  "maxProducts": 5,
  "search": "WordPress",
  "pageSize": 5,
  "runTimeoutSecs": 180,
  "maxRequests": 12,
  "maxResponseBytes": 2097152
}' |
apify call peerless_columbine/independent-woocommerce-catalog --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,peerless_columbine/independent-woocommerce-catalog"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/nFLfy1fNqGA79ZYXu/builds/3cNo0DjUL8kVY1OUM/openapi.json