WooCommerce Product Scraper — Prices, Stock & Change Monitor

Turn a WooCommerce store URL into a clean product catalog—or a watchlist that tells you what changed.

Export product names, prices, sale prices, currency, SKUs, availability, images and categories. Schedule repeat runs to identify price drops, restocks and new listings without comparing spreadsheets by hand.

No store login, data-provider subscription or API key is required for supported public stores. No browser, proxy rotation or CAPTCHA solver is used.

Works only where the store exposes an accessible public WooCommerce Store API and permits this access. It does not work on every shop. Blocked, redirected, robots-disallowed and unsupported stores are reported individually.

🔎 Two useful modes

Mode	What you receive
Export product catalog	Current product records, suitable for CSV, spreadsheets and product research.
Monitor price and stock changes	First complete baseline, then only changed products on subsequent runs.

The input is a store homepage, not a marketplace search URL. For example, https://store.com, or https://store.com/wordpress if that is the WordPress installation. A full /wp-json/wc/store/v1/products collection URL is accepted without query parameters. Product, /shop/ and category URLs are rejected instead of silently expanding them into an entire-store scrape.

The prefilled example is WooCommerce.com, which sells extensions and themes. It returns real products, not mock rows. Replace it with your desired store.

🚀 Quick start

1. Paste one or more store homepage URLs.
2. Choose Export product catalog or Monitor price and stock changes.
3. Optionally enter words in Search for products to search inside each store.
4. Set the maximum products/pages per store and start the run.
5. Open Dataset for results and Storage → Key-value store for the CSV and RUN_SUMMARY.json.

Export ten products

{
  "storeUrls": ["https://woocommerce.com"],
  "mode": "catalog",
  "maxProductsPerStore": 10,
  "saveCsvFile": true
}

Monitor a filtered catalog

{
  "storeUrls": ["https://woocommerce.com"],
  "mode": "monitor",
  "searchQuery": "Per Product Shipping",
  "maxProductsPerStore": 100,
  "monitorKey": "demo-extensions",
  "outputInitialSnapshot": true,
  "saveCsvFile": true
}

The first complete run saves the baseline. Later runs using the same monitor name, store and filters compare against it. Changing a filter starts a different baseline, avoiding misleading missing-product alerts across different searches. Different initiating Apify users and Actor deployments have separate named state stores.

Increase the limits for large catalogs. If the store reports 1,759 matching products but your limit is 500, catalog mode can export the first 500; monitor mode will preserve the prior baseline rather than comparing a truncated list.

What changes are reported?

• price_drop / price_rise, with old price, new price and percentage change.
• back_in_stock / out_of_stock when known boolean availability changes.
• new_product when an ID first appears in the same result scope.
• missing_from_results when a previous ID is absent from a complete scan.
• Currency, sale status, price-range or product-detail changes when observed.

Multiple changes to one product share one output record. A new baseline is labelled record_type: baseline, not presented as hundreds of new-product alerts. Set outputInitialSnapshot: false to save the first baseline silently.

An unchanged monitor produces zero dataset rows and a header-only CSV. This is a successful check, not an error. Check the run summary for the store status.

No false disappearance alerts from failed scans

The Actor checks pagination totals, page sizes, duplicate IDs and changing totals. A blocked, failed, capped or inconsistent scan never advances the monitoring baseline. If output is truncated by the result spending limit, the baseline is also preserved so unreported changes are not silently consumed.

missing_from_results is not proof that a product was deleted, discontinued or sold. Store visibility, filtering and catalog changes can affect exposure. Pagination is not a transactional database snapshot; changes during scanning cannot be ruled out completely.

📁 Output and downloads

Field	Meaning
record_type	product, baseline or change
store_url, product_id, product_url	Store-scoped product identity and source link
name, sku, product_type	Published product fields when available
price, regular_price, sale_price	Decimal strings converted using the API's currency precision
min_price, max_price	Public price range when supplied
currency, in_stock, on_sale	Observed currency and availability/sale flags
old_price, old_in_stock, change_types	Previous values and detected changes
price_change_percent	Percentage change when prices are comparable and old price is nonzero
image_urls, categories	Up to ten image URLs and twenty categories
scan_complete, observed_at	Completeness and collection time

Files in the default key-value store:

• woocommerce_results.csv — dataset output; configurable filename, enabled by default. CSV cells are escaped to reduce spreadsheet formula injection.
• RUN_SUMMARY.json — every store's status, limits, counts and errors.
• PARTIAL_PRODUCTS_01.json, etc. — per-store page checkpoints, including incomplete scans. Recovery data, not automatic resume or a monitor baseline.

Monitoring baselines are compact records in a separate persistent named key-value store. Do not delete that store if you want the watchlist to continue. Run schedules with the same monitor name/store serially. Conflict detection is optimistic, not a distributed lock, so overlapping runs are unsupported.

Who is this for?

• E-commerce sellers watching competitor pricing and promotions.
• Sourcing teams comparing retail price points with supplier quotations.
• Agencies exporting client-owned catalogs or maintaining product watchlists.
• Researchers collecting public catalog snapshots with clear source evidence.

It does not automatically match products to Alibaba, calculate profit margins, change store prices, publish listings or make purchases.

Important limits

• Reads only the documented public Store API. No Admin API, customer/order records, cookies, credentials, private data or paid provider are accessed.
• Public stock status is not exact inventory, sales or demand. Individual variation prices/SKUs and reviews are not separately crawled in this version.
• Names are limited to 1,000 characters and SKUs to 255. Descriptions are omitted.
• Prices reflect the store's response context. Currency changes are not treated as price drops; no exchange-rate conversion or tax/shipping calculation occurs.
• HTTPS only. Internal addresses and IP-literal URLs are rejected. DNS is checked and connections pinned to a public address with certificate verification.
• Respects robots rules, uses one paced read-only request at a time, and stops on access denial/rate limiting. Redirects are not followed; provide the final canonical installation URL. No workaround is attempted for protected stores.
• Default limits: 500 products/store, 100 pages/store, up to 20 stores. Hard maximum: 10,000 products/store. These are limits, not completeness guarantees.
• A hard Apify timeout can interrupt final CSV/state writes. Earlier dataset writes and per-page checkpoints can still be available.

API, MCP and scheduling

Deploy to your own Apify account first. Use the actual deployed Actor ID with the Apify API, or load that Actor through the Apify MCP server. The JSON inputs are the same as the form; no deployed Actor ID is invented here.

For monitoring, schedule the same input and monitor name in Apify. Connect the result dataset to your existing Sheets, Make, n8n or webhook workflow. The Actor does not independently send emails or messages.

If you enable per-result monetization, one dataset row is one product/baseline/ changed product. CSVs and status reports add no dataset rows. No subscription, paid data source, monetization setting or public publication has been activated by this repository. See DEPLOYMENT.md.

Validation and development

See VALIDATION.md for observed live results and remaining limits. Do not equate tests against one official store with universal store coverage.

python -m pip install -r requirements-dev.txt
python -m src.main --check
python -m unittest discover -s tests -v

Tests use synthetic public API responses and mocks; ordinary CI never contacts stores. python -m src.live_check <public-demo-url> is a separate, explicit, small live check that respects the same access safeguards.

Built against the official Store API and Products API. Independent project, not affiliated with or endorsed by WooCommerce/Automattic. Use only data you are authorized to access and respect applicable store terms.