WooCommerce Scraper (Store Setup & Payment Check)

WooCommerce Scraper — Store Setup, Payment Gateways and Product Data

WooCommerce Scraper (Store Setup & Payment Check) qualifies a WooCommerce merchant before it touches the catalogue: it reads the store's own public wp-json, wc/store/v1 and cart endpoints for Store API reachability, true catalogue size, accepted payment gateways, served currency, default customer country and installed extensions — then pulls products, categories, tags, brands, attributes, reviews, pages, posts, comments or users from the same run. Every response is structured JSON, ready to pass directly to an LLM, index into a vector store, or feed a monitoring pipeline. No login, API key or cart write is needed — every endpoint it reads is the same public endpoint a shopper's own browser calls. Point it at a domain list once, and you know which stores are worth crawling before you spend a single catalogue request.

What is WooCommerce Scraper (Store Setup & Payment Check)?

It is an Apify Actor that takes a list of WooCommerce store URLs and, for each one, writes a store setup profile row — API reachability, catalogue size, payment gateways, currency, default customer country, installed extensions — then optionally pulls the store's catalogue in the same run. No competing WooCommerce scraper checked for this README reads payment gateways, currency, default customer country or installed extensions before scraping; the ones checked go straight to product extraction. That store-qualification step is what this Actor adds, and it's why the store profile row exists as its own record type instead of just extra product columns. No store account, login or API key is required — everything is read from endpoints the store already answers to any anonymous visitor.

Key capabilities, read from the input schema and source:

• 🧾 Store setup profile — one row per merchant with Store API/WordPress API reachability, catalogue size and size tier, store name/description/timezone, and the full list of REST namespaces the site publishes
• 💳 Payment gateway check — the gateway IDs the store's empty-cart endpoint offers (stripe, woocommerce_payments, ppcp-gateway, and so on), plus the currency and default customer country served
• 🧩 Extension detection — the WooCommerce extensions that register themselves on the cart endpoint (subscriptions, smart coupons, table-rate shipping, delivery dates…)
• 📦 Catalogue pull — 12 record types (products, categories, brands, tags, attributes, reviews, pages, posts, comments, post-categories, post-tags, users), each with its own filters and up to 1,000 records per store
• 🚦 Merchant filters — skip stores without the Store API, keep only stores above a minimum catalogue size, keep only stores whose default customer country matches a list you set
• ⚡ Qualify-only mode — stop after the profile row and skip the catalogue crawl entirely, for screening a long prospect list cheaply

What data can you get with WooCommerce Scraper (Store Setup & Payment Check)?

Every run can return two kinds of rows in the same dataset: one store profile row per merchant, and catalogue rows for whichever record type you choose.

Result type	Extracted fields	Primary use case
Store profile (row_type: store_profile)	store_api_available, wordpress_api_available, catalogue_size, catalogue_tier, payment_methods, currency_served_to_visitor, default_customer_country, detected_extensions, rest_namespaces, store_name, store_description, store_timezone, store_gmt_offset	Prospect qualification, payment-gateway market research, integration readiness checks
Products (row_type: products)	id, sku, name, type, prices, price_value, on_sale, is_in_stock, average_rating, review_count, categories, tags, brands, attributes, variations, images, add_to_cart	Catalogue export, price monitoring, competitor tracking
Categories / brands / tags / attributes	Raw store fields (id, name, slug, count, …), exactly as the store's API publishes them	Taxonomy mapping, building filter values for a second run
Reviews	Raw store fields (rating, review text, reviewer, date, product id, …)	Reputation and sentiment tracking
Pages / posts / comments / users	Raw WordPress REST fields (title, content, author, date, …)	Content and editorial monitoring on the same domain

Store setup & payment profile

This is the record type the other WooCommerce scrapers checked for this README don't return. It answers, per merchant, whether the site is a reachable WooCommerce storefront at all (store_api_available), how big its catalogue really is (catalogue_size, read from one X-WP-Total header — no paging), which payment gateway IDs it offers, which currency and default customer country it serves, and which WooCommerce extensions register themselves on the cart. A field the store didn't publish is emitted as null, never guessed:

{
  "row_type": "store_profile",
  "store": "https://example-shop.com",
  "store_api_available": true,
  "catalogue_size": 214,
  "catalogue_tier": "small",
  "payment_methods": ["stripe", "woocommerce_payments", "cod"],
  "currency_served_to_visitor": "USD",
  "default_customer_country": "US",
  "detected_extensions": ["woocommerce-smart-coupons", "woocommerce-subscriptions"],
  "checked_at": "2026-08-16T09:12:04Z"
}

Product catalogue rows

When resource is products (the default), each row carries the full Store API product object: identity (id, sku, slug, type), pricing already converted to real currency units (price_value, regular_price_value, sale_price_value) alongside the raw prices block, stock (is_in_stock, stock_availability, low_stock_remaining), and taxonomy (categories, tags, brands, attributes). Variable products carry a variations id list you can expand into full variation rows with include_variations.

Why not build this yourself?

The WooCommerce Store API this Actor reads is public and keyless, so nothing here is technically locked away — but turning "GET a few endpoints" into a reliable qualification-and-crawl pipeline has real edges. per_page caps at 100 on the Store API — send 101 and the store answers HTTP 400 rest_invalid_param, and the pagination loop has to know that before it ever runs. Sort support differs per endpoint: products accept 11 orderby values, the taxonomy endpoints (categories, brands, tags) accept only slug/title/popularity, and sending an unsupported value to the wrong endpoint gets a hard 400, not a graceful fallback — this Actor maps supported sorts per resource and retries without orderby if the store still rejects it. Prices arrive as integers in the store's minor currency unit, so 9900 means 99.00 only once you know the store serves 2 decimal places. And a domain list mixes reachable WooCommerce stores, WordPress sites with no storefront, and sites that block a bare request outright — this Actor escalates automatically through no proxy, then an Apify datacenter proxy, then a residential proxy, before giving up on a store. Building and maintaining that logic yourself, across every store in a prospect list, is the part this Actor removes.

How to scrape WooCommerce stores with WooCommerce Scraper (Store Setup & Payment Check)?

1. Open the Actor's page on the Apify Store and click Try for free (or Run, if you already have it saved).
2. Enter one or more store home pages in startUrls — a bare domain or a sub-directory install both work. For a longer list, point dev_fileupload at a text or JSON file of URLs instead.
3. Set the real query controls: resource for the record type, limit for records per store, sort/order, and the merchant filters onlyStoreCountries and minCatalogueSize if you only want stores that qualify.
4. Start the run.
5. Open the dataset when it finishes and export as JSON or CSV, or pull it through the Apify API.

{
  "startUrls": ["https://example-shop.com", "https://another-store.com"],
  "resource": "products",
  "limit": 50,
  "onlyStoreCountries": "US,CA",
  "minCatalogueSize": 50
}

How to run multiple stores in one job

Add every store to the startUrls array, or upload a text file (one URL per line) or a JSON array of URLs to dev_fileupload — its contents are merged into the same list. Stores are worked through one at a time in a single run, with a short pause between them, so a 200-store list takes proportionally longer than a 20-store one; there is no documented concurrency setting to raise. For very large lists, split them across a few runs instead.

⬇️ Input

Point the Actor at store URLs, choose whether to write a store profile row, a catalogue pull, or both, and layer on merchant and catalogue filters. Nothing is required — leaving startUrls empty and adding no URLs anywhere fails the run.

Parameter	Required	Type	Description	Example value
startUrls	No	array	🏪 Merchant store URLs to qualify	["https://example-shop.com"]
url	No	array	Store URLs (alternative field), merged with startUrls	["https://another-store.com"]
dev_fileupload	No	string	📥 URL of a text/JSON file of store URLs, merged in	https://example.com/prospects.txt
includeStoreProfile	No	boolean, default true	🧾 Write a store setup profile row per store	true
profileOnly	No	boolean, default false	⚡ Stop after the profile row; skip the catalogue crawl	true
checkPaymentMethods	No	boolean, default true	💳 Read payment gateway IDs, currency and default customer country	true
detectExtensions	No	boolean, default true	🧩 List WooCommerce extensions detected on the cart endpoint	true
attachStoreSetupToRows	No	boolean, default true	🔗 Stamp the store setup onto every catalogue row	true
skipUnsupportedStores	No	boolean, default true	🚦 Skip the catalogue crawl for sites without the WooCommerce Store API	true
onlyStoreCountries	No	string	🌍 Comma-separated ISO-2 country codes; keeps only stores whose default customer country matches	US,CA,GB
minCatalogueSize	No	integer, default 0, min 0	📦 Skip merchants publishing fewer than this many products	50
resource	No	string, default products, enum: products, categories, brands, tags, attributes, reviews, pages, posts, comments, post-categories, post-tags, users	📁 Which record type to pull for qualifying stores	products
limit	No	integer, default 10, min 1, max 1000	♾️ Maximum catalogue records per store	100
include_variations	No	boolean, default false	Also return every variation of variable products found	false
format	No	string, default md, enum: md, text, html	How product descriptions and review bodies are rendered	md
sort	No	string, default date, enum: "", date, modified, id, include, title, slug, price, popularity, rating, menu_order, comment_count	Sort order requested from the store; unsupported values fall back to the store's own order	id
order	No	string, default "", enum: "", asc, desc	Sort direction, applied only when the sort value is supported	asc
search	No	string	Return only records matching this text (not supported for attributes/reviews)	hoodie
category	No	string	Product category ID(s), comma separated. Products only	15,22
tag	No	string	Product tag ID(s), comma separated. Products only	8
sku	No	string	Return only products with this SKU. Products only	SKU-1029
product_type	No	string, enum: "", simple, grouped, external, variable, wbs_bundle, variation	Return only products of this type. Products only	variable
stock	No	string, enum: "", instock, outofstock, onbackorder	Return only products with this stock status. Products only	instock
rating	No	string, enum: "", 1, 2, 3, 4, 5	Return only products at this rounded star rating. Products only	4
min_price	No	integer, min 0	Minimum price in the store's minor currency unit. Products only	1999
max_price	No	integer, min 0	Maximum price in the store's minor currency unit. Products only	4999
featured	No	boolean, default false	Return only products marked featured. Products only	false
sale	No	boolean, default false	Return only products currently on sale. Products only	true
proxyConfiguration	No	object	Apify proxy configuration; falls back to no proxy, then datacenter, then residential	{"useApifyProxy": true}
dev_proxy_config	No	string	🌐 Your own HTTP(S) proxy URL instead of the Apify proxy	http://user:pass@proxy.example.com:8000
dev_custom_headers	No	string	📜 Extra HTTP headers as JSON array/object or Name: value lines	[{"name":"Accept-Language","value":"en-GB"}]
dev_custom_cookies	No	string	🍰 Extra HTTP cookies as JSON array/object or name=value lines	[{"name":"session","value":"abc123"}]
dev_transform_fields	No	string	♻️ Comma-separated field paths to keep; dots for nested keys, indices for array elements	prices.price,sku,name
dev_dataset_name	No	string	📁 Write to a named dataset instead of the default one; supports {ACTOR}, {DATE}, {TIME} masks	leads-{DATE}
dev_dataset_clear	No	boolean, default false	Empty the named dataset before writing (no effect without dev_dataset_name)	false
dev_no_strip	No	boolean, default false	Keep empty (null/""/[]/{}) values instead of stripping them	false

Common pitfall: min_price and max_price are entered in the store's minor currency unit, not a decimal amount — on a store that serves 2 decimal places, 1999 means 19.99, not 1999.00. The store profile row's currency_minor_unit_served field tells you which unit a given store uses before you set these.

Example input — screen a prospect list without touching any catalogue:

{
  "startUrls": [
    "https://example-shop.com",
    "https://another-store.com"
  ],
  "profileOnly": true,
  "skipUnsupportedStores": true
}

⬆️ Output

Results are written to the run's Apify dataset — one JSON row per store profile and per catalogue record, ready to export as JSON, CSV, Excel, or fetched through the Apify API. Empty values are stripped by default (dev_no_strip keeps them as null); booleans and zeros are always kept.

Scraped results

A run with resource: "products", attachStoreSetupToRows: true and limit: 3 produces one store profile row followed by the catalogue rows:

[
  {
    "row_type": "store_profile",
    "url": "https://example-shop.com",
    "name": "Example Shop",
    "store": "https://example-shop.com",
    "resource_type": "store_profile",
    "checked_at": "2026-08-16T09:12:04Z",
    "store_api_available": true,
    "wordpress_api_available": true,
    "store_api_namespace": "wc/store/v1",
    "store_api_route_count": 31,
    "store_name": "Example Shop",
    "store_description": "Outdoor gear and apparel",
    "store_home": "https://example-shop.com",
    "store_timezone": "America/New_York",
    "store_gmt_offset": -4,
    "rest_namespaces": ["oembed/1.0", "wp/v2", "wc/store/v1", "wc/store"],
    "rest_namespace_count": 4,
    "catalogue_size": 214,
    "catalogue_size_available": true,
    "catalogue_tier": "small",
    "cart_api_available": true,
    "payment_methods": ["stripe", "woocommerce_payments", "cod"],
    "payment_method_count": 3,
    "payment_methods_available": true,
    "currency_served_to_visitor": "USD",
    "currency_symbol_served": "$",
    "currency_minor_unit_served": 2,
    "default_customer_country": "US",
    "default_customer_state": "NY",
    "detected_extensions": ["woocommerce-smart-coupons", "woocommerce-subscriptions"],
    "detected_extension_count": 2
  },
  {
    "row_type": "products",
    "url": "https://example-shop.com/product/trail-jacket",
    "id": 1042,
    "name": "Trail Jacket",
    "slug": "trail-jacket",
    "type": "simple",
    "sku": "TJ-100",
    "on_sale": true,
    "prices": {
      "price": "8999",
      "regular_price": "10999",
      "currency_code": "USD",
      "currency_minor_unit": 2
    },
    "price_value": 89.99,
    "regular_price_value": 109.99,
    "currency_code": "USD",
    "average_rating": "4.5",
    "review_count": 32,
    "is_in_stock": true,
    "categories": [{"id": 15, "name": "Jackets"}],
    "store": "https://example-shop.com",
    "resource_type": "products",
    "store_name": "Example Shop",
    "catalogue_size": 214,
    "payment_methods": ["stripe", "woocommerce_payments", "cod"]
  },
  {
    "row_type": "products",
    "id": 1043,
    "name": "Trail Backpack",
    "sku": "TB-210",
    "type": "simple",
    "on_sale": false,
    "price_value": 64.0,
    "currency_code": "USD",
    "is_in_stock": true,
    "average_rating": "4.0",
    "review_count": 11,
    "store": "https://example-shop.com",
    "resource_type": "products"
  },
  {
    "row_type": "products",
    "id": 1044,
    "name": "Insulated Bottle",
    "sku": "IB-05",
    "type": "simple",
    "on_sale": false,
    "price_value": 24.5,
    "currency_code": "USD",
    "is_in_stock": false,
    "stock_availability": {"text": "Out of stock"},
    "average_rating": null,
    "review_count": 0,
    "store": "https://example-shop.com",
    "resource_type": "products"
  }
]

Every field transform_product writes is available, not just the dataset's default view columns — slug, parent, variation, short_description, description, sale_price_value, price_range, images, tags, brands, attributes, variations, grouped_products, has_options, is_purchasable, is_on_backorder, low_stock_remaining, sold_individually, add_to_cart and extensions all ship on every product row; they're simply not in the dataset's default table view. Switching resource to any other record type returns that type's fields exactly as the store's own API publishes them.

How can I use the data extracted with WooCommerce Scraper (Store Setup & Payment Check)?

• 🏢 Ecommerce agencies and SaaS app vendors: run a domain list with profileOnly on and get a scored prospect list back — catalogue_tier, payment_methods and detected_extensions tell you which merchants are worth a pitch before you spend a sales call on them.
• 🤖 AI engineers and LLM developers: an agent issues a store URL, receives the structured profile-plus-catalogue JSON, and passes it to the model as grounded context for a merchant-research or competitive-analysis task — no scraping code inside the agent itself.
• 📊 Market researchers: aggregate payment_methods and detected_extensions across a vertical to track gateway adoption (stripe vs woocommerce_payments vs ppcp-gateway) and extension market share.
• 🧩 Product and partnerships teams: store_api_route_count, rest_namespaces and application_passwords_url tell an integration engineer what a prospective merchant's site exposes before the first onboarding call.

How do you monitor a merchant's store setup over time?

Store setup drifts: a merchant adds a gateway, swaps currencies for a region, installs a subscriptions plugin, or grows past your minimum catalogue threshold. Re-running this Actor against the same domain list on a schedule turns each of those into a diffable event instead of something you find out about by accident. Compare payment_methods between runs to catch a gateway switch, catalogue_size/catalogue_tier to catch catalogue growth, detected_extensions to catch a new plugin, and store_api_available/setup_error to catch a merchant migrating off WooCommerce entirely. Use dev_dataset_name with the {DATE} mask (for example leads-{DATE}) so each scheduled run lands in its own named dataset, and diff consecutive runs outside the Actor.

To automate the schedule itself, use Apify's built-in Schedules to run this Actor on a cron interval, and either poll the resulting dataset through the Apify API or attach an Apify webhook to the run to push each run's completion to your own endpoint.

Integrate WooCommerce Scraper (Store Setup & Payment Check) and automate your workflow

WooCommerce Scraper (Store Setup & Payment Check) works with any language or tool that can send an HTTP request.

REST API with Python

import requests

ACTOR_ID = "YOUR_USERNAME~woocommerce-scraper-store-setup-and-payment-check"
TOKEN = "YOUR_APIFY_TOKEN"

resp = requests.post(
    f"https://api.apify.com/v2/acts/{ACTOR_ID}/run-sync-get-dataset-items",
    params={"token": TOKEN},
    json={"startUrls": ["https://example-shop.com"], "resource": "products", "limit": 20},
)
rows = resp.json()
products = [r for r in rows if r.get("row_type") == "products"]
print(products[0]["name"], products[0]["price_value"])

MCP for query-grounded AI agents

This Actor is reachable through Apify's hosted MCP server at https://mcp.apify.com, which exposes any public Actor as a callable tool to an MCP-aware client. For a local server limited to just this Actor: npx @apify/actors-mcp-server --tools YOUR_USERNAME/woocommerce-scraper-store-setup-and-payment-check. An agent calls the tool with a store URL, receives the profile-plus-catalogue JSON back, and grounds its answer in it. Works with Claude Desktop, Cursor, and other MCP-compatible clients.

Scheduled monitoring and delivery

Use Apify's Schedules to trigger this Actor on a cron interval, and Apify webhooks to notify your own endpoint when a run finishes so you can pull the fresh dataset without polling.

Is it legal to scrape WooCommerce store data?

Yes — this Actor reads only endpoints a store's own storefront already answers to any anonymous visitor, so it returns nothing a shopper couldn't see by loading the shop page. What it returns is business and product data — payment gateways offered, catalogue contents, store configuration — not personal data about individuals, so GDPR/CCPA framing generally doesn't apply to the profile and product rows; the applicable considerations are the target site's terms of service and, in the EU, database-rights protection over a compiled catalogue. The reviews record type is the one exception worth a second look: a store's product reviews can carry a reviewer's name alongside their comment, which is personal data even when posted publicly. Scraping for internal monitoring or research carries a different risk profile than scraping to build a dataset for AI training or redistribution. Consult your legal team for commercial use cases involving bulk data storage.

Frequently asked questions

Can I filter stores by country or currency?

You can filter on onlyStoreCountries, but read what it actually filters before you rely on it. It matches the default customer country WooCommerce's cart endpoint serves — the store's base country, or a geolocated guess based on where the request came from — not a list of countries the store ships to. ⚠️ The same is true of currency_served_to_visitor: it's what WooCommerce quotes to this request, which multi-currency setups can vary by visitor location, not a guaranteed read of the merchant's own books.

How do I control which catalogue records come back?

Set resource to one of the 12 supported record types, limit for how many per store (1–1000, default 10), and sort/order for ordering. Each resource supports a different subset of sort values — an unsupported one is logged and the store's own default order is used instead.

How does WooCommerce Scraper handle blocked requests?

It escalates automatically through three proxy tiers — no proxy, then an Apify datacenter proxy, then a residential proxy — retrying up to 3 times on the residential tier, and it sends a standard browser User-Agent. You can also supply dev_custom_headers and dev_custom_cookies, or your own proxy via dev_proxy_config.

Does it extract payment gateways and store setup?

Yes — every store gets a row_type: "store_profile" row with payment_methods (gateway IDs like stripe, woocommerce_payments, ppcp-gateway), currency_served_to_visitor, default_customer_country and detected_extensions. Gateway display titles are not included: WooCommerce only publishes those behind a checkout nonce that a logged-out client doesn't have, so this Actor returns IDs and nothing invented in their place. If checkPaymentMethods or detectExtensions is off, or the cart endpoint doesn't answer, those fields come back empty and setup_error explains why.

How many results does it return per store?

Up to limit catalogue records per store (1–1000, default 10), plus one store profile row on top of that count when includeStoreProfile is on. The Store API itself caps each request page at 100 records; the Actor pages automatically up to your limit.

How do I monitor a merchant's setup over time?

Run this Actor on the same domain list on a schedule, write each run to a dated named dataset with dev_dataset_name: "leads-{DATE}", and diff payment_methods, catalogue_size, detected_extensions and store_api_available against the previous run to catch what changed.

Does it work with Claude, ChatGPT and AI agent frameworks?

Yes, two ways: as an HTTP endpoint any agent framework can call directly (see the Python example above), or through Apify's hosted MCP server at https://mcp.apify.com, which lets MCP-aware clients like Claude Desktop call it as a tool and ground their answers in the returned JSON.

How does it compare to other WooCommerce scrapers?

As observed on their Apify Store listings on 2026-08-16, competing WooCommerce scrapers such as trovevault/woocommerce-products-scraper, piotrv1001/woocommerce-product-scraper and jupri/woocommerce extract product catalogues — name, price, SKU, stock, ratings, categories — but none of them checks payment gateways, currency, default customer country, installed extensions, or whether a domain is a reachable WooCommerce storefront in the first place. This Actor adds that qualification step as its own record type, in the same run as the catalogue pull.

Can I use this without managing proxies or store credentials?

Yes. No login, API key, consumer key/secret or cookie is required for any endpoint this Actor reads, and by default it makes direct requests, escalating to an Apify datacenter proxy and then a residential proxy on its own only if a store blocks the direct request. You can still supply your own proxy via dev_proxy_config or an Apify proxy configuration via proxyConfiguration if you want to control it directly.

Your feedback

Found a bug, or a store setup field you'd like added? Open an issue on this Actor's Issues tab on Apify Console — reports directly shape what ships next.