Big W AU Product Scraper

Scrape product listings, categories, prices, and detailed item data from Big W Australia (bigw.com.au).

Use this scraper for price monitoring, market research, product research, and availability checks on Australia's Big W retail catalog.

Disclaimer: This is an unofficial scraper and is not affiliated with, endorsed by, or sponsored by Big W or Woolworths Group.

Supported operations

Operation	Description
search	Keyword search via sitemap URL matching
category	List products for a category
categories	Full category tree from homepage data
item	Full product details with apix price enrichment
search_filters	Static category facets and sort options
probe	Health check for sitemap and apix backends

Not supported: store locator, reviews, barcode lookup, postcode-specific pricing.

Input guide by operation

The Apify Console form shows all fields at once. Only fill the fields relevant to your chosen operation.

search (default)

Field	Required	Notes
query	Yes	e.g. sale, milk, lego
page	No	Default 1
limit	No	Default 5, max 50
max_scan_urls	No	Safety cap for sitemap scan (default 250000)

category

Field	Required	Notes
category_id or slug or url	One required	e.g. slug=baby, category_id=61
page, limit	No	Pagination
max_scan_products	No	Default 1500

item

Field	Required	Notes
item_id or slug or url	One required	e.g. item_id=9900053407

categories

Field	Required	Notes
level	No	Filter by depth 1–6
enabled_only	No	Default true

search_filters / probe

No additional fields required.

Example inputs

Default search (fast, non-empty):

{
  "operation": "search",
  "query": "sale",
  "limit": 5
}

Category listing:

{
  "operation": "category",
  "slug": "baby",
  "limit": 5
}

Full item details:

{
  "operation": "item",
  "item_id": "9900053407"
}

Category tree:

{
  "operation": "categories"
}

Output fields

Search / category listings

id, slug, source_url, name, category, breadcrumbs

Item (full enrichment)

id, name, brand, source_url, category, categories, breadcrumbs, description, ingredients, usage_instructions, allergens, specifications, barcode, price, discount_price, currency, price_info, unit_price, price_per_unit_*, size, color, image, images, variants, options, stock_status, in_stock, attributes

Categories

id, slug, name, short_name, parent_id, depth, enabled, source_url, breadcrumbs, category

Run summary is stored in the OUTPUT key-value record. Dataset items are normalized product rows.

Proxy behavior

• Default strategy: direct_then_proxy — direct requests first, proxy fallback on block.
• Modes: auto (recommended), direct, apify, custom via proxyMode input or APIFY_WAVE1_PROXY_MODE env.
• Geography: Australian site; AU Apify proxy or custom AU proxy recommended when blocked.
• Retries: Rotates curl_cffi / httpx / tls_client / cloudscraper clients with browser cookies on 403, 429, 502, 503.
• Structured errors: Returns status: blocked or error summary instead of crashing when all strategies fail.

Limitations

• Search uses sitemap URL slug matching, not the native Big W search API (which is unstable). Broad queries can take several seconds while scanning sitemaps.
• Category listing matches product URL tokens against category slug/name — not full apix category verification.
• Pricing is national (NAT) via apix; no postcode/store-specific availability.
• Probe returns diagnostics only (may produce an empty dataset).

Free trial limit

Free/trial Apify users are limited to 10 result requests per UTC day. Paid users receive normal configured limits. Set disableFreeTrialGuard: true to bypass for private testing.

Monetization

Pay-per-event pricing:

• Actor start: ~$0.005
• Dataset item (result-item): ~$0.0003