# BWS Product Scraper 🍺 (`mrdoe/bws-actor`) Actor

Extract product data from bws.com.au, Australia's largest liquor retailer. Scrape any category or search page for prices, specials, brands, and ratings — with auto-pagination. Ideal for price monitoring, competitor research, and catalog analysis. AU residential proxy included.

- **URL**: https://apify.com/mrdoe/bws-actor.md
- **Developed by:** [MrDoe](https://apify.com/mrdoe) (community)
- **Categories:** E-commerce, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.80 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

![Bws Actor hero](https://api.apify.com/v2/key-value-stores/kE36venAoVchGsE6b/records/bws-actor--hero.png)

### What does BWS Product Scraper do?

**BWS Product Scraper** collects clean, structured product data from any category or search results page on [bws.com.au](https://www.bws.com.au), one of Australia's largest liquor retailers. Give it a listing URL — a beer, wine, or spirits category, or a search results page — and it returns structured data for every product listed there, automatically working through the site's "Load More" pagination to collect as many products as you need.

Running on the Apify platform means automatic proxy handling, one-click scheduling for recurring price checks, dataset export in multiple formats, and an API for pulling results straight into your own systems — no infrastructure to manage.

### Why use BWS Product Scraper?

- **Price monitoring** — track prices and special offers across the BWS catalog over time.
- **Competitor research** — benchmark your own pricing against BWS's listed prices.
- **Catalog and product research** — pull clean product names, brands, images, volumes, and ratings to enrich your own product database.
- **Assortment analysis** — see which brands and products BWS stocks in a given category.
- **Recurring monitoring** — schedule runs to keep price and availability data current.

Designed for practical, cost-efficient product data collection: clean structured output without unnecessary data, and control over run size with `maxItems`.

### What data can you extract?

![Bws Actor dataset output](https://api.apify.com/v2/key-value-stores/kE36venAoVchGsE6b/records/bws-actor--output.png)

![Bws Actor data fields](https://api.apify.com/v2/key-value-stores/kE36venAoVchGsE6b/records/bws-actor--fields.png)

- SKU / product ID
- Product name and brand
- Product URL
- Image URL
- Volume and pack size (e.g. Bottle, Pack (6), Case (24))
- Price and regular price
- Special/discount status and savings amount
- Currency
- Rating and review count
- Sponsored listing flag
- Source category URL

### How to use it

![How Bws Actor works](https://api.apify.com/v2/key-value-stores/kE36venAoVchGsE6b/records/bws-actor--how-it-works.png)

1. Click **Try for free** or **Run** on the Actor page.
2. In **Start URLs**, paste one or more BWS category or search URLs (e.g. `https://www.bws.com.au/beer/australian-beer`).
3. Optionally adjust **Max products per start URL**. Leave it at `0` to collect everything available on that listing.
4. Click **Start** and wait for the run to finish.
5. Open the **Dataset** tab to preview, filter, and export your results.

### Input Parameters

| Parameter | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `startUrls` | array | Yes | `[{ "url": "https://www.bws.com.au/beer/australian-beer" }]` | Category or search listing pages to scrape. Add multiple URLs to scrape several listings in one run. |
| `maxItems` | integer | No | `5` | Maximum products to collect **per start URL**. `0` = no limit — collect everything available on that listing. |
| `proxyConfiguration` | object | No | Apify Proxy (Residential, AU) | Proxy settings for the crawl. |

Example input:

```json
{
    "startUrls": [{ "url": "https://www.bws.com.au/beer/australian-beer" }],
    "maxItems": 0,
    "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"], "apifyProxyCountry": "AU" }
}
```

### Output Data

Each dataset item represents one product. Every start URL you provide gets its own `maxItems` budget, and duplicate products (matched by SKU) are removed automatically.

```json
{
    "sku": "363586",
    "name": "Stone & Wood Pacific Ale Bottles 330mL",
    "brand": "Stone & Wood",
    "url": "https://www.bws.com.au/product/363586/stone-wood-pacific-ale-bottles-330ml",
    "imageUrl": "https://edgmedia.bws.com.au/bws/media/products/363576-1.png?impolicy=Prod_Retina_SM",
    "volume": "330ML",
    "packLabel": "Case (24)",
    "price": 84,
    "regularPrice": 90,
    "onSpecial": true,
    "savingsAmount": 6,
    "currency": "AUD",
    "rating": 4.5,
    "reviewCount": 155,
    "sponsored": true,
    "categoryUrl": "https://www.bws.com.au/beer/australian-beer"
}
```

| Field | Description |
| --- | --- |
| `sku` | BWS's internal product ID. |
| `name` | Full product name (brand + title). |
| `brand` | Product brand/producer. |
| `url` | Canonical product page URL. |
| `imageUrl` | Product image URL. |
| `volume` | Bottle/can size, when available. |
| `packLabel` | Selected pack configuration shown on the listing (e.g. `Bottle`, `Pack (6)`, `Case (24)`). |
| `price` | Current listed price for the shown pack size. |
| `regularPrice` | Regular price before any discount. Equal to `price` when not on special. |
| `onSpecial` | Whether the product is currently discounted from its regular price. |
| `savingsAmount` | Dollar amount saved, when on special. |
| `rating` | Average star rating (0–5), when reviews exist. Otherwise `null`. |
| `reviewCount` | Number of customer reviews. |
| `sponsored` | Whether the listing is a sponsored/promoted placement. |
| `categoryUrl` | The start URL this product was collected from. |

### Usage Examples

- **Track weekly specials**: Point `startUrls` at your key categories and schedule a weekly run to see how prices and specials change.
- **Build a competitor price index**: Scrape a set of category URLs, export to CSV, and join against your own catalog by product name or brand.
- **Research a category**: Set `maxItems` to `0` on a single category URL to pull the full assortment for analysis.

### Tips for Best Results

- Use a specific leaf category URL (e.g. `/beer/australian-beer`, `/beer/craft-beer`) rather than a top-level hub page (e.g. `/beer`) — hub pages link to subcategories but don't list products themselves.
- Use `maxItems: 0` when you want the full listing; otherwise set a smaller number for quick tests or sampling.
- BWS applies bot protection to non-residential traffic — keep the default Australian residential proxy setting for reliable results.
- Provide multiple start URLs to cover several categories in a single run; each gets its own `maxItems` budget.

### Integrations

Connect this Actor to Make, Zapier, Google Sheets, Slack, or your own systems via Apify's [integrations](https://apify.com/integrations) and [API](https://docs.apify.com/api/v2), or schedule recurring runs directly from the Apify platform.

### Export Formats

Datasets can be exported as JSON, CSV, Excel, or XML directly from the **Dataset** tab, or pulled programmatically via the Apify API.

### FAQ

**How many products can I collect?**
As many as the listing exposes. Set `maxItems: 0` to collect the full category or search listing.

**Can I scrape a whole category?**
Yes — point a start URL at any leaf category page and the Actor will page through all available products up to your `maxItems` setting.

**Does `maxItems = 0` collect everything?**
Yes, for each start URL. The Actor keeps clicking "Load More" until BWS stops returning new products.

**Can I provide multiple URLs?**
Yes. Add multiple entries to `startUrls`; `maxItems` applies separately to each one.

**Are duplicates removed?**
Yes, products are deduplicated by SKU within each listing page.

**Why can some fields be null?**
Not every product has a rating or review — those fields are `null` when the site doesn't show them.

**Can I schedule price monitoring?**
Yes — use the Apify platform's built-in Scheduler to run this Actor automatically on a recurring basis.

**Is cache available?**
Not currently required for this Actor; each run collects fresh data directly from the listing pages you provide.

**Is the data current?**
Data reflects what was published on the site at the time of the run.

**What happens when a product is unavailable?**
If a product is out of stock or removed, it simply won't appear in the listing — the Actor only reports what BWS currently displays.

### Legal / Responsible Use

This Actor only collects publicly visible product listing data. Users are responsible for complying with BWS's terms of use, applicable law, and responsible data-use requirements.

### Preview

![Bws Actor input options](https://api.apify.com/v2/key-value-stores/kE36venAoVchGsE6b/records/bws-actor--input.png)

# Actor input Schema

## `startUrls` (type: `array`):

Category, brand, or search listing pages on bws.com.au to scrape products from, e.g. https://www.bws.com.au/beer/australian-beer

## `maxItems` (type: `integer`):

Maximum number of products to scrape from each start URL. Set to 0 for no limit (scrape until the 'Load More' button is exhausted).

## `proxyConfiguration` (type: `object`):

Proxy settings for the crawl. BWS actively blocks non-residential traffic, so an Australian residential proxy is strongly recommended.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.bws.com.au/beer/australian-beer"
    }
  ],
  "maxItems": 5,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ],
    "apifyProxyCountry": "AU"
  }
}
```

# Actor output Schema

## `products` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.bws.com.au/beer/australian-beer"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("mrdoe/bws-actor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://www.bws.com.au/beer/australian-beer" }] }

# Run the Actor and wait for it to finish
run = client.actor("mrdoe/bws-actor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.bws.com.au/beer/australian-beer"
    }
  ]
}' |
apify call mrdoe/bws-actor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,mrdoe/bws-actor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/F5tllajihP05rFQdb/builds/5IqaBuG6VQb9A8get/openapi.json