# John Lewis Scraper — UK Department Store Prices & Stock (`studio-amba/johnlewis-scraper`) Actor

Scrape product data from johnlewis.com, the UK's largest department-store group. Extract names, brands, prices, ratings, stock and images by category or search term. No login, no cookies.

- **URL**: https://apify.com/studio-amba/johnlewis-scraper.md
- **Developed by:** [Studio Amba](https://apify.com/studio-amba) (community)
- **Categories:** E-commerce
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.20 / 1,000 result scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## John Lewis Scraper

Pull product data from johnlewis.com, John Lewis & Partners -- the UK's largest department-store group, spanning fashion, home, electricals, furniture and beauty. This scraper reads the same JSON data John Lewis' own Next.js storefront embeds in every category and search page, so it returns clean, structured data -- brand, price, rating, stock, category and images -- without any per-product page fetching.

### How to scrape John Lewis data

Give the actor either a search term (e.g. `"headphones"`, `"air fryer"`, `"dress"`) or one or more John Lewis category page URLs (e.g. `https://www.johnlewis.com/browse/electricals/televisions/view-all-tvs/_/N-6srf`), and it paginates through the results until it hits `maxResults` or runs out of matching products. You can combine both in a single run.

Under the hood, every John Lewis category and search page is server-rendered by a Next.js (Pages Router) app that embeds the full product listing -- title, brand, price, rating, review count, stock status, images and the breadcrumb category path -- as JSON inside a single `<script id="__NEXT_DATA__">` tag on the page. This actor requests that page directly and reads the embedded JSON instead of rendering the page in a browser, which makes it fast and avoids the fragility of parsing rendered HTML markup.

If you don't provide any input, it defaults to searching for `"dress"` as a demo.

#### Why this needs a residential proxy

johnlewis.com blocks Apify's automatic/datacenter proxy pool outright for both HTTP/1.1 and HTTP/2 requests -- the connection doesn't complete at all, on every attempt tested. A residential exit IP gets a clean response every time, with no retries needed. Because of this, the `proxyConfiguration` input is prefilled with the `RESIDENTIAL` group and `GB` country and should be left as-is; switching to the automatic pool will make every request fail.

### Input

| Field | Type | Description |
|-------|------|-------------|
| `searchQuery` | string | Single search term, e.g. `"headphones"` |
| `searchQueries` | array | Multiple search terms, combined with `startUrls` if both are given (max 20) |
| `startUrls` | array | John Lewis category (`/browse/...`) page URLs to scrape (max 20) |
| `maxResults` | integer | Cap on number of products returned across all inputs combined (default: 100, max: 2000) |
| `proxyConfiguration` | object | Apify proxy settings. Prefilled to `RESIDENTIAL` / `GB` -- required, see above |

### What you get back

| Field | Type | Example |
|-------|------|---------|
| `name` | string | `"Sony WH-CH720 Noise Cancelling Bluetooth Wireless Over-Ear Headphones"` |
| `brand` | string | `"Sony"` |
| `price` | number | null | `67.00` |
| `originalPrice` | number | null | `null` (only populated when a higher pre-markdown price is exposed) |
| `currency` | string | `"GBP"` |
| `productId` | string | `"110194491"` |
| `sku` | string | `"110163003"` |
| `inStock` | boolean | null | `true` (straight from John Lewis' own stock flag; `null` only if that flag is missing entirely) |
| `rating` | number | null | `4.6` on a 0-5 scale |
| `reviewCount` | integer | null | `2551` |
| `category` | string | null | `"Headphones"` |
| `categories` | array | null | Full breadcrumb path, e.g. `["Electricals", "Headphones"]` |
| `imageUrl` | string | null | Primary product image |
| `imageUrls` | array | All product images found on the listing card |
| `url` | string | Full product page URL |
| `source` | string | Always `"johnlewis.com"` |
| `searchQuery` | string | null | The search term used to find this product, when scraped via search |
| `categoryUrl` | string | null | The category page used to find this product, when scraped via a category URL |
| `scrapedAt` | string | ISO timestamp |

### Sample output

```json
{
  "name": "Sony WH-CH720 Noise Cancelling Bluetooth Wireless Over-Ear Headphones with Mic/Remote",
  "brand": "Sony",
  "price": 67.00,
  "originalPrice": null,
  "currency": "GBP",
  "productId": "110194491",
  "sku": "110163003",
  "inStock": true,
  "rating": 4.6,
  "reviewCount": 2551,
  "category": "Headphones",
  "categories": ["Electricals", "Headphones"],
  "imageUrl": "https://media.johnlewiscontent.com/i/JohnLewis/110163003?",
  "imageUrls": [
    "https://media.johnlewiscontent.com/i/JohnLewis/110163003?",
    "https://media.johnlewiscontent.com/i/JohnLewis/110163003alt2"
  ],
  "url": "https://www.johnlewis.com/sony-wh-ch720-noise-cancelling-bluetooth-wireless-over-ear-headphones-with-mic-remote/p110194491",
  "source": "johnlewis.com",
  "searchQuery": "headphones",
  "categoryUrl": null,
  "scrapedAt": "2026-09-14T19:42:31.145Z"
}
```

### Use cases

- **Price monitoring** -- track John Lewis' price changes over time on the products you sell against, department store-wide.
- **Assortment research** -- John Lewis carries a wide mix of own-brand (ANYDAY, John Lewis & Partners) and third-party brands across fashion, home, electricals and beauty; see which brands and categories it's carrying at what price points.
- **Rating and review tracking** -- `rating` and `reviewCount` come straight from the listing, no extra requests needed, useful for popularity/quality signals across a category.
- **Stock tracking** -- `inStock` reflects John Lewis' own listing-level stock flag.
- **Market comparison** -- as the UK's largest department-store group, John Lewis is a natural comparison point against Debenhams, Next, House of Fraser, Selfridges and Harrods.

### Coverage

v1 covers the UK flagship storefront (`www.johnlewis.com`, GBP pricing).

### Performance and cost

The scraper reads John Lewis' own server-rendered listing JSON directly -- no per-product page fetches, no browser rendering -- so throughput is one HTTP request per 72 products. A run of 100 products typically completes in under 15 seconds. This actor charges $0.005 per run (start fee) plus $0.002 per result -- a 100-product run costs about $0.205 in platform fees. Your run's usage cost only settles after the run reports SUCCEEDED -- checking cost mid-run will undercount it.

### Limitations

- `originalPrice` is only populated when John Lewis' own promotion-history data exposes a pre-markdown price higher than the current one -- many products simply don't have one.
- `category`/`categories` reflect the breadcrumb of the page a product was found on (the category URL, or the generic "search results" label for search), not a per-product category lookup.
- v1 targets the UK flagship catalogue (GBP) only.

### Related scrapers

If you're covering UK department-store and general retail more broadly, also check out:

- [Debenhams Scraper](https://apify.com/studio-amba/debenhams-scraper) -- relaunched UK department-store brand
- [Selfridges Scraper](https://apify.com/studio-amba/selfridges-scraper) -- UK luxury department store
- [Harrods Scraper](https://apify.com/studio-amba/harrods-scraper) -- UK luxury department store
- [Next Co UK Scraper](https://apify.com/studio-amba/next-co-uk-scraper) -- UK multi-brand fashion retailer
- [Boots Scraper](https://apify.com/studio-amba/boots-scraper) -- UK health, beauty and lifestyle retail

### Support

Found an issue or need a different locale covered? Open an issue on this actor's Apify Store page and we'll take a look.

# Actor input Schema

## `searchQuery` (type: `string`):

Search term to find products on johnlewis.com (e.g. "headphones", "dress", "air fryer").

## `searchQueries` (type: `array`):

One or more search terms to run through John Lewis' site search. Combined with any Category URLs below. Capped at 20 terms per run.

## `startUrls` (type: `array`):

johnlewis.com category (browse) pages to scrape, e.g. https://www.johnlewis.com/browse/women/womens-dresses/*/N-flw or https://www.johnlewis.com/browse/electricals/televisions/view-all-tvs/*/N-6srf. A single category can hold thousands of products; use Max Results to cap how many are returned. Capped at 20 URLs per run.

## `maxResults` (type: `integer`):

Stop after this many product rows across all category URLs and search queries combined. Some John Lewis categories hold 7,000+ products, so this cap always applies. Hard-capped at 2000 per run regardless of the value entered.

## `proxyConfiguration` (type: `object`):

johnlewis.com blocks Apify's automatic/datacenter proxy pool outright (verified live) -- RESIDENTIAL is required and prefilled here. Do not switch this to the automatic pool, it will not connect.

## Actor input object example

```json
{
  "searchQuery": "headphones",
  "searchQueries": [],
  "startUrls": [],
  "maxResults": 100,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ],
    "apifyProxyCountry": "GB"
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQuery": "headphones",
    "maxResults": 100,
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ],
        "apifyProxyCountry": "GB"
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("studio-amba/johnlewis-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQuery": "headphones",
    "maxResults": 100,
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
        "apifyProxyCountry": "GB",
    },
}

# Run the Actor and wait for it to finish
run = client.actor("studio-amba/johnlewis-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQuery": "headphones",
  "maxResults": 100,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ],
    "apifyProxyCountry": "GB"
  }
}' |
apify call studio-amba/johnlewis-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,studio-amba/johnlewis-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/B6dnnspcKE04HJLlw/builds/gB2auo1M6dTtxODpx/openapi.json