# Johnlewis Products Details Crawler (`alkausari_mujahid/johnlewis-products-details-crawler`) Actor
Extract full product details from John Lewis — prices, specs, images, descriptions, and weight-based cost addition — Automatically calls the John Lewis Listings Crawler to discover all variant URLs, then scrapes each one at speed using CheerioCrawler.
- **URL**: https://apify.com/alkausari\_mujahid/johnlewis-products-details-crawler.md
- **Developed by:** [Alkausari M](https://apify.com/alkausari_mujahid) (community)
- **Categories:** E-commerce, Automation, Integrations
- **Stats:** 8 total users, 2 monthly users, 67.5% runs succeeded, NaN bookmarks
- **User rating**: No ratings yet
## Pricing
from $5.00 / 1,000 results
This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
**Extract structured product data from [johnlewis.com](https://www.johnlewis.com) at scale.**
Prices, stock, ratings, images, specs, breadcrumbs — 25+ fields per product, ready for JSON / CSV / Excel. No code required.
> Built and maintained by **alkausari_mujahid**.
---
### ✦ Highlights
- ⚡ **Fast at scale** — concurrent crawling, ~60 products/min
- 📊 **Any format** — JSON, CSV, Excel, XML, or live via the Apify API
- 💰 **Predictable cost** — pay per result, not per try
- 🖱 **Zero code** — paste a URL, click Start
- 🛡 **Anti-blocking built-in** — residential proxies, rotation, smart retries
- ✓ **Complete data** — 25+ fields per product, nothing left behind
---
### ⚙ How it works
1. **Paste a URL** — single product, category page, or search results.
2. **Click Start** — the actor handles proxies, retries, and parsing for you.
3. **Download or pipe** — JSON, CSV, or Excel out of the box, or pull straight from the API.
```jsonc
// Example input
{
"startUrls": [
{ "url":"https://www.johnlewis.com/browse/men/mens-shoes/_/N-7onz"}
],
"lastPageNum": 1,
"maxRequestsPerCrawl": 100,
"proxyConfiguration": { "useApifyProxy": false, "apifyProxyGroups": ["RESIDENTIAL"] }
}
````
***
For category, brand, or search URLs, this Actor automatically calls the companion **[John Lewis Products Listings Crawler](https://apify.com/alkausari_mujahid/john-lewis-products-listings-crawler)** to collect all product URLs first, then scrapes each. Direct product URLs are scraped immediately — no companion needed.
> \[IMPORTANT]
> **Billing:** when both stages run, each is billed as a **separate Actor run** in your Apify Console. The listings stage runs first; this Actor shows zero compute until it begins processing detail pages.
### 📦 What you get back
Every product returns a single structured record: You can group variants by product id like `p113951807`
```json
{
"type": "product",
"name": "Kickers Kick Hi Leather Boots",
"brand": "Kickers",
"colour": "Black",
"size": "5",
"stockStatus": "In Stock",
"currentPrice": "£110.00",
"previousPrice": null,
"weightPrice": "£3.15",
"finalPrice": 113.15,
"promotions": ["Buy one, get one 50% off"],
"reviewsCount": "124 Reviews",
"reviewsStars": "4.5 out of 5 stars",
"mainImage": "https://johnlewis.scene7.com/is/image/JohnLewis/...",
"images": [
{ "mediaContentType": "IMAGE", "originalSource": "https://..." }
],
"breadcrumbs": ["Home", "Women", "Shoes & Boots", "Ankle Boots"],
"categoryPath": "Home > Women > Shoes & Boots > Ankle Boots",
"mainCategory": "Women",
"subCategory": "Shoes & Boots",
"productId": "238795771",
"productCode": "86064111",
"specifications": {
"material": "Leather upper",
"sole": "Rubber",
"weight": "350g"
},
"weight": { "unit": "kg", "value": 0.35 },
"url": "https://www.johnlewis.com/kickers-kick-hi-leather-boots/p238795771",
"scrapedAt": "2026-05-08T10:23:45.000Z"
}
```
#### Fields captured
| Identity | Pricing | Availability | Media | Taxonomy | Reviews | Specs |
| --- | --- | --- | --- | --- | --- | --- |
| `name` | `currentPrice` | `stockStatus` | `mainImage` | `breadcrumbs[]` | `reviewsCount` | `specifications` |
| `brand` | `previousPrice` | `size` | `images[]` | `categoryPath` | `reviewsStars` | `weight` |
| `colour` | `weightPrice` | `promotions[]` | | `mainCategory` | | `productId` |
| | `finalPrice` | | | `subCategory` | | `productCode` |
Plus `url` and `scrapedAt` on every record.
***
### 💡 Use cases
- **Price monitoring** — track price changes for specific products or whole categories over time. Spot drops and promos early.
- **Stock alerts** — check which sizes and colours are in or out of stock across a range. Drive restock workflows.
- **Competitor research** — benchmark John Lewis pricing and range to inform your own merchandising.
- **Inventory analysis** — export specs, weights, and categories to build a structured product catalogue.
- **Data & BI pipelines** — pipe straight into Google Sheets, BigQuery, Snowflake, or your own tools via the Apify API and webhooks.
***
### 🚀 Get started
#### From the Apify Console
1. Open the actor on Apify.
2. Paste one or more John Lewis URLs into **Start URLs**.
3. (Optional) Set `maxRequestsPerCrawl`, enable Apify Proxy.
4. Click **Start** — results appear in the **Dataset** tab as they arrive.
#### From the Apify API
```bash
curl -X POST https://api.apify.com/v2/acts/alkausari_mujahid~johnlewis-products-details-crawler/runs?token=YOUR_TOKEN \
-H "Content-Type: application/json" \
-d '{
"startUrls": [{ "url": "https://www.johnlewis.com/browse/women/womens-shoes-boots/_/N-ebjt" }],
"maxRequestsPerCrawl": 1000
}'
```
#### Schedule it
Use Apify **Schedules** to run daily/hourly and push results to your warehouse, Google Sheets, shopify or webhook of choice.
***
### 📮 Support
Bugs, feature requests, custom scraping work — open an issue on Apify or reach out:
- **Maintainer:** Alkausari M
- **Actor:** [`alkausari_mujahid/johnlewis-products-details-crawler`](https://apify.com/alkausari_mujahid/johnlewis-products-details-crawler)
- **Need a different retailer scraped?** Alkausari builds custom Apify actors on request.
***
This actor is independent and not affiliated with, endorsed by, or sponsored by John Lewis & Partners. Use scraped data responsibly and in accordance with the target site's terms of service and applicable law.
# Actor input Schema
## `startUrls` (type: `array`):
Provide John Lewis listing pages (brand/search/browse) and/or direct product detail page URLs. Listing pages will be processed via the listings scraper; product URLs will be scraped directly.
## `lastPageNum` (type: `integer`):
How many listing pages to crawl per start URL. For example, set to 3 to crawl pages 1, 2 and 3 of a category. Set to any big number to crawl all available pages.
## `pricePerKG` (type: `integer`):
Enter a number that you want to set as price per kg. It will be applied to products weight if exist.
## `maxRequestsPerCrawl` (type: `integer`):
Maximum number of product detail pages the scraper will process. Set to 0 for unlimited.
## `proxyConfiguration` (type: `object`):
Proxy settings to use during crawling.
## Actor input object example
```json
{
"startUrls": [
{
"url": "https://www.johnlewis.com/browse/electricals/ipad-tablets/view-all-tablets/apple/ipad-air-2025/_/N-5nlvZ1z13zz4Zq1rt"
}
],
"lastPageNum": 1,
"pricePerKG": 5,
"maxRequestsPerCrawl": 200,
"proxyConfiguration": {
"useApifyProxy": true,
"apifyProxyGroups": [
"RESIDENTIAL"
],
"proxyUrls": []
}
}
```
# Actor output Schema
## `results` (type: `string`):
No description
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"startUrls": [
{
"url": "https://www.johnlewis.com/browse/electricals/ipad-tablets/view-all-tablets/apple/ipad-air-2025/_/N-5nlvZ1z13zz4Zq1rt"
}
],
"proxyConfiguration": {
"useApifyProxy": true,
"apifyProxyGroups": [
"RESIDENTIAL"
],
"proxyUrls": []
}
};
// Run the Actor and wait for it to finish
const run = await client.actor("alkausari_mujahid/johnlewis-products-details-crawler").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"startUrls": [{ "url": "https://www.johnlewis.com/browse/electricals/ipad-tablets/view-all-tablets/apple/ipad-air-2025/_/N-5nlvZ1z13zz4Zq1rt" }],
"proxyConfiguration": {
"useApifyProxy": True,
"apifyProxyGroups": ["RESIDENTIAL"],
"proxyUrls": [],
},
}
# Run the Actor and wait for it to finish
run = client.actor("alkausari_mujahid/johnlewis-products-details-crawler").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"startUrls": [
{
"url": "https://www.johnlewis.com/browse/electricals/ipad-tablets/view-all-tablets/apple/ipad-air-2025/_/N-5nlvZ1z13zz4Zq1rt"
}
],
"proxyConfiguration": {
"useApifyProxy": true,
"apifyProxyGroups": [
"RESIDENTIAL"
],
"proxyUrls": []
}
}' |
apify call alkausari_mujahid/johnlewis-products-details-crawler --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=alkausari_mujahid/johnlewis-products-details-crawler",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "Johnlewis Products Details Crawler",
"description": "Extract full product details from John Lewis — prices, specs, images, descriptions, and weight-based cost addition — Automatically calls the John Lewis Listings Crawler to discover all variant URLs, then scrapes each one at speed using CheerioCrawler.",
"version": "0.0",
"x-build-id": "4urZdXs8b2T3kqKTT"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/alkausari_mujahid~johnlewis-products-details-crawler/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-alkausari_mujahid-johnlewis-products-details-crawler",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/alkausari_mujahid~johnlewis-products-details-crawler/runs": {
"post": {
"operationId": "runs-sync-alkausari_mujahid-johnlewis-products-details-crawler",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/alkausari_mujahid~johnlewis-products-details-crawler/run-sync": {
"post": {
"operationId": "run-sync-alkausari_mujahid-johnlewis-products-details-crawler",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"required": [
"startUrls"
],
"properties": {
"startUrls": {
"title": "Start URLs",
"type": "array",
"description": "Provide John Lewis listing pages (brand/search/browse) and/or direct product detail page URLs. Listing pages will be processed via the listings scraper; product URLs will be scraped directly.",
"default": [
{
"url": "https://www.johnlewis.com/browse/electricals/ipad-tablets/view-all-tablets/apple/ipad-air-2025/_/N-5nlvZ1z13zz4Zq1rt"
}
],
"items": {
"type": "object",
"required": [
"url"
],
"properties": {
"url": {
"type": "string",
"title": "URL of a web page",
"format": "uri"
}
}
}
},
"lastPageNum": {
"title": "Last Page Number to Crawl",
"type": "integer",
"description": "How many listing pages to crawl per start URL. For example, set to 3 to crawl pages 1, 2 and 3 of a category. Set to any big number to crawl all available pages.",
"default": 1
},
"pricePerKG": {
"title": "Price per KG",
"type": "integer",
"description": "Enter a number that you want to set as price per kg. It will be applied to products weight if exist.",
"default": 5
},
"maxRequestsPerCrawl": {
"title": "Max Detail Pages to Scrape",
"minimum": 0,
"type": "integer",
"description": "Maximum number of product detail pages the scraper will process. Set to 0 for unlimited.",
"default": 200
},
"proxyConfiguration": {
"title": "Proxy Configuration",
"type": "object",
"description": "Proxy settings to use during crawling.",
"default": {
"useApifyProxy": true,
"apifyProxyGroups": [
"RESIDENTIAL"
],
"proxyUrls": []
}
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```