📌 John Lewis Product & Variant Scraper – Extract Products at Scale

Overview

This scraper extracts comprehensive product data from johnlewis.com, including all color variants, detailed specifications, images, pricing, breadcrumbs, and more.

It collects structured information such as:

• 🏷️ Product title (brand + name)
• 💰 Current & previous prices, promotions
• 🎨 All color variants (automatically generates and crawls variant URLs)
• 🖼️ High-quality product images (main + all alternates)
• 📊 Full specifications table
• 🧭 Breadcrumbs & category path
• 📄 Product description
• ⚖️ Custom pricing logic (weight-based calculated price + final price selection)
• ⏰ Scraped timestamp

The scraper is fast, reliable, and production-ready, ideal for:

• Price monitoring & competitor analysis
• Building product catalogs
• Market research & trend analysis
• Integrating data into e-commerce platforms, dashboards, or databases

---

⚙️ Input Configuration

The scraper accepts the following parameters using a JSON input object.

Example input:

{
    "maxRequestsPerCrawl": 1000,
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": ["RESIDENTIAL"],
        "apifyProxyCountry": "US"
    },
    "startUrls": [
        {
            "url": "https://www.johnlewis.com/browse/women/womens-dresses/_/N-flw"
        }
    ]
}

## The output JSON will include the following details of a product:
### Example output:  

```json
{
  "type": "product",
  "brand": "Apple",
  "name": "2025 Apple iPad Air 13\", M3 Processor, iPadOS, Wi-Fi & Cellular, 128GB",
  "reviewsCount": "3 Reviews",
  "reviewsStars": "Product review details, this product has received, on average, 5.00 out of 5 star rating.",
  "previousPrice": null,
  "currentPrice": "£849.00",
  "variantAttributes": "Blue",
  "stockStatus": "In Stock",
  "promotions": null,
  "breadcrumbs": [
    "Homepage",
    "Electricals",
    "iPad & Tablets",
    "View All Tablets",
    "Shop all Apple"
  ],
  "categoryPath": "Homepage > Electricals > iPad & Tablets > View All Tablets > Shop all Apple",
  "mainCategory": "Electricals",
  "subCategory": "iPad & Tablets",
  "images": [
    "https://media.johnlewiscontent.com/i/JohnLewis/113464999?fmt=auto&$background-off-white$",
    "https://media.johnlewiscontent.com/i/JohnLewis/113464999alt1?fmt=auto&$background-off-white$",
    "https://media.johnlewiscontent.com/i/JohnLewis/113464999alt2?fmt=auto&$background-off-white$",
    "https://media.johnlewiscontent.com/i/JohnLewis/113464999alt3?fmt=auto&$background-off-white$",
    "https://media.johnlewiscontent.com/i/JohnLewis/113464999alt4?fmt=auto&$background-off-white$",
    "https://media.johnlewiscontent.com/i/JohnLewis/113464999alt5?fmt=auto&$background-off-white$",
    "https://media.johnlewiscontent.com/i/JohnLewis/113464999alt6?fmt=auto&$background-off-white$",
    "https://media.johnlewiscontent.com/i/JohnLewis/113464999alt7?fmt=auto&$background-off-white$",
    "https://media.johnlewiscontent.com/i/JohnLewis/113464999alt8?fmt=auto&$background-off-white$",
    "https://media.johnlewiscontent.com/i/JohnLewis/113464999alt9?fmt=auto&$background-off-white$",
    "https://media.johnlewiscontent.com/v/JohnLewis/113464702vid1?fmt=auto&$background-off-white$"
  ],
  "mainImage": "https://media.johnlewiscontent.com/i/JohnLewis/113464999?fmt=auto&$background-off-white$",
  "url": "https://www.johnlewis.com/2025-apple-ipad-air-13-inch-m3-processor-ipados-wi-fi-cellular-128gb/p113465265",
  "productId": "113465265",
  "descriptionCode": "Product code: 86047133",
  "description": "The 13-inch iPad Air.[1] Built for Apple Intelligence.[2] Supercharged by the lightning-fast Apple M3 chip. It features a stunning Liquid Retina display, a 12MP Center Stage front camera for even better video calls, and blazing-fast Wi-Fi 6E [3] and 5G capability.[4] And it works with Apple Pencil Pro, Magic Keyboard and Smart Folio, so you can multitask, study, work, play and create from anywhere.\n\n13-inch Liquid Retina Display\nThe gorgeous Liquid Retina display features advanced technologies like P3 wide colour, True Tone and ultra-low reflectivity, which make everything look stunning.[1]\n\nBuilt for Apple Intelligence\nApple Intelligence is the personal intelligence system that helps you write, express yourself and get things done effortlessly. With groundbreaking privacy protections integrated into the core of iPad, it gives you peace of mind that no one else can access your data - not even Apple.[2] (Apple Intelligence available now)\n\nPerformance and storage\nM3 is a powerful chip built for Apple Intelligence and brings amazing performance for advanced creative and product tasks, graphics-intensive games and smooth multitasking. And with all-day battery life, you can keep working and playing wherever you go.[5] Choose up to 1TB of storage depending on the room you need for apps, music, movies and more.[6]\n\niPadOS + apps\niPadOS makes iPad more productive, intuitive and versatile. With iPadOS, run multiple apps at once, use Apple Pencil to write in any text field with Scribble, and edit and share photos. Stage Manager makes multitasking easy with resizable, overlapping apps and external display support. iPad Air comes with essential apps like Safari, Messages and Keynote, with over a million more apps designed specifically for iPad available on the App Store.\n\nApple Pencil and Magic Keyboard\nApple Pencil Pro transforms iPad Air into an immersive drawing canvas and the world’s best note-taking device. Apple Pencil (USB-C) is also compatible with iPad Air. Magic Keyboard for iPad Air delivers an amazing typing experience with a built-in trackpad and 14-key function row, and it doubles as a protective cover on the go. Accessories sold separately.\n\nAdvanced cameras\niPad Air features a 12MP Center Stage front camera that’s perfect for video calls and selfies. The 12MP Wide back camera with True Tone flash is perfect for document scanning and capturing photos and 4K videos.\n\nConnectivity\nWi-Fi 6E gives you fast wireless connections for quick transfers of photos, documents and large video files.[3] And with 5G cellular capabilities, you have the option to stay connected when you’re away from Wi-Fi and add a flexible data plan whenever you need it.[4] Connect to external displays and more with the USB-C connector.\n\nUnlock and pay with Touch ID\nTouch ID is built into the top button, so you can use your fingerprint to unlock your iPad Air, sign in to apps and make payments securely with Apple Pay.\n\n\n\nAccessories sold separately and subject to availability. Compatibility varies by generation. Apps are available on the App Store. Title availability is subject to change. Third-party software sold separately. \n\n\n\n[1] The display has rounded corners. When measured diagonally as a rectangle, the 13-inch iPad Air is 12.9 inches. Actual viewable area is less. \n\n[2] Apple Intelligence is available in beta on iPad mini (A17 Pro) and iPad models with M1 and later, with Siri and device language set to English (Australia, Canada, Ireland, New Zealand, South Africa, UK or US), as an iPadOS 18 update. Additional features and languages will be available in early April 2025. Some features may not be available in all regions or languages. \n\n\n[3] Wi-Fi 6E available in countries and regions where supported. \n\n[4] Data plan required. 5G is available in selected markets and through selected network providers. Speeds vary based on site conditions and network. For details on 5G support, contact your provider and see apple.com/uk/ipad/cellular. \n\n\n[5] Battery life varies by use and configuration. See apple.com/uk/batteries for more information. \n\n[6] Available space is less and varies due to many factors. Storage capacity is subject to change based on software version, settings and iPad model. 1GB = 1 billion bytes; 1TB = 1 trillion bytes. Actual formatted capacity less. \n\n\n\n\nPlease be aware:\nThis iPad includes a USB-C charging cable, but does not include a USB-C power adapter. Therefore, if you do not already own one, you will need to purchase the Apple 20W USB-C Power Adapter in order to charge this iPad.\n\n\nNeed a hand with set up?\nWe can support with setting up user accounts, creating initial passcodes, downloading historic apps, and making sure everything's running as it should be.\n  \nJust bring your new device to one of our stores, and our Partners in Technical Support will take it from there.\n  \nFind out more details and pricing for our tech support services.",
  "specifications": {
    "accessories_included": "USB‑C Charge Cable (1m)",
    "batteries_included": "Yes",
    "battery_life_details": "Up to 9 hours (web use)",
    "bluetooth_version": "5.3",
    "brand": "Apple",
    "brightness": "600nits",
    "camera_frontfacing": "Landscape 12MP Center Stage camera, ƒ/2.0 aperture, Smart HDR 4",
    "camera_rearfacing": "12MP Wide camera, ƒ/1.8 aperture",
    "cellular_generation": "5G",
    "computing_features": "Touch Screen",
    "connectivity_type": "Cellular, Wi-Fi, Bluetooth",
    "country_of_origin": "China",
    "dimensions": "H0.61 x W17.85 x D28.06cm",
    "display": "LED backlit Multi‑Touch display with IPS technology, Anti-reflective coating, Fingerprint-resistant oleophobic coating",
    "graphics_card_series": "Integrated - Apple",
    "graphics_card_type": "Integrated",
    "guarantee": "2 year guarantee included",
    "hdr": "Yes",
    "location_services": "via Wi-Fi, via GPS",
    "manufacturer_part_number_mpn": "MCJ24KN/A",
    "memory_ram": "8GB",
    "memory_card_slots": "No",
    "model_name_number": "MCJ24KN/A",
    "nfc_enabled": "No",
    "number_of_processor_cores": "8",
    "number_of_screens": "1",
    "operating_system": "iPadOS",
    "operating_system_version": "iOS 18",
    "pixels_per_inch_ppi": "264",
    "power_supply": "Rechargeable Battery, USB",
    "processor": "Apple M3 Chip",
    "processor_type": "Apple M3",
    "range": "Apple iPad Air",
    "screen_resolution": "2048 x 2732 pixels",
    "screen_size": "13.0\"",
    "sensors_trackers": "Accelerometer, Gyroscope, Light",
    "sim_card_type": "eSIM",
    "touchscreen": "Touch Screen Enabled",
    "weight": "617.00g",
    "wired_connections": "USB-C 2.0",
    "wireless": "WiFi 6E - 802.11ax Enabled",
    "zoom_digital": "5.00x"
  },
  "weightPrice": "£3.08",
  "finalPrice": 852.08,
  "scrapedAt": "2025-12-25T08:23:36.125Z"
}

If you need support let me know!