📌 John Lewis Product & Variant Scraper – Extract Products at Scale

Overview

This scraper extracts comprehensive product data from johnlewis.com, including all color variants, detailed specifications, images, pricing, breadcrumbs, and more.

It collects structured information such as:

• 🏷️ Product title (brand + name)
• 💰 Current & previous prices, promotions
• 🎨 All color variants (automatically generates and crawls variant URLs)
• 🖼️ High-quality product images (main + all alternates)
• 📊 Full specifications table
• 🧭 Breadcrumbs & category path
• 📄 Product description
• ⚖️ Custom pricing logic (weight-based calculated price + final price selection)
• ⏰ Scraped timestamp

The scraper is fast, reliable, and production-ready, ideal for:

---

🎯 Ideal Use Cases

• E-commerce catalog creation & syncing
• Shopify / WooCommerce product imports
• Price monitoring & competitor tracking
• Market research & data analysis
• Data pipelines for BI dashboards
• Automated dropshipping workflows

---

⚙️ Input Configuration

The scraper accepts the following parameters using a JSON input object.

Example input:

{
    "maxRequestsPerCrawl": 1000,
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": ["RESIDENTIAL"],
        "apifyProxyCountry": "US"
    },
    "startUrls": [
        {
            "url": "https://www.johnlewis.com/browse/women/womens-dresses/_/N-flw"
        }
    ]
}

---

## The output JSON will include the following details of a product:
### Example output:  

```json
{
  "type": "product",
  "brand": "Apple",
  "name": "2025 Apple iPad 11\", A16 Processor, iPadOS, Wi-Fi, 128GB",
  "variantAttributes": "Silver",
  "stockStatus": "In Stock",
  "previousPrice": null,
  "currentPrice": "£309.00",
  "promotions": [
    "Price matched"
  ],
  "reviewsCount": "819 Reviews",
  "reviewsStars": "Product review details, this product has received, on average, 4.90 out of 5 star rating.",
  "breadcrumbs": [
    "Homepage",
    "Electricals",
    "iPad & Tablets",
    "View All Tablets",
    "Shop all Apple"
  ],
  "categoryPath": "Homepage > Electricals > iPad & Tablets > View All Tablets > Shop all Apple",
  "mainCategory": "Electricals",
  "subCategory": "iPad & Tablets",
  "images": [
    {
      "mediaContentType": "IMAGE",
      "originalSource": "https://media.johnlewiscontent.com/i/JohnLewis/113465758?fmt=auto&$background-off-white$"
    },
    {
      "mediaContentType": "IMAGE",
      "originalSource": "https://media.johnlewiscontent.com/i/JohnLewis/113465758alt1?fmt=auto&$background-off-white$"
    },
    {
      "mediaContentType": "IMAGE",
      "originalSource": "https://media.johnlewiscontent.com/i/JohnLewis/113465758alt2?fmt=auto&$background-off-white$"
    },
    {
      "mediaContentType": "IMAGE",
      "originalSource": "https://media.johnlewiscontent.com/i/JohnLewis/113465758alt3?fmt=auto&$background-off-white$"
    },
    {
      "mediaContentType": "IMAGE",
      "originalSource": "https://media.johnlewiscontent.com/i/JohnLewis/113465758alt4?fmt=auto&$background-off-white$"
    },
    {
      "mediaContentType": "IMAGE",
      "originalSource": "https://media.johnlewiscontent.com/i/JohnLewis/113465758alt5?fmt=auto&$background-off-white$"
    },
    {
      "mediaContentType": "IMAGE",
      "originalSource": "https://media.johnlewiscontent.com/i/JohnLewis/113465758alt6?fmt=auto&$background-off-white$"
    },
    {
      "mediaContentType": "IMAGE",
      "originalSource": "https://media.johnlewiscontent.com/i/JohnLewis/113465758alt7?fmt=auto&$background-off-white$"
    }
  ],
  "mainImage": "https://media.johnlewiscontent.com/i/JohnLewis/113465758?fmt=auto&$background-off-white$",
  "url": "https://www.johnlewis.com/2025-apple-ipad-11-inch-a16-processor-ipados-wi-fi-128gb/silver/p113465783",
  "productId": "113465783",
  "productCode": "86064111",
  "description": "The 11-inch iPad is now more capable than ever with the superfast A16 chip, Liquid Retina display,[1] advanced cameras, fast Wi-Fi 6, USB-C connector, and four gorgeous colours. iPad delivers a powerful way to create, stay connected and get things done — all for a surprisingly affordable price.\n\n11-Inch Liquid Retina Display\nThe gorgeous Liquid Retina display is an amazing way to watch movies or draw your next masterpiece.[1] True Tone adjusts the display to the colour temperature of the room to make viewing comfortable in any light. \n\nPerformance and storage\nThe superfast A16 chip delivers a boost in performance for your favourite activities. And with all-day battery life, iPad is perfect for playing immersive games and editing photos and videos.[2] Storage starts at 128GB and goes up to 512GB.[3]\n\niPadOS + Apps\niPadOS makes iPad more productive, intuitive and versatile. With iPadOS, run multiple apps at once, use Apple Pencil to write in any text field with Scribble, and edit and share photos. Stage Manager makes multitasking easy with resizable, overlapping apps and external display support. iPad comes with essential apps like Safari, Messages and Keynote, with over a million more apps designed specifically for iPad available on the App Store.\n\nApple Pencil and Magic Keyboard Folio\nWith Apple Pencil (USB-C), iPad transforms into an immersive drawing canvas and the world’s best note-taking device. Magic Keyboard Folio features a versatile two-piece design, with a detachable keyboard and a protective back panel that both attach magnetically to iPad. Apple Pencil (1st generation) is also compatible with iPad.[4] \n\nAdvanced cameras\niPad features a 12MP Center Stage front camera that’s perfect for video calls and selfies. The 12MP Wide back camera with True Tone flash is perfect for document scanning and capturing photos and 4K videos.\n\n\nFast Wi-Fi connectivity\nWi-Fi 6 gives you fast access to your files, uploads and downloads, and lets you seamlessly stream your favourite shows.\n\n\nUnlock and pay with Touch ID\nTouch ID is built into the top button, so you can use your fingerprint to unlock your iPad, sign in to apps and make payments securely with Apple Pay.\n\n\n\nAccessories sold separately and subject to availability. Compatibility varies by generation. Apps are available on the App Store. Title availability is subject to change. Third-party software sold separately.\n\n\n\n [1] The display has rounded corners. When measured diagonally as a rectangle, the 11-inch iPad is 10.86 inches. Actual viewable area is less. \n [2] Battery life varies by use and configuration. See apple.com/uk/batteries for more information. \n [3] Available space is less and varies due to many factors. Storage capacity is subject to change based on software version, settings and iPad model. 1GB = 1 billion bytes; 1TB = 1 trillion bytes. Actual formatted capacity less. \n [4] USB-C to Apple Pencil Adapter required to work with iPad (A16) and iPad (10th generation). Subject to availability.\n\n\n\n\n\n\n\n\nPlease be aware:\nThis iPad includes a USB-C charging cable, but does not include a USB-C power adapter. Therefore, if you do not already own one, you will need to purchase the Apple 20W USB-C Power Adapter in order to charge this iPad.\n\n\nNeed a hand with set up?\nWe can support with setting up user accounts, creating initial passcodes, downloading historic apps, and making sure everything's running as it should be.\n  \nJust bring your new device to one of our stores, and our Partners in Technical Support will take it from there.\n  \nFind out more details and pricing for our tech support services.",
  "descriptionHtml": "<p>The 11-inch iPad is now more capable than ever with the superfast A16 chip, Liquid Retina display,[1] advanced cameras, fast Wi-Fi 6, USB-C connector, and four gorgeous colours. iPad delivers a powerful way to create, stay connected and get things done — all for a surprisingly affordable price.</p>\n\n<p><b>11-Inch Liquid Retina Display</b><br>\nThe gorgeous Liquid Retina display is an amazing way to watch movies or draw your next masterpiece.[1] True Tone adjusts the display to the colour temperature of the room to make viewing comfortable in any light. </p>\n\n<p><b>Performance and storage</b><br>\nThe superfast A16 chip delivers a boost in performance for your favourite activities. And with all-day battery life, iPad is perfect for playing immersive games and editing photos and videos.[2] Storage starts at 128GB and goes up to 512GB.[3]</p>\n\n<p><b>iPadOS + Apps</b><br>\niPadOS makes iPad more productive, intuitive and versatile. With iPadOS, run multiple apps at once, use Apple Pencil to write in any text field with Scribble, and edit and share photos. Stage Manager makes multitasking easy with resizable, overlapping apps and external display support. iPad comes with essential apps like Safari, Messages and Keynote, with over a million more apps designed specifically for iPad available on the App Store.</p>\n\n<p><b>Apple Pencil and Magic Keyboard Folio</b><br>\nWith Apple Pencil (USB-C), iPad transforms into an immersive drawing canvas and the world’s best note-taking device. Magic Keyboard Folio features a versatile two-piece design, with a detachable keyboard and a protective back panel that both attach magnetically to iPad. Apple Pencil (1st generation) is also compatible with iPad.[4]</p> \n\n<p><b>Advanced cameras</b><br>\niPad features a 12MP Center Stage front camera that’s perfect for video calls and selfies. The 12MP Wide back camera with True Tone flash is perfect for document scanning and capturing photos and 4K videos.</p>\n\n\n<p><b>Fast Wi-Fi connectivity</b><br>\nWi-Fi 6 gives you fast access to your files, uploads and downloads, and lets you seamlessly stream your favourite shows.</p>\n\n\n<p><b>Unlock and pay with Touch ID</b><br>\nTouch ID is built into the top button, so you can use your fingerprint to unlock your iPad, sign in to apps and make payments securely with Apple Pay.</p>\n\n<br>\n\n<p><em>Accessories sold separately and subject to availability. Compatibility varies by generation. Apps are available on the App Store. Title availability is subject to change. Third-party software sold separately.</em></p>\n\n<p>\n\n<em> [1] The display has rounded corners. When measured diagonally as a rectangle, the 11-inch iPad is 10.86 inches. Actual viewable area is less. </em><br>\n<em> [2] Battery life varies by use and configuration. See apple.com/uk/batteries for more information. </em><br>\n<em> [3] Available space is less and varies due to many factors. Storage capacity is subject to change based on software version, settings and iPad model. 1GB = 1 billion bytes; 1TB = 1 trillion bytes. Actual formatted capacity less. </em><br>\n<em> [4] USB-C to Apple Pencil Adapter required to work with iPad (A16) and iPad (10th generation). Subject to availability.</em></p>\n\n\n\n\n\n\n<br>\n\n<p><b>Please be aware:</b>\nThis iPad includes a USB-C charging cable, but does not include a USB-C power adapter. Therefore, if you do not already own one, you will need to purchase the <b>Apple 20W USB-C Power Adapter</b> in order to charge this iPad.</p>\n\n<br>\n<p><strong>Need a hand with set up?</strong><br>\nWe can support with setting up user accounts, creating initial passcodes, downloading historic apps, and making sure everything's running as it should be.</p>\n  \n<p>Just bring your new device to one of our stores, and our Partners in Technical Support will take it from there.</p>\n  \n<p>Find out more details and pricing for our tech support services.</p>",
  "specifications": {
    "accessories_included": "USB‑C Charge Cable (1m)",
    "batteries_included": "Yes",
    "battery_life_details": "Up to 10 hours",
    "bluetooth_version": "5.3",
    "brand": "Apple",
    "brightness": "500nits",
    "camera_frontfacing": "Landscape 12MP Center Stage camera, ƒ/2.4 aperture, Smart HDR 4",
    "camera_rearfacing": "12MP Wide camera, ƒ/1.8 aperture",
    "cellular_generation": "Not Cellular",
    "computing_features": "Touch Screen",
    "connectivity_type": "Wi-Fi, Bluetooth",
    "country_of_origin": "China",
    "dimensions": "H0.70 x W17.95 x D24.86cm",
    "display": "LED backlit Multi‑Touch display with IPS technology, True Tone display, Fingerprint-resistant oleophobic coating",
    "graphics_card_series": "Integrated - Apple",
    "graphics_card_type": "Integrated",
    "guarantee": "2 year guarantee included",
    "hdr": "Yes",
    "image_megapixels": "12.00MP",
    "location_services": "via Wi-Fi",
    "manufacturer_part_number_mpn": "MD3Y4KN/A",
    "memory_card_slots": "No",
    "model_name_number": "MD3Y4KN/A",
    "nfc_enabled": "No",
    "number_of_processor_cores": "5",
    "number_of_screens": "1",
    "operating_system": "iPadOS",
    "operating_system_version": "iOS 18",
    "pixels_per_inch_ppi": "264",
    "power_supply": "Rechargeable Battery, USB",
    "processor": "Apple A16",
    "range": "Apple iPad",
    "screen_resolution": "1640 x 2360 pixels",
    "screen_size": "11.0\"",
    "sensors_trackers": "Accelerometer, Gyroscope, Light",
    "sim_card_type": "No SIM Card",
    "touchscreen": "Touch Screen Enabled",
    "weight": "477.00 g",
    "wired_connections": "USB-C 2.0",
    "wireless": "WiFi 6 - 802.11ax Enabled",
    "zoom_digital": "5.00x"
  },
  "weightPrice": "£2.38",
  "finalPrice": 311.38,
  "weight": {
    "unit": "kg",
    "value": 0.477
  },
  "scrapedAt": "2026-01-01T15:44:07.840Z"
}

If you need support let me know!