Johnlewis Products Listings Crawler

Collect every product URL from any John Lewis category, brand, or search page — across all pagination, including every colour variant. Paste a listing URL, get a clean dataset of product URLs ready to plug straight into the John Lewis Product Details Crawler — or use however you like.

Built and maintained by Alkausari M.

---

✦ Highlights

• ⚡ Fast pagination — walks every page in the listing automatically
• 🎨 Colour variants included — captures each variant URL, not just the parent product
• 📊 Any format — JSON, CSV, Excel, XML, or live via the Apify API
• 💰 Predictable cost — pay per URL collected
• 🖱 Zero code — paste a URL, click Start
• 🛡 Anti-blocking built-in — residential proxies, rotation, smart retries

---

⚙ How it works

1. Paste a URL — category, brand, or search results page from John Lewis.
2. Click Start — pagination, proxies, and retries handled for you.
3. Download or pipe — JSON, CSV, Excel, or pull straight from the API.

// Example input
{
    "startUrls": [
        { "url": "https://www.johnlewis.com/brand/flexlux/_/N-pze9" }
    ],
    "lastPageNum": 0,
    "maxRequestsPerCrawl": 50,
    "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}

• lastPageNum — max page number per URL. Enter a big number to crawl all pages.
• maxRequestsPerCrawl — hard cap on total pages across all URLs. 0 = unlimited.

This Actor is the first stage of a two-Actor pipeline. Feed the URL dataset into the companion John Lewis Product Details Crawler to extract full product data — prices, sizes, images, specs, stock.

---

📦 What you get back

Each item in the dataset is one product variant URL: You can group variants by product id like p113951807

[
  { "url": "https://www.johnlewis.com/flexlux-volden-relax-recliner-chair/copparo-sandy-biege/p113091750" },
  { "url": "https://www.johnlewis.com/flexlux-volden-relax-recliner-chair/copparo-deep-forest-grey/p113091750" },
  { "url": "https://www.johnlewis.com/flexlux-volden-relax-recliner-chair/dakota-modern-cognac-brown/p113091750" }
]

Download as JSON, CSV, Excel, or HTML from the run's Storage tab — or pull live via the Apify API.

---

💡 Use cases

• Catalogue building — retailers and resellers can build a complete product feed from any John Lewis brand or category.
• Price & stock tracking — collect URLs daily and feed them into the details crawler to track changes over time.
• Market research — see what's available in a specific category, brand, or price range.
• Inventory sync — dropshippers can keep listings in sync by re-running whenever stock or variants change.
• Full-data pipelines — pair with the Product Details Crawler for complete product records — titles, prices, size options, images, descriptions, availability.

---

📮 Support

Bugs, feature requests, or custom scraping work — open an issue on Apify or email alkausarimujahid@gmail.com.

---