🛒 Lidl Scraper & Weekly Flyers (8 European Countries)

Extract Verified In-Store Leaflets, Supermarket Groceries & Online Catalog with 100% Shelf Prices

High-speed European retail intelligence engine. Automatically extract verified in-store weekly leaflet promotions, supermarket grocery assortments, and online catalog pricing across 8 European countries into a standardized, unified schema.

---

🎯 Commercial Differentiation (B2B Value)

Most retail scrapers rely on heavy, resource-intensive browser emulation that fails when querying grocery inventories or gets blocked by perimeter firewalls. Other tools pull stale, crowdsourced data with missing prices and restrictive open-database licenses.

This Actor implements a direct, first-party intelligence pipeline that delivers 100% verified shelf prices with zero third-party contamination:

Capability	Generic Scrapers	Lidl Enterprise Intelligence (This Actor)
In-Store Weekly Promotions (Leaflets & Special Deals)	❌ Unavailable	✅ Full Coverage (Promotional dates, discount %, and HD page crops)
Supermarket Assortment (Food, Cleaning, Cosmetics)	❌ Incomplete / Blocked	✅ Comprehensive (Fresh food, private labels, pack sizes, shelf prices)
Price Data Completeness	⚠️ Frequently null	✅ 100% Verified Shelf Prices (Zero null prices)
Anti-Bot Resilience	❌ Repeated HTTP 403/429	✅ Advanced Browser Fingerprint Spoofing & Session Rotation
Infrastructure Overhead	⚠️ Heavy (2GB+ RAM)	✅ Ultra-Lightweight (256MB – 512MB RAM container)
Data Licensing	⚠️ Restrictive Copyleft (ODbL)	✅ 100% First-Party Data (Clean for B2B & Commercial Use)
Visual Validation	❌ Raw text / JSON only	✅ Built-in Visual Table with High-Resolution Product Photos

---

🌍 Market Coverage

Seamlessly toggle between 8 regional retail operations with zero configuration:

Market	Domain	Primary Currency	Assortment Scope
Spain (ES)	lidl.es	EUR (€)	Weekly Leaflets, Supermarket Assortment & Online Shop
Germany (DE)	lidl.de	EUR (€)	Weekly Leaflets, Supermarket Assortment & Online Shop
France (FR)	lidl.fr	EUR (€)	Weekly Leaflets, Supermarket Assortment & Online Shop
United Kingdom (GB)	lidl.co.uk	GBP (£)	Weekly Leaflets & Supermarket Assortment
Italy (IT)	lidl.it	EUR (€)	Weekly Leaflets & Supermarket Assortment
Portugal (PT)	lidl.pt	EUR (€)	Weekly Leaflets & Supermarket Assortment
Poland (PL)	lidl.pl	PLN (zł)	Weekly Leaflets, Supermarket Assortment & Online Shop
Netherlands (NL)	lidl.nl	EUR (€)	Weekly Leaflets, Supermarket Assortment & Online Shop

---

🛡️ Enterprise-Grade Architecture & Resilience

• Advanced Browser Fingerprint Spoofing: Proprietary client TLS layer mimics genuine browser connection characteristics to bypass perimeter traffic inspection without the latency and memory overhead of headless browsers.
• Intelligent Session Management: Automatic proxy session rotation paired with adaptive exponential backoff prevents IP reputation degradation and ensures continuous throughput.
• Strict Data Integrity Contracts: Every extracted data point passes through schema-level sanitization (decimal normalization, HTML stripping, automated discount percentage calculation).
• Enterprise Security & Zero PII: Strict URL allowlisting protects against SSRF vulnerabilities. Extracts strictly public product catalog data; zero customer reviews, personal names, or account data are ever processed (GDPR compliant).

---

⚡ Zero-Config Quick Start

1. Open this Actor on the Apify Console and click Start.
2. By default, it extracts a balanced sample of 100 products and 10 weekly flyer pages with verified prices, discounts, package sizes, and high-resolution photographs.
3. Export to JSON, CSV, or Excel, or inspect the live photo catalog in the Visual Overview tab.

Sample Input

{
  "country": "ES",
  "categories": ["weekly_flyers", "food", "cleaning", "cosmetics"],
  "searchQueries": ["milbona", "cien", "w5"],
  "maxItems": 100
}

---

📊 Standardized Product Schema

All records are normalized into a clean, unified structure regardless of country:

{
  "product_id": "11011151",
  "name": "MILBONA Yogur frutas con trozos",
  "brand": "MILBONA",
  "category_macro": "food",
  "subcategory": "Yogur",
  "price": 2.09,
  "original_price": 2.15,
  "discount_percentage": 2,
  "currency": "EUR",
  "unit_price_text": null,
  "package_size": "400gr",
  "availability": "IN_STORE",
  "country": "ES",
  "source_vector": "web_catalog",
  "product_url": "https://www.lidl.es/p/milbona-yogur-frutas-con-trozos/p11011151",
  "image_url": "https://www.lidl.es/assets/gcp2fd08fa4104447b6b69fa8f182c3002a.png",
  "additional_images": ["https://www.lidl.es/assets/gcp223b11909cbb43b48fd602de8983f526.png"],
  "valid_from": "2026-10-01",
  "valid_to": "2026-10-31",
  "scraped_at": "2026-10-15T09:31:21.567203+00:00"
}

---

💰 Pricing & Predictable Cost Control

This Actor uses Apify's Pay-Per-Event (PPE) pricing model. You only pay for verified data delivered to your dataset:

Event	Price (USD)	Description
actor-start	$0.01	One-time execution start fee covering container initialization.
product-scraped	$0.0025	Per unique validated product ($2.50 per 1,000 items).
flyer-page-processed	$0.005	Per processed promotional flyer page (HD scans, dates, hot-spots).

Exact Budget Ceiling: The Actor halts immediately upon reaching your maxItems or total budget limit. You are never billed beyond your configured maximum.

💡 Example Cost Estimates

Volume	Approximate Run Time	Total Estimated Cost
Sample Test (100 products)	~9 seconds	$0.26
Category Scan (500 products)	~25 seconds	$1.26
Full Assortment (1,000 products)	~45 seconds	$2.51
Multi-Store Basket (5,000 products)	~3 minutes	$12.51

---

🤖 AI Agents & Integrations (MCP Ready)

Easily connect this Actor with modern AI automation and workflow pipelines:

• Model Context Protocol (MCP): Native compatibility with Claude Code, Cursor, Antigravity, and AI Agents for autonomous market research.
• Direct Export: Instant 1-click download in JSON, CSV, or Excel (.xlsx).
• REST API & Webhooks: Trigger downstream syncs whenever new weekly flyer deals are published.
• Google Sheets: Connect directly using Apify's API endpoint with =IMPORTDATA().

---

💼 B2B & Enterprise Use Cases

• Private Label Intelligence: Monitor price elasticity, shrinkflation, and promotional cadences across proprietary store brands (Milbona, Cien, W5, Parkside).
• Weekly Promotional Aggregators: Seamlessly ingest in-store leaflet promotions, promotional dates, and catalog images into coupon or shopping comparison portals.
• Dynamic Pricing Algorithms: Feed clean, real-time retail price points directly into predictive pricing and revenue optimization models.
• Inflation & Market Baskets: Track price movements across identical consumer baskets in 8 European markets over time.

---

❓ Frequently Asked Questions (FAQ)

Can I scrape in-store weekly flyer deals that are not listed in the online shop?

Yes. Unlike basic web scrapers that only read the e-commerce storefront, this Actor directly captures the official in-store promotional leaflets with full promotional validity dates, discount percentages, and high-resolution scanned page crops.

Are prices 100% verified?

Yes. Every single item contains real shelf prices and discount amounts extracted directly from first-party store endpoints. You will never encounter price: null records.

How do I export data to Excel or Google Sheets?

After running the Actor, go to the Storage tab in Apify Console and select Excel (.xlsx) or CSV. You can also copy the API endpoint into Google Sheets.

Can I run this on a recurring schedule?

Yes. In the Apify Console, navigate to the Schedules tab to run this Actor automatically every week when new weekly flyer promotions go live (e.g. Mondays or Thursdays).

---

🤝 Custom Integrations & Dedicated Pipelines

Need custom data pipelines, high-volume dedicated scraping, or automated delivery to Snowflake, BigQuery, AWS S3, or PostgreSQL?

Reach out to our engineering team through the Issues or contact options on our Apify profile.