Lidl Product Scraper

Extract Lidl US product data from search pages, product listings, and individual product pages. Collect current pricing, images, GTINs, stock status, merchandising details, and descriptive product content in a structured dataset for research, catalog monitoring, and assortment analysis.

Features

• Keyword and URL input — Start from a search keyword or a Lidl product URL
• Supports multiple Lidl page types — Search pages, product list pages, category-backed list pages, and product detail pages
• Rich product coverage — Collect pricing, package details, images, stock status, and product descriptions
• Built-in pagination control — Limit collection by result count and maximum pages
• Clean datasets — Omits null-only values so exports stay easier to use

Use Cases

Assortment Monitoring

Track Lidl product availability, prices, and merchandising placement across repeated runs. Build structured snapshots of changing product catalogs over time.

Price Intelligence

Collect current price, regular price, promotional price, and base-price text for downstream analysis. Compare pricing changes across searches, categories, or recurring products.

Product Research

Gather product descriptions, package types, origin details, and GTINs for catalog enrichment. Use the output in internal product databases or analytics workflows.

Competitive Analysis

Monitor how Lidl structures search results and broad product listings. Compare assortment breadth and packaging across categories relevant to your business.

---

Input Parameters

Parameter	Type	Required	Default	Description
url	String	No	https://www.lidl.com/search/products/milk	Lidl search, product list, product detail, or supported mobile API URL
keyword	String	No	milk	Search term used when url is not provided
results_wanted	Integer	No	20	Maximum number of products to collect
max_pages	Integer	No	5	Safety cap on paginated requests
proxyConfiguration	Object	No	{"useApifyProxy": false}	Optional proxy settings

At least one of url or keyword must be provided. If both are supplied, url takes priority.

---

Output Data

Each dataset item can contain:

Field	Type	Description
rank	Integer	Position of the product in the collected output
source_type	String	Input mode used for this record
source_url	String	Lidl page or API URL used as the source
source_keyword	String	Search keyword, when applicable
source_total_results	Integer	Total products reported by the source
store_id	String	Store identifier used for price context
product_id	String	Lidl product ID
item_id	String	Lidl item ID
product_url	String	Lidl product detail URL
name	String	Product name
gtin	String	Primary GTIN
alternative_gtins	Array	Additional GTIN values
description	String	Short package description
long_description_html	String	Rich product description
long_description_text	String	Plain-text product description
primary_image_url	String	Main product image
image_urls	Array	All product image URLs
image_ids	Array	Product image asset IDs
category_ids	Array	Category IDs associated with the product
brand_names	Array	Brand names when present
quantity	Number	Package quantity
package_type	String	Package type code
weight	Number	Weight when available
height	Number	Height when available
length	Number	Length when available
volume	Number	Volume when available
origin_country	String	Country of origin
origin_region	String	Region of origin
allergens	Array	Listed allergens
ingredients	Array	Listed ingredients
current_price	Number	Current price
regular_price	Number	Regular price
promotion_price	Number	Promotional price
mylidl_price	Number	myLidl member price
upcoming_price	Number	Upcoming price
currency	String	Currency code
base_price_text	String	Base-price text such as per-ounce cost
base_quantity_value	Number	Base quantity value
base_quantity_unit	String	Base quantity unit
current_price_type	String	Price type label
current_price_start_date	String	Price start date
current_price_end_date	String	Price end date
stock_status_code	String	Stock status
merchandising_module	String	Merchandising module description
aisle	Number	Aisle number
tags	Array	Product tags
contains_alcohol	Boolean	Whether the product contains alcohol
bogo_info	Object	Buy-one-get-one information when present

Only fields with real values are saved.

---

Usage Examples

Search by Keyword

{
    "keyword": "milk",
    "results_wanted": 20,
    "max_pages": 3
}

Start from a Lidl Search URL

{
    "url": "https://www.lidl.com/search/products/milk",
    "results_wanted": 30,
    "max_pages": 4
}

Collect from a Lidl Product Listing

{
    "url": "https://www.lidl.com/products?category=all&sort=productAtoZ",
    "results_wanted": 25,
    "max_pages": 2
}

Collect a Single Product Page

{
    "url": "https://www.lidl.com/products/1257738"
}

Sample Output

{
    "rank": 1,
    "source_type": "searchUrl",
    "source_url": "https://www.lidl.com/search/products/milk",
    "source_keyword": "milk",
    "source_total_results": 288,
    "store_id": "US01053",
    "product_id": "1257738",
    "item_id": "245421",
    "product_url": "https://www.lidl.com/products/1257738",
    "name": "fairlife fat free ultra-filtered milk",
    "gtin": "0856312002757",
    "alternative_gtins": [
        "0856312002757",
        "856312002757"
    ],
    "description": "52 fl. oz.",
    "long_description_text": "Next time you pour a nice glass of cold milk, be confident about the quality of what you’re drinking. With 50% less sugar than regular milk, 13 grams of high quality protein per serving and no artificial growth hormones, fairlife fat free ultra-filtered milk is the choice to make when you want the best.",
    "primary_image_url": "https://production-endpoint.azureedge.net/images/74P32D1G6LFJAC1GF0QJ0C0/224fc2e6-a089-46c0-96fc-6db99c0c928d/921405_500x500_500x500.tif.jpg",
    "category_ids": [
        "OCI1000079",
        "OCI2000110",
        "OCI1000136",
        "OC006335"
    ],
    "package_type": "FL",
    "current_price": 5.32,
    "regular_price": 5.32,
    "currency": "USD",
    "base_price_text": "10.2 ¢ per fl.oz.",
    "stock_status_code": "INSTOCK",
    "merchandising_module": "Chiller",
    "aisle": 1,
    "contains_alcohol": false
}

---

Tips for Best Results

Start Small First

• Begin with results_wanted: 20 to validate the exact page type you want
• Increase limits only after confirming the returned fields match your needs

Use the Most Specific URL You Have

• Use search URLs for focused keyword pulls
• Use product listing URLs when you want broad category-style coverage
• Use product detail URLs when you only need one product

Keep Pagination Intentional

• Use max_pages as a safety limit for broad product listings
• Combine results_wanted and max_pages to control runtime and export size

Integrations

Connect your data with:

• Google Sheets — Build sortable product and pricing trackers
• Airtable — Maintain searchable product catalogs
• Make — Trigger monitoring workflows and alerts
• Zapier — Send product updates into other business systems
• Webhooks — Forward fresh runs into your own pipeline

Export Formats

• JSON — For APIs, scripts, and structured processing
• CSV — For spreadsheet analysis
• Excel — For business reporting
• XML — For external system integrations

---

Frequently Asked Questions

Can I use a keyword instead of a URL?

Yes. If you provide keyword, the actor collects products from Lidl search results.

Which Lidl URLs are supported?

Search URLs, product listing URLs, product detail URLs, and equivalent supported mobile API URLs.

What happens if I provide both url and keyword?

url takes priority so the actor follows the exact page you supplied.

Does the actor save empty or null fields?

No. Fields without useful values are removed before the item is stored.

Can I control how many products are returned?

Yes. Use results_wanted and max_pages together to control collection size.

Support

For issues or feature requests, contact support through the Apify Console.

Resources

• Apify Documentation
• Apify API Reference
• Apify Schedules

---

Legal Notice

This actor is designed for legitimate data collection purposes. Users are responsible for ensuring compliance with Lidl terms, applicable laws, and their own downstream data usage requirements.