Lidl Product Scraper

Extract comprehensive product data from Lidl US including prices, images, GTINs, stock status, merchandising details, and full product descriptions. Collect results by keyword search or from any supported Lidl page type for catalog analysis, pricing research, and assortment monitoring.

Features

• Keyword and URL input - Start with any search term, a Lidl search URL, product listing, or a single product detail page
• Rich product coverage - Pricing data (current, regular, promotion, myLidl, upcoming), package dimensions, origin details, allergens, ingredients, and more
• Automatic pagination - Collect products across multiple pages with configurable result and page limits
• Clean output - Only fields with actual values appear in your dataset, making exports easier to work with
• Multiple source modes - Search, category listing, item lookup, and single product detail modes built in

Use Cases

Price Monitoring

Track current, regular, and promotional prices across repeated runs. Compare pricing changes by category or keyword to identify trends and promotions.

Assortment Analysis

Build structured product catalogs with GTINs, package types, merchandising modules, and aisle placement to understand Lidl's product mix.

Product Research

Collect descriptions, ingredients, allergens, and origin data for internal databases, competitive intelligence, or supply chain research.

Catalog Enrichment

Use the structured output to enrich existing product catalogs with Lidl-specific identifiers, pricing, and image assets.

---

Input Parameters

Parameter	Type	Required	Default	Description
url	String	No	https://www.lidl.com/search/products/milk	Lidl search, product list, product detail, or supported API URL
keyword	String	No	milk	Search term used when url is not provided
results_wanted	Integer	No	20	Maximum products to collect
max_pages	Integer	No	5	Pagination safety limit
At least one of url or keyword must be supplied. When both are provided, url takes priority.

---

Output Data

Each dataset item may contain:

Field	Type	Description
rank	Integer	Position in the collected output
source_type	String	Input mode used for this record
source_url	String	Lidl page or API source
source_keyword	String	Search keyword when applicable
source_total_results	Integer	Total results from the source
store_id	String	Store identifier for pricing context
product_id	String	Lidl product identifier
item_id	String	Lidl item identifier
product_url	String	Lidl product detail page
name	String	Product name
gtin	String	Primary GTIN
alternative_gtins	Array	Additional GTIN values
description	String	Short package description
long_description_html	String	Rich product description
long_description_text	String	Plain text description
primary_image_url	String	Main product image
image_urls	Array	All product images
image_ids	Array	Image asset identifiers
category_ids	Array	Category identifiers
brand_names	Array	Brand names
quantity	Number	Package quantity
package_type	String	Package type code
weight	Number	Product weight
height	Number	Product height
length	Number	Product length
volume	Number	Product volume
origin_country	String	Country of origin
origin_region	String	Region of origin
allergens	Array	Listed allergens
ingredients	Array	Listed ingredients
current_price	Number	Current selling price
regular_price	Number	Regular price
promotion_price	Number	Promotional price
mylidl_price	Number	myLidl member price
upcoming_price	Number	Upcoming price
currency	String	Currency code
base_price_text	String	Per-unit price text
base_quantity_value	Number	Base quantity for pricing
base_quantity_unit	String	Base quantity unit
current_price_type	String	Price type (REGULAR, PROMOTION)
current_price_start_date	String	Price effective start date
current_price_end_date	String	Price effective end date
stock_status_code	String	Stock status (INSTOCK, etc.)
merchandising_module	String	In-store module description
aisle	Number	Aisle placement
tags	Array	Product tags
contains_alcohol	Boolean	Alcohol content flag
bogo_info	Object	Buy-one-get-one details

Only fields with actual values are saved in your dataset.

---

Usage Examples

Search by Keyword

Collect products matching a search term:

{
    "keyword": "organic milk",
    "results_wanted": 20,
    "max_pages": 3
}

Start from a Lidl Search URL

Use a specific search results page:

{
    "url": "https://www.lidl.com/search/products/milk",
    "results_wanted": 30,
    "max_pages": 4
}

Collect a Product Listing

Get all products from a category listing:

{
    "url": "https://www.lidl.com/products?category=all&sort=productAtoZ",
    "results_wanted": 50,
    "max_pages": 2
}

Single Product Detail

Get full data for one product:

{
    "url": "https://www.lidl.com/products/1257738"
}

---

Sample Output

{
    "rank": 1,
    "source_type": "searchUrl",
    "source_url": "https://www.lidl.com/search/products/milk",
    "source_keyword": "milk",
    "store_id": "US01053",
    "product_id": "1257738",
    "product_url": "https://www.lidl.com/products/1257738",
    "name": "fairlife fat free ultra-filtered milk",
    "gtin": "0856312002757",
    "description": "52 fl. oz.",
    "current_price": 5.32,
    "regular_price": 5.32,
    "currency": "USD",
    "base_price_text": "10.2 cents per fl.oz.",
    "stock_status_code": "INSTOCK",
    "merchandising_module": "Chiller",
    "aisle": 1,
    "primary_image_url": "https://production-endpoint.azureedge.net/images/74P32D1G6LFJAC1GF0QJ0C0/93d23a72-a4aa-4cd7-854d-dde284a7574e/921405_500x500_500x500.tif.jpg",
    "contains_alcohol": false
}

---

Tips for Best Results

Start Small First

• Use results_wanted: 20 on your first run to verify the output matches your needs
• Increase limits after confirming field coverage

Choose the Right URL Type

• Search URLs for keyword-focused extractions
• Product listing URLs for broad category coverage
• Product detail URLs for individual product lookups

Control Collection Size

• Set max_pages as a safety limit to prevent overly long runs
• Adjust both results_wanted and max_pages to balance runtime against data volume

---

Integrations

Connect your data with:

• Google Sheets - Build trackable pricing and product spreadsheets
• Airtable - Create searchable product databases
• Make - Design automated monitoring workflows
• Zapier - Route product updates into your business tools
• Webhooks - Forward datasets into custom pipelines

Export Formats

• JSON - For APIs, scripts, and programmatic use
• CSV - For spreadsheet-based analysis
• Excel - For business reporting and charting
• XML - For external system integrations

---

Frequently Asked Questions

Can I search by keyword instead of URL?

Yes. Provide the keyword field with any search term, and the actor collects results from Lidl search.

Which Lidl page types are supported?

Search pages, product listing pages with categories, product detail pages, and the equivalent mobile API URLs all work with this actor.

What happens when both url and keyword are set?

The url field takes priority. This lets you use a specific page while still having the keyword available for reference.

How many products can I collect?

There is no hard limit. Use results_wanted to cap the total, and max_pages to limit how many paginated requests the actor makes.

Are empty or null fields included in the output?

No. Fields without real values are omitted before storage so your exports stay clean and focused.

Does this work with Lidl stores outside the US?

The actor defaults to the US store context. Different store IDs may produce results for other Lidl markets, but US is the primary target.

How fast does the actor complete?

With default settings (20 results), it typically finishes in seconds. Larger collections scale predictably with the number of pages needed.

Can I schedule regular runs?

Yes. Apify schedules let you run this actor daily or weekly to track pricing and availability changes over time.

What pricing data is available?

Current price, regular price, promotional price, myLidl member price, upcoming price, and per-unit base price text are all captured when available.

Is product availability included?

Yes. The stock_status_code field indicates whether the product is in stock, and merchandising placement details are included.

---

Support

For issues or feature requests, contact support through the Apify Console.

Resources

• Apify Documentation
• Apify API Reference
• Scheduling Runs

---

Legal Notice

This actor is designed for legitimate data collection purposes. Users are responsible for ensuring compliance with Lidl's terms of service and applicable laws. Use collected data responsibly and respect the website's rate limits.