# Lidl Product Scraper (`datascrapers/lidl-scraper`) Actor

Lidl product dataset from European shop search queries or category and product URLs, including prices, ratings, and optional product-page details.

- **URL**: https://apify.com/datascrapers/lidl-scraper.md
- **Developed by:** [Farhan Ali](https://apify.com/datascrapers) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.90 / 1,000 products

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

**Lidl Product Scraper** creates a structured dataset of Lidl products collected from Lidl country shops such as [lidl.de](https://www.lidl.de). Each dataset item represents one product and can include identifiers, title and brand, price and currency, category, rating, availability, and optional product-page details (description, EANs, variants, media, and delivery information). Query the source with search terms (`searchQueries`) or Lidl category, section, search, or product URLs (`startUrls`), scoped by `countryCode`. Control the result limit with `maxItems`, and retrieve records through the Apify Dataset API or export them as JSON, CSV, Excel, XML, or another supported format.

### Dataset at a glance

| Property | Value |
|---|---|
| Source | Lidl country shops (`lidl.de`, `lidl.at`, `lidl.ch`, `lidl.fr`, `lidl.es`, `lidl.it`, `lidl.nl`, `lidl.be`, `lidl.pt`, `lidl.pl`, `lidl.cz`, `lidl.sk`, `lidl.hu`, `lidl.ro`, `lidl.ie`, `lidl.co.uk`) |
| Record unit | One Lidl product |
| Input methods | Search terms (`searchQueries`) and/or Lidl category, section, search, or product URLs (`startUrls`) |
| Main identifiers | `productId`, product URL (`url`) |
| Delivery | Apify Dataset and API |
| Export formats | JSON, CSV, Excel, XML, HTML (Apify dataset exports) |
| Update model | Fresh records per Actor run |
| Pricing | $1 / 1,000 products; +$2 / 1,000 product details; Actor start $0.00005 |

### Coverage and available records

The Actor collects public Lidl catalog products from one or both entry points. When both are set, the seeds are combined.

- **Search-based**: `searchQueries` (for example `heissluftfritteuse`) resolve against the shop selected by `countryCode` (default `DE`).
- **URL-based**: `startUrls` accepts Lidl category, section, search, or product pages. The host on each URL selects the country shop and overrides `countryCode` for that seed.

Record types and limits:

- **Listing-level records** are always collected: identifiers, title, brand, price, currency, image, category, rating, availability, country, and source metadata.
- **Product-page fields** are conditional: description, EANs, variants, media, and delivery information are returned only when `fetchProductDetails` is enabled and `detailsFetched` is `true`.
- **Result cap**: `maxItems` limits the number of products collected (`0` means unlimited, the default). Pagination continues until the source returns no further products or the cap is reached.

Supported `countryCode` values: `DE`, `AT`, `CH`, `FR`, `ES`, `IT`, `NL`, `BE`, `PT`, `PL`, `CZ`, `SK`, `HU`, `RO`, `IE`, `GB`.

Known exclusions: content Lidl only shows behind login is not collected; each run captures catalog state at run time (no historical snapshots); pages outside the supported Lidl country shops are not collected. Assortment and prices differ by country shop.

### Data dictionary

Field names below match dataset record JSON properties exactly. Fields marked *conditional* appear only when `fetchProductDetails` is enabled. Values come from Lidl for the selected shop; the Actor does not calculate prices or ratings. `scrapedAt` and `detailsFetched` are set by the Actor.

#### Product listing fields

| Field | Type | Nullable | Description | Example |
|---|---|---:|---|---|
| `productId` | string | No | Lidl product identifier; recommended deduplication key | `276353` |
| `erpNumber` | string | Yes | Article / ERP number | `276353` |
| `title` | string | Yes | Product title | `SilverCrest® Heißluftfritteuse` |
| `fullTitle` | string | Yes | Full listing title | `SilverCrest® Heißluftfritteuse XXL 6,5 l` |
| `brand` | string | Yes | Brand name | `SilverCrest` |
| `price` | number | Yes | Numeric price in `currency` | `49.99` |
| `currency` | string | Yes | ISO currency code | `EUR` |
| `currencySymbol` | string | Yes | Currency symbol as shown by Lidl | `€` |
| `url` | string | Yes | Product page URL; secondary deduplication key | `https://www.lidl.de/p/silvercrest-heissluftfritteuse-xxl-6-5-l/p276353` |
| `image` | string | Yes | Primary product image URL | `https://www.lidl.de/media/product/...` |
| `category` | string | Yes | Category label | `Küche & Haushalt` |
| `ratingAverage` | number | Yes | Average rating | `4.5` |
| `ratingCount` | number | Yes | Number of ratings | `1287` |
| `availability` | string | Yes | Availability text as shown by Lidl | `Online verfügbar` |
| `onlineAvailable` | boolean | Yes | Whether the product is marked available online | `true` |
| `countryCode` | string | No | Lidl shop country code | `DE` |
| `sourceQuery` | string | No | Search term or start URL that produced the record | `heissluftfritteuse` |
| `scrapedAt` | string | No | UTC timestamp when the record was written (ISO 8601) | `2026-07-26T12:00:00.000000+00:00` |
| `detailsFetched` | boolean | No | Whether product-page fields were attached | `false` |

#### Product detail fields (conditional — `fetchProductDetails`)

These fields appear when `fetchProductDetails` is `true` and `detailsFetched` is `true`. Nested shapes follow what Lidl renders; they are omitted when the source does not supply them.

| Field | Type | Nullable | Description | Example |
|---|---|---:|---|---|
| `description` | string | Yes | Product description from the product page | Product-page description text |
| `eans` | string\[] | Yes | European Article Numbers when listed | EAN strings from the product page |
| `variants` | object\[] | Yes | Variant records: `{erpNumber, title, price, currency}` | See source when variants exist |
| `deliveryInfo` | object | Yes | Delivery information when Lidl renders it | Object as returned by the source |

Additional product-page media is included when present. The Actor does not invent missing EANs, variants, media, or delivery values.

### Example dataset record

Listing-level record for the Germany shop search `heissluftfritteuse` (`fetchProductDetails` off). Product-page fields are absent because details were not requested.

```json
{
  "productId": "276353",
  "erpNumber": "276353",
  "title": "SilverCrest® Heißluftfritteuse",
  "fullTitle": "SilverCrest® Heißluftfritteuse XXL 6,5 l",
  "brand": "SilverCrest",
  "price": 49.99,
  "currency": "EUR",
  "currencySymbol": "€",
  "url": "https://www.lidl.de/p/silvercrest-heissluftfritteuse-xxl-6-5-l/p276353",
  "category": "Küche & Haushalt",
  "ratingAverage": 4.5,
  "ratingCount": 1287,
  "availability": "Online verfügbar",
  "onlineAvailable": true,
  "countryCode": "DE",
  "sourceQuery": "heissluftfritteuse",
  "scrapedAt": "2026-07-26T12:00:00.000000+00:00",
  "detailsFetched": false
}
```

The record above was produced with this input:

```json
{
  "searchQueries": ["heissluftfritteuse"],
  "fetchProductDetails": false,
  "countryCode": "DE",
  "maxItems": 25
}
```

### Query and input reference

| Input | Type | Required | Default | Accepted values | Description |
|---|---|---:|---|---|---|
| `searchQueries` | array (string) | No\* | — (prefill `heissluftfritteuse`) | Product search terms | Search entry point. Uses the `countryCode` shop |
| `startUrls` | array (requestListSources) | No\* | — | Lidl category, section, search, or product URLs | URL entry point. Country is taken from the host |
| `fetchProductDetails` | boolean | No | `false` | `true` / `false` | Attach product-page fields (charged as product details) |
| `countryCode` | string | No | `DE` | `DE`, `AT`, `CH`, `FR`, `ES`, `IT`, `NL`, `BE`, `PT`, `PL`, `CZ`, `SK`, `HU`, `RO`, `IE`, `GB` | Default shop for `searchQueries`. Ignored when a start URL sets the host |
| `maxItems` | integer | No | `0` | `0` or any positive integer | Maximum products to collect; `0` = unlimited (console prefill `25`) |
| `proxyConfiguration` | object | No | Apify proxy, `RESIDENTIAL` group | Apify proxy groups or custom proxies | Residential proxies are recommended |

\* Provide `searchQueries` and/or `startUrls` — at least one source is required.

Minimal request:

```json
{ "searchQueries": ["heissluftfritteuse"], "countryCode": "DE" }
```

Advanced request (combined seeds with product-page fields):

```json
{
  "searchQueries": ["heissluftfritteuse"],
  "startUrls": [{ "url": "https://www.lidl.de/h/garten-balkon/h10067558" }],
  "fetchProductDetails": true,
  "countryCode": "DE",
  "maxItems": 25,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}
```

### Retrieve the data through the API

Records are delivered through the Apify Actor and Dataset APIs — not an official Lidl API.

1. Start the Actor with a JSON input (console or API).
2. Wait for the run to finish, or use a synchronous endpoint if you want the response inline.
3. Retrieve items from the run's default dataset.
4. Paginate or export the dataset.

Python example:

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR-APIFY-TOKEN")

run_input = {
    "searchQueries": ["heissluftfritteuse"],
    "countryCode": "DE",
    "maxItems": 10,
    "fetchProductDetails": True,
}

run = client.actor("datascrapers/lidl-scraper").call(run_input=run_input)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["productId"], item["title"], item["price"])
```

Apify generates ready-to-run Python, JavaScript, and cURL examples on the Actor's [API tab](https://apify.com/datascrapers/lidl-scraper/api). Do not put a real API token in shared code or URLs.

### Data quality and record handling

- **Conditional fields**: product-page fields are present only when `fetchProductDetails` is enabled and `detailsFetched` is `true`. Listing-only runs return a leaner record.
- **Source changes**: Lidl page structure and values can change; unreadable fields are omitted or returned as null rather than fabricated.
- **Deduplication**: within a run, products are de-duplicated by shop and product identifier. Across runs, records are appended. Use `productId` as the stable key, or `url` as a secondary key, and filter repeated runs against previously stored IDs.
- **Failures**: one failed product or seed does not stop the run. The run stops if a spending limit is reached.
- **Normalization**: `price` and `ratingAverage` are numeric; `scrapedAt` is an ISO 8601 UTC timestamp written by the Actor. Currency, availability text, and ratings come from Lidl.

### Export and pipeline examples

| Destination | Recommended method | Typical use |
|---|---|---|
| PostgreSQL / Supabase | Dataset API poll or webhook consumer | Store catalog prices and article numbers |
| Google Sheets | Apify Google Sheets integration | Share shortlists and price checks |
| PIM / catalog systems | Dataset API or completion webhook | Product-feed refresh |
| S3 / cloud storage | Scheduled export via Apify scheduler + integration | Archival of market snapshots |

### Pricing and cost examples

The Actor uses pay-per-event pricing. Each run also charges a one-time Actor start. Residential proxies are recommended. Proxy and compute are billed by your Apify plan.

| Event | Trigger | Rate |
|---|---|---|
| Actor start | Once per run | $0.00005 |
| `dataset-item` (Product) | Each product written to the dataset | $0.001 / product ($1 / 1,000) |
| `product-details` | Product-page fields collected (`fetchProductDetails`) | $0.002 / product ($2 / 1,000) |

Example costs (Actor start negligible; plan-tier discounts not applied):

| Records | Configuration | Estimated base cost |
|---:|---|---:|
| 1,000 | Listing only | $1.00 |
| 10,000 | Listing only | $10.00 |
| 1,000 | Listing + product details | $3.00 |
| 10,000 | Listing + product details | $30.00 |

Estimates depend on the verified pricing model and the options selected for the run.

### Limitations and responsible data use

- The Actor collects publicly accessible data from supported Lidl country shops only.
- Field availability depends on what Lidl renders at run time; some values can be null or missing, and site changes can alter fields.
- The Actor does not provide historical snapshots unless you store them yourself. Assortment and prices differ by country shop.
- Large runs should use residential proxies; without them, coverage may degrade.
- You are responsible for compliance with Lidl's terms of service, applicable privacy law, and any contractual obligations before using the data.

### Dataset questions

#### What does one dataset item represent?

One Lidl product. Variant data, when requested, is nested on that product (`variants`), so a product with several size or option variants still produces a single dataset item.

#### Which field should I use as a unique identifier?

`productId` is the recommended deduplication key. `url` is a reasonable secondary key. The same product can exist in more than one country shop; combine `productId` with `countryCode` when you merge multi-country runs.

#### Are fields nullable or conditional?

Yes. Product-page fields (description, EANs, variants, media, delivery information) exist only when `fetchProductDetails` is enabled and `detailsFetched` is `true`. Within listing records, fields that Lidl does not render for a product are omitted or null.

#### How do I select a Lidl country shop?

Set `countryCode` for `searchQueries` (default `DE`). For `startUrls`, the hostname selects the shop (`lidl.de`, `lidl.fr`, `lidl.co.uk`, and the other supported hosts) and overrides `countryCode` for that seed.

#### Can I retrieve the records as CSV or JSON?

Yes. The dataset can be exported as JSON, CSV, Excel, XML, or HTML from the Apify Console, and queried through the Dataset API.

#### Does the Actor return historical data?

No. Each run captures the state of the catalog at run time. To track price or availability changes, schedule repeated runs and store the outputs yourself.

#### What counts as a billable result?

Three pay-per-event charges apply: Actor start once per run ($0.00005), a `dataset-item` charge for every product record ($0.001, or $1 per 1,000), and a `product-details` charge for each product whose product-page fields are collected ($0.002, or $2 per 1,000). A listing-only run incurs Actor start plus `dataset-item`.

### Related datasets from Data Scrapers

- **[IKEA Product Scraper](https://apify.com/datascrapers/ikea-scraper)** — European home and household products that can be compared with Lidl assortment and prices.
- **[Target.com Product Scraper](https://apify.com/datascrapers/target-scraper)** — US retail catalog records for the same price-monitoring pipelines.
- **[Amazon Search Scraper](https://apify.com/datascrapers/amazon-scraper)** — Marketplace search products for cross-retailer comparison with Lidl items.
- **[Costco Product Scraper](https://apify.com/datascrapers/costco-scraper)** — Warehouse-club grocery and household products in overlapping categories.
- **[Instacart Scraper](https://apify.com/datascrapers/instacart-scraper)** — Grocery catalog products for broader retail and availability analysis.

### Data Scrapers support

Need an additional field, record type, or export workflow? Contact Data Scrapers at stardustspotlight@gmail.com. Include a sample source URL, required fields, expected record volume, and preferred delivery format.

# Actor input Schema

## `searchQueries` (type: `array`):

Product search terms (e.g. "heissluftfritteuse", "garden lights"). Uses the selected country shop. Provide searchQueries and/or startUrls.

## `startUrls` (type: `array`):

Lidl category, section, search, or product URLs (e.g. https://www.lidl.de/h/garten-balkon/h10067558 or https://www.lidl.de/q/search?q=philips). Country/locale are inferred from the host.

## `fetchProductDetails` (type: `boolean`):

When enabled, opens each product page to extract full details (description, summary, EANs, variants, media, delivery info). When disabled, returns listing/card fields only (faster).

## `countryCode` (type: `string`):

Lidl country shop used for searchQueries when no start URL sets the host. Ignored for startUrls (host wins).

## `maxItems` (type: `integer`):

Maximum number of products to scrape (0 = unlimited)

## `proxyConfiguration` (type: `object`):

Proxy settings for anti-bot protection

## Actor input object example

```json
{
  "searchQueries": [
    "heissluftfritteuse"
  ],
  "startUrls": [
    {
      "url": "https://www.lidl.de/h/garten-balkon/h10067558"
    }
  ],
  "fetchProductDetails": false,
  "countryCode": "DE",
  "maxItems": 25,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset containing scraped Lidl products

## `runStats` (type: `string`):

Aggregate scrape statistics for this run

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "heissluftfritteuse"
    ],
    "startUrls": [
        {
            "url": "https://www.lidl.de/h/garten-balkon/h10067558"
        }
    ],
    "maxItems": 25,
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("datascrapers/lidl-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQueries": ["heissluftfritteuse"],
    "startUrls": [{ "url": "https://www.lidl.de/h/garten-balkon/h10067558" }],
    "maxItems": 25,
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("datascrapers/lidl-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "heissluftfritteuse"
  ],
  "startUrls": [
    {
      "url": "https://www.lidl.de/h/garten-balkon/h10067558"
    }
  ],
  "maxItems": 25,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}' |
apify call datascrapers/lidl-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datascrapers/lidl-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/g0nWcoRTKPYtB7ToT/builds/Zm0rJVu4L5TmIXwkZ/openapi.json