# Universal Recipe Data Scraper (`automation-lab/universal-recipe-data-scraper`) Actor

Extract normalized ingredients, instructions, nutrition, ratings, images, and metadata from public recipe URLs using Schema.org Recipe JSON-LD.

- **URL**: https://apify.com/automation-lab/universal-recipe-data-scraper.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Universal Recipe Data Scraper

Turn public recipe page URLs into consistent, typed recipe records.

Universal Recipe Data Scraper reads Schema.org `Recipe` JSON-LD already published in a page's HTML and normalizes it for datasets, meal-planning apps, nutrition pipelines, recommendations, RAG systems, and content operations.

It extracts ingredients, ordered method steps and sections, timings, yield, cuisine, categories, nutrition, ratings, authors, images, videos, dates, and URL provenance. One dataset item is emitted for each distinct Recipe object.

### What this Recipe scraper does

1. Accepts public HTTP or HTTPS recipe page URLs.
2. Fetches pages with lightweight HTTP requests.
3. Finds Recipe objects in standalone JSON-LD, arrays, and `@graph`.
4. Normalizes common Schema.org scalar, array, image, author, and instruction variants.
5. Deduplicates repeated Recipe objects on the same page.
6. Charges only for recipes successfully saved to the default dataset.

No browser, login, cookie, or private API is required for the minimum workflow. Direct requests are the default. A proxy is used only when you explicitly configure one.

### Who is it for

- **Recipe and meal-planning app teams** importing recipes from user-supplied URLs.
- **Culinary publishers** auditing or synchronizing structured recipe metadata.
- **Nutrition and data teams** collecting ingredient and nutrition facts at scale.
- **Recommendation engineers** creating normalized recipe catalogs.
- **RAG and AI builders** preparing clean recipe documents for retrieval.
- **Analysts** comparing recipe structure, timings, cuisines, and ratings across sites.

Choose this Actor for source-agnostic direct recipe URLs. For discovering and crawling the BBC Good Food catalog itself, use the related BBC-specific Actor listed below.

### Why use structured Recipe JSON-LD

Many cooking websites expose machine-readable Schema.org data for search engines. Reading that surface is faster and less expensive than rendering every page in a browser.

This Actor keeps source values where useful:

- Schema.org durations remain values such as `PT30M`.
- Nutrition values retain published units such as `420 calories`.
- Ingredient and method order is preserved.
- Redirected, canonical, and original input URLs remain distinguishable.
- Optional raw JSON-LD supports custom downstream enrichment.

Coverage depends on the target page publishing a valid `Recipe` object in its initial HTML.

### Extracted recipe data

| Field | Description |
| --- | --- |
| `name` | Required recipe name |
| `description` | Recipe summary when published |
| `author` | Author or organization names and URLs |
| `datePublished`, `dateModified` | Source date values |
| `prepTime`, `cookTime`, `totalTime` | Schema.org/ISO 8601 duration strings |
| `recipeYield` | Serving or quantity labels |
| `recipeCategory` | Normalized category labels |
| `recipeCuisine` | Cuisine labels |
| `keywords` | Source keyword labels |
| `ingredients` | Ingredient lines in source order |
| `instructions` | Ordered steps and nested method sections |
| `nutrition` | Calories, macros, sodium, serving size, and related values |
| `aggregateRating` | Rating value and available rating/review counts |
| `images` | Image URLs, dimensions, and captions |
| `videos` | Video URLs and available media metadata |
| `sourceUrl` | Original user-supplied URL |
| `fetchedUrl` | Final URL after redirects |
| `canonicalUrl` | Recipe or page canonical URL |
| `scrapedAt` | UTC extraction timestamp |
| `rawJsonLd` | Original Recipe object when enabled |

### Get started

1. Open the Actor in Apify Console.
2. Add one or more public recipe page URLs to **Recipe page URLs**.
3. Keep `maxItems` small for the first run.
4. Leave the proxy disabled unless a target site blocks direct requests.
5. Click **Start**.
6. Open the **Dataset** tab and export JSON, CSV, Excel, XML, or another supported format.

A page can contain multiple distinct Recipe objects. `maxItems` limits successful recipe records, not input URLs.

### Input parameters

#### `startUrls`

Required array of public HTTP(S) recipe pages. Strings and Apify request-list objects are accepted by the code; Console uses request-list objects.

Fragments are removed and duplicate URLs are fetched once. Redirects are followed and recorded in `fetchedUrl`.

#### `maxItems`

Maximum recipes to emit. Default: `100`. Range: `1` to `10,000`.

The Actor stops scheduling new URL batches after the limit is reached. If the final fetched page contains several recipes, only records fitting the requested result limit are saved.

#### `maxConcurrency`

Maximum pages fetched in parallel. Default: `5`. Range: `1` to `20`.

Reduce this for sensitive or rate-limited domains. Concurrency is deliberately bounded per run.

#### `requestTimeoutSecs`

HTTP timeout for one attempt. Default: `30` seconds. Range: `5` to `120`.

Network errors, HTTP 429, and temporary 5xx responses receive bounded exponential-backoff retries. Stable 4xx responses are not retried blindly.

#### `includeRawJsonLd`

Adds the original structured Recipe object as `rawJsonLd`. Default: `false`.

Enable it for custom field mapping or provenance. Disable it for smaller datasets.

#### `normalizeText`

Collapses repeated whitespace in text fields. Default: `true`.

Disable it when source line breaks and spacing are significant to your workflow.

#### `proxyConfiguration`

Optional Apify or custom proxy settings. No proxy is used by default and there is no automatic residential or browser fallback.

### Example input

```json
{
  "startUrls": [
    {
      "url": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream"
    }
  ],
  "maxItems": 1,
  "maxConcurrency": 2,
  "includeRawJsonLd": false,
  "normalizeText": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

### Example output

This abbreviated record reflects current extraction from the public sample URL:

```json
{
  "name": "Classic scones with jam & clotted cream",
  "description": "You can have a batch of scones on the table in 20 minutes...",
  "author": [
    {
      "name": "Jane Hornby",
      "url": "https://www.bbcgoodfood.com/author/janehornby"
    }
  ],
  "cookTime": "PT10M",
  "totalTime": "PT15M",
  "recipeYield": ["8"],
  "recipeCategory": ["Afternoon tea", "Breakfast", "Snack", "Treat"],
  "recipeCuisine": ["British"],
  "ingredients": [
    "350g self-raising flour, plus more for dusting",
    "¼ tsp salt",
    "1 tsp baking powder"
  ],
  "instructions": [
    {
      "type": "step",
      "name": null,
      "text": "Heat the oven to 220C/200C fan/gas 7...",
      "url": null
    }
  ],
  "nutrition": {
    "calories": "285 calories",
    "proteinContent": "5 grams protein"
  },
  "sourceUrl": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream",
  "fetchedUrl": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream",
  "canonicalUrl": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream",
  "scrapedAt": "2026-07-27T03:30:00.000Z"
}
```

Null values mean the source Recipe object did not publish that property. Arrays can be empty for the same reason.

### How much does it cost to extract recipe data?

Pricing has two event types:

- **Start:** $0.005 once per run.
- **Recipe:** one event per successfully normalized and saved recipe.

The Free-tier recipe price is **$0.000041071 per recipe**. Paid Apify plans receive progressively lower event prices, down to $0.00001 on the Diamond tier.

Approximate Free-tier examples, including one start event:

| Successful recipes | Recipe events | Approximate total |
| ---: | ---: | ---: |
| 1 | 0.000041071 USD | 0.005041071 USD |
| 100 | 0.0041071 USD | 0.0091071 USD |
| 1,000 | 0.041071 USD | 0.046071 USD |

Failed, unsupported, duplicate, rejected, and limit-exceeding Recipe objects are not charged as recipe events. Optional proxy traffic can increase underlying platform usage but does not create another Actor charge event.

### Catalog synchronization workflow

For a repeat data pipeline:

1. Supply the current batch of public recipe URLs.
2. Keep `canonicalUrl` as the stable catalog key where available.
3. Store `fetchedUrl` to monitor redirects.
4. Compare `dateModified` and normalized fields with the previous dataset.
5. Schedule recurring Actor runs in Apify.
6. Send completed datasets to a webhook, cloud storage, database, or automation tool.

Use `includeRawJsonLd: true` when your downstream mapper needs fields outside the normalized contract.

### Nutrition and recommendation workflow

The Actor preserves nutrition strings and ingredient lines rather than guessing units or converting quantities. A downstream pipeline can:

- parse units with domain-specific rules;
- map ingredients to a food database;
- calculate dietary tags;
- generate recipe embeddings;
- rank recipes by cuisine, timing, yield, or nutrition.

This separation avoids presenting inferred nutrition as source-published fact.

### API with cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~universal-recipe-data-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "startUrls": [{"url": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream"}],
    "maxItems": 1
  }'
```

For production systems, send the token in an authorization header rather than logging it in URLs.

### API with JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/universal-recipe-data-scraper').call({
  startUrls: [
    { url: 'https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream' },
  ],
  maxItems: 1,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

### API with Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("automation-lab/universal-recipe-data-scraper").call(run_input={
    "startUrls": [
        {"url": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream"}
    ],
    "maxItems": 1,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)
```

### Use through Apify MCP

Add this Actor to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/universal-recipe-data-scraper"
```

#### Claude Desktop, Cursor, and VS Code setup

Claude Desktop, Cursor, and VS Code clients can use this HTTP MCP configuration:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/universal-recipe-data-scraper"
    }
  }
}
```

Example prompts:

- "Extract the ingredients and ordered method from these recipe URLs."
- "Build a normalized recipe dataset and keep raw JSON-LD for audit."
- "Compare cooking time, yield, and nutrition across this recipe list."

### Errors and retry behavior

The Actor labels URL failures with concise typed codes:

- `FETCH_FAILED` — network failure or non-successful HTTP response.
- `UNSUPPORTED_CONTENT` — the response is not HTML.
- `NO_RECIPE_DATA` — no valid Schema.org Recipe object appears in initial HTML.

A mixed run saves valid recipes and logs failed URLs. If every processed URL fails, the run exits non-zero instead of silently returning an empty successful dataset.

### Limits and website coverage

- Pages must expose Recipe JSON-LD in their initial HTML.
- JavaScript-only structured data is not rendered.
- Login-only and private pages are unsupported.
- Some sites block direct cloud or datacenter requests.
- A configured proxy can help delivery but cannot create missing structured data.
- Site-specific fields outside the normalized contract require `includeRawJsonLd`.
- Nutrition values are not medically validated or converted.
- The Actor does not discover recipe URLs from searches, collections, or sitemaps.

A browser fallback is intentionally not automatic, keeping memory and cost predictable.

### Performance tips

- Group URLs by domain and use conservative concurrency for sensitive sites.
- Use direct requests first.
- Keep `includeRawJsonLd` off unless needed.
- Set `maxItems` to the size your downstream system can consume.
- Split very large URL collections into scheduled batches.
- Inspect failure codes before enabling a proxy.

### Responsible use and legality

Only process public pages you are authorized to access. Follow applicable website terms, robots guidance, copyright rules, database rights, privacy laws, and rate limits.

Recipe facts and structured metadata can still be protected or restricted in some jurisdictions. The Actor does not grant rights to republish source text, images, videos, or branding. Review your intended storage and reuse with appropriate legal guidance.

Avoid collecting personal data not needed for your workflow. Do not use the Actor to bypass authentication or access controls.

### Related Automation Lab Actor

- [BBC Good Food Recipe Catalogue Scraper](https://apify.com/automation-lab/bbc-good-food-recipe-catalogue-scraper) — discover and extract the BBC Good Food catalog when you need source-specific collection, search, or sitemap workflows rather than supplied recipe URLs.

### Troubleshooting

#### Why did a page return `NO_RECIPE_DATA`?

Open the page source and check for an `application/ld+json` script with `"@type": "Recipe"`. A visual recipe page can omit structured data or insert it only after JavaScript runs.

#### Why did one URL fail while others succeeded?

The Actor intentionally isolates URL failures. Review the code and URL in the run log. HTTP 403 commonly means the domain blocks the current delivery route; try a permitted proxy configuration or use another public source URL.

#### Why is a field null or an array empty?

Schema.org Recipe properties are optional. The Actor does not invent absent dates, nutrition, ratings, images, or timings.

#### Why are there fewer records than URLs?

Duplicate URLs are fetched once, unsupported pages emit no recipe, and `maxItems` can stop output before all inputs are processed. Conversely, one URL can emit multiple distinct recipes.

#### Can I preserve original whitespace and custom fields?

Yes. Set `normalizeText` to `false` and `includeRawJsonLd` to `true`.

### FAQ

#### Does it scrape any website?

It supports public HTML pages that publish valid Schema.org Recipe JSON-LD. It does not promise universal delivery through anti-bot systems or extraction from site-specific visual HTML without Recipe structured data.

#### Does it use a browser?

No. It is HTTP-first and uses 256 MB memory by default.

#### Does it automatically use residential proxies?

No. Proxy use is explicit, so runs do not trigger unmeasured residential traffic automatically.

#### Are failed URLs charged as recipes?

No. Only successfully normalized Recipe objects saved to the dataset produce a Recipe event. The one-time Start event still applies once a run starts.

#### Can a page return multiple recipes?

Yes. Each distinct Recipe JSON-LD object can become one dataset item, subject to deduplication and `maxItems`.

#### Can I export the results?

Yes. The default Apify dataset supports JSON, CSV, Excel, XML, RSS, and other platform export formats.

# Actor input Schema

## `startUrls` (type: `array`):

Public HTTP(S) recipe pages containing Schema.org Recipe JSON-LD. Duplicate URLs and URL fragments are removed.

## `maxItems` (type: `integer`):

Stop after this many successfully normalized Recipe objects. A page can contain more than one recipe.

## `maxConcurrency` (type: `integer`):

Maximum recipe pages fetched in parallel. Reduce this for rate-limited websites.

## `requestTimeoutSecs` (type: `integer`):

Maximum time to wait for one HTTP response before a bounded retry.

## `includeRawJsonLd` (type: `boolean`):

Add the original structured Recipe object as rawJsonLd for custom downstream processing. Disabled by default to keep datasets compact.

## `normalizeText` (type: `boolean`):

Collapse repeated spaces and line breaks in text fields. Disable to preserve the source text formatting.

## `proxyConfiguration` (type: `object`):

Optional Apify or custom proxy settings for sites that block direct requests. No proxy is used unless configured here.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream"
    },
    {
      "url": "https://www.bbcgoodfood.com/recipes/easy-pancakes"
    }
  ],
  "maxItems": 10,
  "maxConcurrency": 5,
  "requestTimeoutSecs": 30,
  "includeRawJsonLd": false,
  "normalizeText": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `overview` (type: `string`):

Open normalized recipe records in the overview dataset view.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream"
        },
        {
            "url": "https://www.bbcgoodfood.com/recipes/easy-pancakes"
        }
    ],
    "maxItems": 10,
    "maxConcurrency": 5
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/universal-recipe-data-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [
        { "url": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream" },
        { "url": "https://www.bbcgoodfood.com/recipes/easy-pancakes" },
    ],
    "maxItems": 10,
    "maxConcurrency": 5,
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/universal-recipe-data-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream"
    },
    {
      "url": "https://www.bbcgoodfood.com/recipes/easy-pancakes"
    }
  ],
  "maxItems": 10,
  "maxConcurrency": 5
}' |
apify call automation-lab/universal-recipe-data-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=automation-lab/universal-recipe-data-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/rnim6enueu2JNGeIh/builds/ewT0UtjPFRfAMvP2a/openapi.json