Universal Recipe Data Scraper

Turn public recipe page URLs into consistent, typed recipe records.

Universal Recipe Data Scraper reads Schema.org Recipe JSON-LD already published in a page's HTML and normalizes it for datasets, meal-planning apps, nutrition pipelines, recommendations, RAG systems, and content operations.

It extracts ingredients, ordered method steps and sections, timings, yield, cuisine, categories, nutrition, ratings, authors, images, videos, dates, and URL provenance. One dataset item is emitted for each distinct Recipe object.

What this Recipe scraper does

1. Accepts public HTTP or HTTPS recipe page URLs.
2. Fetches pages with lightweight HTTP requests.
3. Finds Recipe objects in standalone JSON-LD, arrays, and @graph.
4. Normalizes common Schema.org scalar, array, image, author, and instruction variants.
5. Deduplicates repeated Recipe objects on the same page.
6. Charges only for recipes successfully saved to the default dataset.

No browser, login, cookie, or private API is required for the minimum workflow. Direct requests are the default. A proxy is used only when you explicitly configure one.

Who is it for

• Recipe and meal-planning app teams importing recipes from user-supplied URLs.
• Culinary publishers auditing or synchronizing structured recipe metadata.
• Nutrition and data teams collecting ingredient and nutrition facts at scale.
• Recommendation engineers creating normalized recipe catalogs.
• RAG and AI builders preparing clean recipe documents for retrieval.
• Analysts comparing recipe structure, timings, cuisines, and ratings across sites.

Choose this Actor for source-agnostic direct recipe URLs. For discovering and crawling the BBC Good Food catalog itself, use the related BBC-specific Actor listed below.

Why use structured Recipe JSON-LD

Many cooking websites expose machine-readable Schema.org data for search engines. Reading that surface is faster and less expensive than rendering every page in a browser.

This Actor keeps source values where useful:

• Schema.org durations remain values such as PT30M.
• Nutrition values retain published units such as 420 calories.
• Ingredient and method order is preserved.
• Redirected, canonical, and original input URLs remain distinguishable.
• Optional raw JSON-LD supports custom downstream enrichment.

Coverage depends on the target page publishing a valid Recipe object in its initial HTML.

Extracted recipe data

Field	Description
name	Required recipe name
description	Recipe summary when published
author	Author or organization names and URLs
datePublished, dateModified	Source date values
prepTime, cookTime, totalTime	Schema.org/ISO 8601 duration strings
recipeYield	Serving or quantity labels
recipeCategory	Normalized category labels
recipeCuisine	Cuisine labels
keywords	Source keyword labels
ingredients	Ingredient lines in source order
instructions	Ordered steps and nested method sections
nutrition	Calories, macros, sodium, serving size, and related values
aggregateRating	Rating value and available rating/review counts
images	Image URLs, dimensions, and captions
videos	Video URLs and available media metadata
sourceUrl	Original user-supplied URL
fetchedUrl	Final URL after redirects
canonicalUrl	Recipe or page canonical URL
scrapedAt	UTC extraction timestamp
rawJsonLd	Original Recipe object when enabled

Get started

1. Open the Actor in Apify Console.
2. Add one or more public recipe page URLs to Recipe page URLs.
3. Keep maxItems small for the first run.
4. Leave the proxy disabled unless a target site blocks direct requests.
5. Click Start.
6. Open the Dataset tab and export JSON, CSV, Excel, XML, or another supported format.

A page can contain multiple distinct Recipe objects. maxItems limits successful recipe records, not input URLs.

Input parameters

startUrls

Required array of public HTTP(S) recipe pages. Strings and Apify request-list objects are accepted by the code; Console uses request-list objects.

Fragments are removed and duplicate URLs are fetched once. Redirects are followed and recorded in fetchedUrl.

maxItems

Maximum recipes to emit. Default: 100. Range: 1 to 10,000.

The Actor stops scheduling new URL batches after the limit is reached. If the final fetched page contains several recipes, only records fitting the requested result limit are saved.

maxConcurrency

Maximum pages fetched in parallel. Default: 5. Range: 1 to 20.

Reduce this for sensitive or rate-limited domains. Concurrency is deliberately bounded per run.

requestTimeoutSecs

HTTP timeout for one attempt. Default: 30 seconds. Range: 5 to 120.

Network errors, HTTP 429, and temporary 5xx responses receive bounded exponential-backoff retries. Stable 4xx responses are not retried blindly.

includeRawJsonLd

Adds the original structured Recipe object as rawJsonLd. Default: false.

Enable it for custom field mapping or provenance. Disable it for smaller datasets.

normalizeText

Collapses repeated whitespace in text fields. Default: true.

Disable it when source line breaks and spacing are significant to your workflow.

proxyConfiguration

Optional Apify or custom proxy settings. No proxy is used by default and there is no automatic residential or browser fallback.

Example input

{
  "startUrls": [
    {
      "url": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream"
    }
  ],
  "maxItems": 1,
  "maxConcurrency": 2,
  "includeRawJsonLd": false,
  "normalizeText": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Example output

This abbreviated record reflects current extraction from the public sample URL:

{
  "name": "Classic scones with jam & clotted cream",
  "description": "You can have a batch of scones on the table in 20 minutes...",
  "author": [
    {
      "name": "Jane Hornby",
      "url": "https://www.bbcgoodfood.com/author/janehornby"
    }
  ],
  "cookTime": "PT10M",
  "totalTime": "PT15M",
  "recipeYield": ["8"],
  "recipeCategory": ["Afternoon tea", "Breakfast", "Snack", "Treat"],
  "recipeCuisine": ["British"],
  "ingredients": [
    "350g self-raising flour, plus more for dusting",
    "¼ tsp salt",
    "1 tsp baking powder"
  ],
  "instructions": [
    {
      "type": "step",
      "name": null,
      "text": "Heat the oven to 220C/200C fan/gas 7...",
      "url": null
    }
  ],
  "nutrition": {
    "calories": "285 calories",
    "proteinContent": "5 grams protein"
  },
  "sourceUrl": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream",
  "fetchedUrl": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream",
  "canonicalUrl": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream",
  "scrapedAt": "2026-07-27T03:30:00.000Z"
}

Null values mean the source Recipe object did not publish that property. Arrays can be empty for the same reason.

How much does it cost to extract recipe data?

Pricing has two event types:

• Start: $0.005 once per run.
• Recipe: one event per successfully normalized and saved recipe.

The Free-tier recipe price is $0.000041071 per recipe. Paid Apify plans receive progressively lower event prices, down to $0.00001 on the Diamond tier.

Approximate Free-tier examples, including one start event:

Successful recipes	Recipe events	Approximate total
1	0.000041071 USD	0.005041071 USD
100	0.0041071 USD	0.0091071 USD
1,000	0.041071 USD	0.046071 USD

Failed, unsupported, duplicate, rejected, and limit-exceeding Recipe objects are not charged as recipe events. Optional proxy traffic can increase underlying platform usage but does not create another Actor charge event.

Catalog synchronization workflow

For a repeat data pipeline:

1. Supply the current batch of public recipe URLs.
2. Keep canonicalUrl as the stable catalog key where available.
3. Store fetchedUrl to monitor redirects.
4. Compare dateModified and normalized fields with the previous dataset.
5. Schedule recurring Actor runs in Apify.
6. Send completed datasets to a webhook, cloud storage, database, or automation tool.

Use includeRawJsonLd: true when your downstream mapper needs fields outside the normalized contract.

Nutrition and recommendation workflow

The Actor preserves nutrition strings and ingredient lines rather than guessing units or converting quantities. A downstream pipeline can:

• parse units with domain-specific rules;
• map ingredients to a food database;
• calculate dietary tags;
• generate recipe embeddings;
• rank recipes by cuisine, timing, yield, or nutrition.

This separation avoids presenting inferred nutrition as source-published fact.

API with cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~universal-recipe-data-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "startUrls": [{"url": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream"}],
    "maxItems": 1
  }'

For production systems, send the token in an authorization header rather than logging it in URLs.

API with JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/universal-recipe-data-scraper').call({
  startUrls: [
    { url: 'https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream' },
  ],
  maxItems: 1,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

API with Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("automation-lab/universal-recipe-data-scraper").call(run_input={
    "startUrls": [
        {"url": "https://www.bbcgoodfood.com/recipes/classic-scones-jam-clotted-cream"}
    ],
    "maxItems": 1,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

Use through Apify MCP

Add this Actor to Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/universal-recipe-data-scraper"

Claude Desktop, Cursor, and VS Code setup

Claude Desktop, Cursor, and VS Code clients can use this HTTP MCP configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/universal-recipe-data-scraper"
    }
  }
}

Example prompts:

• "Extract the ingredients and ordered method from these recipe URLs."
• "Build a normalized recipe dataset and keep raw JSON-LD for audit."
• "Compare cooking time, yield, and nutrition across this recipe list."

Errors and retry behavior

The Actor labels URL failures with concise typed codes:

• FETCH_FAILED — network failure or non-successful HTTP response.
• UNSUPPORTED_CONTENT — the response is not HTML.
• NO_RECIPE_DATA — no valid Schema.org Recipe object appears in initial HTML.

A mixed run saves valid recipes and logs failed URLs. If every processed URL fails, the run exits non-zero instead of silently returning an empty successful dataset.

Limits and website coverage

• Pages must expose Recipe JSON-LD in their initial HTML.
• JavaScript-only structured data is not rendered.
• Login-only and private pages are unsupported.
• Some sites block direct cloud or datacenter requests.
• A configured proxy can help delivery but cannot create missing structured data.
• Site-specific fields outside the normalized contract require includeRawJsonLd.
• Nutrition values are not medically validated or converted.
• The Actor does not discover recipe URLs from searches, collections, or sitemaps.

A browser fallback is intentionally not automatic, keeping memory and cost predictable.

Performance tips

• Group URLs by domain and use conservative concurrency for sensitive sites.
• Use direct requests first.
• Keep includeRawJsonLd off unless needed.
• Set maxItems to the size your downstream system can consume.
• Split very large URL collections into scheduled batches.
• Inspect failure codes before enabling a proxy.

Responsible use and legality

Only process public pages you are authorized to access. Follow applicable website terms, robots guidance, copyright rules, database rights, privacy laws, and rate limits.

Recipe facts and structured metadata can still be protected or restricted in some jurisdictions. The Actor does not grant rights to republish source text, images, videos, or branding. Review your intended storage and reuse with appropriate legal guidance.

Avoid collecting personal data not needed for your workflow. Do not use the Actor to bypass authentication or access controls.

Related Automation Lab Actor

• BBC Good Food Recipe Catalogue Scraper — discover and extract the BBC Good Food catalog when you need source-specific collection, search, or sitemap workflows rather than supplied recipe URLs.

Troubleshooting

Why did a page return NO_RECIPE_DATA?

Open the page source and check for an application/ld+json script with "@type": "Recipe". A visual recipe page can omit structured data or insert it only after JavaScript runs.

Why did one URL fail while others succeeded?

The Actor intentionally isolates URL failures. Review the code and URL in the run log. HTTP 403 commonly means the domain blocks the current delivery route; try a permitted proxy configuration or use another public source URL.

Why is a field null or an array empty?

Schema.org Recipe properties are optional. The Actor does not invent absent dates, nutrition, ratings, images, or timings.

Why are there fewer records than URLs?

Duplicate URLs are fetched once, unsupported pages emit no recipe, and maxItems can stop output before all inputs are processed. Conversely, one URL can emit multiple distinct recipes.

Can I preserve original whitespace and custom fields?

Yes. Set normalizeText to false and includeRawJsonLd to true.

FAQ

Does it scrape any website?

It supports public HTML pages that publish valid Schema.org Recipe JSON-LD. It does not promise universal delivery through anti-bot systems or extraction from site-specific visual HTML without Recipe structured data.

Does it use a browser?

No. It is HTTP-first and uses 256 MB memory by default.

Does it automatically use residential proxies?

No. Proxy use is explicit, so runs do not trigger unmeasured residential traffic automatically.

Are failed URLs charged as recipes?

No. Only successfully normalized Recipe objects saved to the dataset produce a Recipe event. The one-time Start event still applies once a run starts.

Can a page return multiple recipes?

Yes. Each distinct Recipe JSON-LD object can become one dataset item, subject to deduplication and maxItems.

Can I export the results?

Yes. The default Apify dataset supports JSON, CSV, Excel, XML, RSS, and other platform export formats.