# Webpage to JSON for AI Agents (MCP) (`s3nafps/webpage-to-json-for-ai-agents`) Actor

Turn any web page into clean, typed JSON with AI. Describe what you want in plain English or a JSON schema; every value comes with a source quote. LLM included, no API key. Pay only for pages with data.

- **URL**: https://apify.com/s3nafps/webpage-to-json-for-ai-agents.md
- **Developed by:** [mohamed senator](https://apify.com/s3nafps) (community)
- **Categories:** AI, MCP servers, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $30.00 / 1,000 page with extracted data

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does Webpage to JSON for AI Agents do?

**Webpage to JSON for AI Agents** turns any public web page into **clean, typed JSON** using an AI model. You describe what you want, in plain English, as a simple list of fields, or as a JSON Schema. You get back structured records, and **every value comes with a verbatim quote from the page** that proves where it came from.

It's a **web-to-JSON API built for AI agents and MCP clients** like Claude, Cursor and ChatGPT: the AI model is **included in the price**, so you don't need your own OpenAI or Anthropic API key.

- 🧾 **One record per page**: product details, an article's metadata, a company's contact info
- 📋 **Or a list of records per page**: every product on a category page, every job on a careers page, every event on a calendar
- 🔍 **Evidence for every value**: values the AI can't back with a quote from the page are dropped, not guessed
- 💸 **Pay only for results**: pages where nothing was found, and pages that fail, are free

### Why use Webpage to JSON for AI Agents?

Writing CSS selectors for every site is slow, and they break when the site changes. LLM-based extraction fixes that, but most tools have two problems for agents: they need your own API key, and they **make up values** when the page doesn't contain them.

This Actor is built around those two problems:

- **No API key needed.** The model runs on our side and is part of the per-page price. An agent can call it through MCP with nothing but a URL and an instruction.
- **Grounded output.** For every field, the model must quote the page. The Actor checks each quote against the real page text. If the quote isn't there, the value is set to `null` and the field is listed in `unverifiedFields`. Your agent never gets an invented price or phone number.
- **Schema-checked.** With a JSON Schema, output is validated and automatically repaired once if it doesn't match.
- **Agent-friendly output.** Small, predictable JSON, not a wall of page text.
- **Works with JavaScript-heavy pages.** Pass the dataset from an [AI-Ready Webpage Extractor](https://apify.com/s3nafps/ai-ready-webpage-extractor) run (which renders JavaScript) as the source, and extract from the already-cleaned content without fetching the pages again.
- **Everything Apify gives you**: API access, scheduling, webhooks, integrations with Make, Zapier, n8n and Google Sheets, and spending limits per run.

#### Common use cases

- **AI agents** that need specific facts from a page (price, date, address, spec) instead of reading the whole thing
- **Product and price monitoring** across shops with different layouts
- **Lead lists** from directories, team pages and exhibitor lists
- **Job, event and real-estate listings** turned into rows for a sheet or a database
- **RAG and knowledge bases** that need metadata fields next to the text

### What data can it extract?

Whatever you ask for. Each output item describes one page:

| Field | Type | Description |
|---|---|---|
| `url` | string | The page URL |
| `status` | string | `ok` (data found, charged), `empty` (nothing matching found, free) or `failed` (free) |
| `data` | object or array | The extracted record, or a list of records in list mode |
| `evidence` | object or array | For each field, the exact quote from the page that supports the value |
| `unverifiedFields` | array | Fields whose quote couldn't be found on the page (set to `null` when evidence is required) |
| `schemaValid` | boolean | Whether the output matches your JSON Schema (if you gave one) |
| `recordCount` | integer | Number of records found on the page |
| `error` | string | What went wrong, for failed pages |

### How to extract structured data from a web page

1. Click **Try for free** and open the **Input** tab.
2. Paste one or more **page URLs**, one per line.
3. Describe **what to extract** in plain English, for example *"product name, price as a number, currency and whether it's in stock"*.
4. Optional: list exact **fields** like `price (number): price without currency symbol` to fix the field names and types, or paste a full **JSON Schema**.
5. Turn on **list mode** if each page has many items (a category page, a job board, an events list).
6. Click **Start** and download your data as JSON, CSV, Excel or HTML.

### How much does it cost to extract data from a web page?

This Actor uses **pay-per-event pricing**:

- **$0.03 per page with extracted data.** The AI model is included.
- **Pages where nothing matching was found are free.** So are pages that fail to load.
- In list mode you still pay **once per page**, however many records it contains. A category page with 40 products costs the same $0.03.

**Example:** extracting product details from 1,000 product pages costs about **$30**. Extracting 40 products from each of 25 category pages (1,000 products) costs about **$0.75**.

You can set a **maximum cost per run** in the run options. The Actor stops cleanly before it goes over.

### Input

See the **Input** tab for every option. The main ones:

- **Page URLs**: the pages to extract from. Each URL is one page; this Actor doesn't crawl links.
- **What to extract**: a plain-English instruction. On its own, the AI picks clear field names.
- **Fields (simple)**: one field per line, in the form `name (type): description`. Types: `string`, `number`, `integer`, `boolean`, `array`.
- **JSON schema (advanced)**: a JSON Schema for one record. It takes priority over Fields, and output is validated against it.
- **List mode**: return an array of records per page.
- **Require evidence** (on by default): drop any value the AI can't support with a quote from the page.
- **Source dataset ID**: extract from an existing Apify dataset instead of fetching URLs.

#### Example input

```json
{
  "urls": ["https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"],
  "instruction": "Extract the book details.",
  "fields": [
    "title",
    "price (number): price without currency symbol",
    "currency: ISO currency code",
    "inStock (boolean)"
  ]
}
```

### Output

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

#### Single record

```json
{
  "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
  "status": "ok",
  "title": "A Light in the Attic | Books to Scrape",
  "data": {
    "title": "A Light in the Attic",
    "price": 51.77,
    "currency": "GBP",
    "inStock": true
  },
  "evidence": {
    "title": "A Light in the Attic",
    "price": "£51.77",
    "currency": "£51.77",
    "inStock": "In stock (22 available)"
  },
  "unverifiedFields": [],
  "schemaValid": true,
  "recordCount": 1,
  "error": null
}
```

#### List mode

```json
{
  "url": "https://example.com/careers",
  "status": "ok",
  "data": [
    { "jobTitle": "Backend Engineer", "location": "Remote", "team": "Platform" },
    { "jobTitle": "Product Designer", "location": "Berlin", "team": null }
  ],
  "evidence": [
    { "jobTitle": "Backend Engineer", "location": "Backend Engineer · Remote", "team": "Platform team" },
    { "jobTitle": "Product Designer", "location": "Product Designer · Berlin" }
  ],
  "recordCount": 2
}
```

### Use it from Claude, Cursor or ChatGPT (MCP)

Add this Actor as a tool through the [Apify MCP server](https://mcp.apify.com/):

```
https://mcp.apify.com/?actors=s3nafps/webpage-to-json-for-ai-agents
```

Then just ask your assistant, for example: *"Get the price, currency and stock status from these three product pages."* The agent sends the URLs and an instruction, and gets back verified JSON. No API key or schema writing needed.

You can also call it from code with the [Apify API](https://docs.apify.com/api/v2) or the Apify client for JavaScript or Python. See the **API** tab.

### Tips for better results

- **Be specific about types and units.** "price as a number without currency symbol" beats "price".
- **Use list mode for listing pages**, and set **Max records per page** to what you need.
- **JavaScript-heavy page?** If a page comes back as `failed` with "No readable text", run it through [AI-Ready Webpage Extractor](https://apify.com/s3nafps/ai-ready-webpage-extractor) first. Then pass that run's dataset ID here as **Source dataset ID**.
- **Very long pages** are trimmed to *Max page characters* (30,000 by default). Raise it if the data you need sits near the bottom.
- **Turn off "Require evidence"** only when you need values the AI has to compute (for example a total or a summary). Those can't be quoted word for word.

### FAQ

#### Does it make up data?

It's designed not to. The model is told to use `null` for anything that isn't on the page, and every value must come with a quote. The Actor checks those quotes against the page text and removes values it can't confirm.

#### Do I need an OpenAI or Anthropic API key?

No. The AI model is included in the price per page.

#### Can it log in or crawl a whole website?

No. It extracts from the public pages you give it. To crawl a site and collect pages first, use [AI-Ready Webpage Extractor](https://apify.com/s3nafps/ai-ready-webpage-extractor) and pass its dataset here.

#### Is it legal to extract data from websites?

Our Actors are ethical and do not extract any private user data, such as email addresses, gender, or location. They only extract what the user has chosen to share publicly. We therefore believe that our Actors, when used for ethical purposes by Apify users, are safe. However, you should be aware that your results could contain personal data. Personal data is protected by the GDPR in the European Union and by other regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers.

#### Something didn't work

Open an issue in the **Issues** tab with the URL and your input, and I'll take a look. If the Actor saves you time, a short review on its Store page helps other people find it.

### Related Actors

- [AI-Ready Webpage Extractor](https://apify.com/s3nafps/ai-ready-webpage-extractor): clean Markdown and RAG chunks from web pages and whole websites, with JavaScript rendering. Its dataset can be the source for this Actor.
- [PDF & Word to Markdown for RAG](https://apify.com/s3nafps/doc-to-markdown): clean Markdown and chunks with page numbers from PDF and Word files.

# Actor input Schema

## `urls` (type: `array`):

Public web pages to extract data from, one URL per line. Each URL is one page (no crawling). Leave empty if you use a dataset as the source.

## `instruction` (type: `string`):

Describe the data you want, e.g. "the product name, price as a number, currency and whether it is in stock". Use this alone, or together with Fields or a JSON schema to add context.

## `fields` (type: `array`):

Optional. One field per line in the form `name (type): description`. Types: string, number, integer, boolean, array. Example: `price (number): current price without currency symbol`. Ignored if a JSON schema is given.

## `extractionSchema` (type: `object`):

Optional. A JSON Schema object describing ONE record. Output is validated against it. Takes priority over Fields.

## `listMode` (type: `boolean`):

Turn on for listing pages (products, jobs, events, team members...). Each page then returns an array of records instead of one.

## `maxRecordsPerPage` (type: `integer`):

Upper limit of records returned per page in list mode.

## `requireEvidence` (type: `boolean`):

When on, any value whose source quote cannot be found on the page is set to null instead of being returned. Recommended for agents and anything you will trust without checking.

## `datasetId` (type: `string`):

Optional. Extract from an existing Apify dataset instead of fetching URLs, e.g. the output of AI-Ready Webpage Extractor. Saves a re-fetch and works for JavaScript-heavy pages that were already rendered.

## `datasetTextField` (type: `string`):

Field holding the page content in the source dataset. Leave empty to auto-detect (markdown, text, content).

## `datasetUrlField` (type: `string`):

Field holding the page URL in the source dataset.

## `maxPages` (type: `integer`):

Maximum number of pages (URLs or dataset items) to process in this run.

## `maxContentChars` (type: `integer`):

Longer pages are trimmed to this length before extraction. Raise it for very long pages.

## `includePageText` (type: `boolean`):

Adds the cleaned Markdown that was sent to the AI. Useful for debugging; makes output larger.

## `maxConcurrency` (type: `integer`):

How many pages to process in parallel.

## `proxyConfiguration` (type: `object`):

Use a proxy if target sites block direct requests.

## Actor input object example

```json
{
  "urls": [
    "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
  ],
  "instruction": "Extract the book title, price as a number, currency code, and whether it is in stock.",
  "listMode": false,
  "maxRecordsPerPage": 50,
  "requireEvidence": true,
  "datasetUrlField": "url",
  "maxPages": 100,
  "maxContentChars": 30000,
  "includePageText": false,
  "maxConcurrency": 5,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
    ],
    "instruction": "Extract the book title, price as a number, currency code, and whether it is in stock."
};

// Run the Actor and wait for it to finish
const run = await client.actor("s3nafps/webpage-to-json-for-ai-agents").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": ["https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"],
    "instruction": "Extract the book title, price as a number, currency code, and whether it is in stock.",
}

# Run the Actor and wait for it to finish
run = client.actor("s3nafps/webpage-to-json-for-ai-agents").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
  ],
  "instruction": "Extract the book title, price as a number, currency code, and whether it is in stock."
}' |
apify call s3nafps/webpage-to-json-for-ai-agents --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,s3nafps/webpage-to-json-for-ai-agents"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/DwU3iDXJH4ebsIaNC/builds/gfhQU84tILsRBqVTe/openapi.json