# AI Web Data Extractor – Any URL to JSON with Your Schema (`filexy634/ai-structured-extractor`) Actor

- **URL**: https://apify.com/filexy634/ai-structured-extractor.md
- **Developed by:** [felix gerald](https://apify.com/filexy634) (community)
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $1.00 / 1,000 page-extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI Web Data Extractor – Any URL to JSON with Your Schema

Describe the fields you want, give it URLs, and get clean structured JSON back. No CSS selectors to write or maintain: the page is cleaned to readable text and an LLM fills in your schema. Built to be called by people, scripts and AI agents (it works as a tool through Apify's MCP server).

### How it works

1. Fetches each page and strips scripts, navigation and noise, keeping headings, lists, tables, links, image URLs and JSON-LD.
2. Sends the cleaned text and your schema to the model with a forced tool call, so the output is always structured.
3. Validates every result against your JSON Schema (`valid` flag) and caches identical requests.

### Bring your own LLM key

This Actor uses **your own API key** (Anthropic, OpenAI or any OpenAI-compatible endpoint such as OpenRouter). Model usage is billed by your provider at cost; you only pay this Actor's small per-page fee. Token counts are included in every result so you can see exactly what each page cost. The key is stored encrypted, is used only for model calls in your run, and is never written to the dataset or logs.

### Input

| Field | Description |
|---|---|
| `urls` | Pages to extract from |
| `schema` | A JSON Schema object, or a simple map of field name to description |
| `instructions` | Optional extra guidance for the model |
| `multiple` | Extract a list of items per page (listing pages) |
| `llmProvider`, `llmApiKey`, `model`, `llmBaseUrl` | Your model access |
| `maxContentChars` | Cap on page text sent to the model (default 30,000) |
| `cacheTtlHours` | Reuse results for identical page content, schema and model (default 24) |

Simple schema example:

```json
{ "title": "Book title", "price": "Price as a number", "in_stock": "true if available" }
```

JSON Schema example:

```json
{ "type": "object", "properties": { "name": { "type": "string" }, "price": { "type": "number" } }, "required": ["name", "price"] }
```

### Output

```json
{
  "url": "https://shop.example/p/123",
  "status": "ok",
  "data": { "name": "Anvil 20kg", "price": 89.99 },
  "valid": true,
  "cached": false,
  "model": "claude-haiku-5-5",
  "inputTokens": 1480,
  "outputTokens": 62
}
```

Pages that fail (not found, empty, blocked, model error) appear with `status: "error"` and are **not charged**. Fields missing on a page come back as `null`; the model is told never to guess.

### Pricing

Charged per page successfully extracted, including cached results. Failed pages are free.

### Limitations

- Static HTML only. Pages that render their content with client-side JavaScript return a clear "no readable content" error.
- Page text is untrusted: the model is instructed to ignore instructions embedded in pages, but always validate outputs before acting on them in automated pipelines.
- Very long pages are truncated at `maxContentChars`; check the `truncated` flag.
- Only extract data you are allowed to process, and follow each site's terms of service and applicable privacy law.

### Developer notes

Event to create in Console → Monetization (delete the default `apify-default-dataset-item` event, otherwise error rows get billed):

- `page-extracted` (suggested start: $0.005)
  Set it as the primary event.

# Actor input Schema

## `urls` (type: `array`):

Web pages to extract data from, one per line.

## `schema` (type: `object`):

Either a JSON Schema object (type: object with properties), or a simple map of field name to a plain-English description, e.g. {"title": "Book title", "price": "Price as a number"}.

## `instructions` (type: `string`):

Optional guidance for the model, e.g. "Prices are in euros" or "Only extract products from the main listing".

## `multiple` (type: `boolean`):

Turn on for listing pages. The result becomes an array with one entry per matching item, each following your schema.

## `llmProvider` (type: `string`):

Which API your key belongs to. Anthropic uses the Messages API; OpenAI also covers any OpenAI-compatible endpoint via the base URL field.

## `llmApiKey` (type: `string`):

Your own API key. It is stored encrypted, used only for this run's model calls and never written to the dataset. LLM usage is billed by your provider, not by this Actor.

## `model` (type: `string`):

Model name. Leave empty for a cheap, fast default (Anthropic: claude-haiku-5-5, OpenAI: gpt-4o-mini).

## `llmBaseUrl` (type: `string`):

Optional. Override for OpenAI-compatible providers such as OpenRouter, e.g. https://openrouter.ai/api/v1.

## `maxContentChars` (type: `integer`):

Pages are cleaned to text and cut at this length to control model cost. Raise it for very long pages.

## `cacheTtlHours` (type: `integer`):

Reuse a previous result when the same page content, schema, instructions and model are seen again within this time. 0 disables the cache.

## `maxConcurrency` (type: `integer`):

How many pages are processed in parallel. Lower it if your LLM provider rate-limits you.

## `proxyConfiguration` (type: `object`):

Optional. Only needed for sites that block data-center traffic.

## Actor input object example

```json
{
  "urls": [
    "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
  ],
  "schema": {
    "title": "Book title",
    "price": "Price as a number, without currency symbol",
    "currency": "ISO currency code",
    "in_stock": "true if the book is in stock, otherwise false",
    "rating": "Star rating as a number from 1 to 5"
  },
  "multiple": false,
  "llmProvider": "anthropic",
  "maxContentChars": 30000,
  "cacheTtlHours": 24,
  "maxConcurrency": 5,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset of extraction results.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
    ],
    "schema": {
        "title": "Book title",
        "price": "Price as a number, without currency symbol",
        "currency": "ISO currency code",
        "in_stock": "true if the book is in stock, otherwise false",
        "rating": "Star rating as a number from 1 to 5"
    },
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("filexy634/ai-structured-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": ["https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"],
    "schema": {
        "title": "Book title",
        "price": "Price as a number, without currency symbol",
        "currency": "ISO currency code",
        "in_stock": "true if the book is in stock, otherwise false",
        "rating": "Star rating as a number from 1 to 5",
    },
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("filexy634/ai-structured-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
  ],
  "schema": {
    "title": "Book title",
    "price": "Price as a number, without currency symbol",
    "currency": "ISO currency code",
    "in_stock": "true if the book is in stock, otherwise false",
    "rating": "Star rating as a number from 1 to 5"
  },
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call filexy634/ai-structured-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,filexy634/ai-structured-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/9BaLsvCd7VLJmD0Bt/builds/O1l7oHKieOFg4YUlT/openapi.json