# AI Web Extractor (`fuze/ai-web-extractor`) Actor

Extract structured data from any web page with plain-English instructions — no CSS selectors. Powered by OpenAI. Actively maintained. · More no-code data tools → datatools-hub.vercel.app

- **URL**: https://apify.com/fuze/ai-web-extractor.md
- **Developed by:** [Fredrik Hansson](https://apify.com/fuze) (community)
- **Categories:** AI, Automation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $17.60 / 1,000 page extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI Web Extractor

**Extract structured data from any web page with a plain-English instruction — no CSS selectors, no brittle scrapers.**

Give it a list of URLs and tell it what you want in plain English ("product name, price and whether it's in stock"). It fetches each page, reads it with OpenAI, and returns clean, consistent JSON — one record per page.

![AI Web Extractor — input and sample output](https://datatools-hub.vercel.app/actor-shots/ai-web-extractor.png)

### Quick start

1. **Fill in the input** — set the fields in the input tab (prefilled with a working example).
2. **Click Start** — the Actor runs and writes one record per item to the dataset.
3. **Get your data** — export to **JSON, CSV or Excel**, or pull it via the API.

```json
// input
{
  "urls": ["https://example.com/product/123"],
  "extractionPrompt": "product name, price, currency, in-stock (true/false), rating"
}
```

```json
// output (one item per URL)
{
  "url": "https://example.com/product/123",
  "success": true,
  "data": { "product_name": "A Light in the Attic", "price": "51.77", "currency": "£", "in_stock": true, "rating": 4 }
}
```

***

### Why it's different

Traditional scrapers break every time a site changes its HTML, and each new site needs new selectors. This Actor uses an LLM to **read the page like a person**, so:

- **Works on any site** — the same instruction extracts from a shop, a news article, a company page, a listing.
- **No selectors, no maintenance** — describe the fields once; it adapts across layouts.
- **Consistent output** — optionally pin the exact JSON shape so every page returns the same fields.
- **Agent-ready** — the perfect tool for AI agents (via the Apify MCP server): "read this page and give me X."

***

### Use cases

- **No-code scraping** — pull prices, specs, contacts, articles or any fields from a list of URLs without writing code.
- **Lead / data enrichment** — enrich a list of company or product URLs into a structured dataset for your CRM or spreadsheet.
- **AI agents** — call it as a tool so an agent can turn any URL into structured JSON on demand.
- **Monitoring** — run it on a schedule to track fields (price, availability, status) across pages over time.

***

### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `urls` | `string[]` | — (required) | The pages to extract from. |
| `extractionPrompt` | `string` | — (required) | Plain-English description of the fields you want. |
| `schemaHint` | `string` | — | Optional: the exact field names / JSON shape you want back, for consistent output across pages. |
| `maxResults` | `integer` | `100` | Max pages to process (one charge per page extracted). |
| `maxConcurrency` | `integer` | `5` | Pages processed in parallel. |
| `model` | `string` | `gpt-4o-mini` | OpenAI model override. |

***

### Output

One dataset item per page:

| Field | Type | Description |
|---|---|---|
| `url` | `string` | The page URL. |
| `success` | `boolean` | Whether extraction succeeded. |
| `data` | `object \| null` | The extracted structured data (shape follows your prompt / `schemaHint`). |
| `error` | `string` | Present only when `success` is false. |

Real OpenAI token usage and an estimated cost are logged per run and stored in `SUMMARY`.

***

### Use it as an AI-agent tool (Apify MCP)

Once published, this Actor is callable through the **Apify MCP server** (`https://mcp.apify.com`). An agent can be told *"open this page and extract the pricing table"* and call this Actor directly — plain-English in, structured JSON out.

***

### Pricing

Pay-per-event: one **`page-extracted`** event per page successfully extracted. Pages that fail (unreachable, blocked) are reported but not charged.

***

### Environment variables

| Variable | Required | Purpose |
|---|---|---|
| `OPENAI_API_KEY` | **Yes** | Powers the extraction. |
| `OPENAI_MODEL` | No | Alternative way to override the model (input `model` takes precedence). |

***

### Notes & limitations

- **v1 reads server-rendered HTML** (fetched with realistic browser headers to get past basic anti-bot). Pages that render their content entirely with client-side JavaScript may return little data — a JavaScript-rendering option is planned for the next version.
- Page text is truncated before being sent to the model to keep token cost bounded; very long pages are summarised to their main content.
- The model is instructed to use `null` for fields it can't find and never to invent data.

***

### Run it locally

```bash
npm install
export OPENAI_API_KEY=...
## Put your input in storage/key_value_stores/default/INPUT.json, then:
apify run
## or: npm run build && node dist/main.js
```

### License

ISC.

### Related actors

- [Argus — Autonomous Intelligence Agent](https://apify.com/fuze/argus) — Describe a mission in plain English; Argus plans what to watch, gathers from public
- [Nordic Public Tenders (Norway, Finland, Sweden)](https://apify.com/fuze/nordic-public-tenders) — Collects public procurement notices from Norway (Doffin), Finland (Hilma) and Sweden
- [AI Company Enrichment](https://apify.com/fuze/ai-company-enrichment) — Turn a list of domains into structured company profiles — firmographics, contacts

# Actor input Schema

## `urls` (type: `array`):

The web pages to extract from (one per line).

## `extractionPrompt` (type: `string`):

Describe, in plain English, the fields you want. E.g. 'product name, price, currency, in-stock (true/false), rating'.

## `schemaHint` (type: `string`):

Optional: give the exact field names / JSON shape you want back, so the output is consistent across pages.

## `maxResults` (type: `integer`):

Maximum number of pages to process. One pay-per-event charge per page successfully extracted.

## `maxConcurrency` (type: `integer`):

How many pages to process in parallel.

## `model` (type: `string`):

Override the extraction model (default gpt-4o-mini).

## Actor input object example

```json
{
  "urls": [
    "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
  ],
  "extractionPrompt": "Extract the product name, price, currency and whether it is in stock.",
  "maxResults": 100,
  "maxConcurrency": 5
}
```

# Actor output Schema

## `extractions` (type: `string`):

All extracted page records in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
    ],
    "extractionPrompt": "Extract the product name, price, currency and whether it is in stock."
};

// Run the Actor and wait for it to finish
const run = await client.actor("fuze/ai-web-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": ["https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"],
    "extractionPrompt": "Extract the product name, price, currency and whether it is in stock.",
}

# Run the Actor and wait for it to finish
run = client.actor("fuze/ai-web-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"
  ],
  "extractionPrompt": "Extract the product name, price, currency and whether it is in stock."
}' |
apify call fuze/ai-web-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fuze/ai-web-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/vClZkGG1bdectSXsF/builds/Za1smYwzaOGC7NFKG/openapi.json