# AI Web Scraper: Extract Data with a Prompt (`bulkleads/ai-web-scraper`) Actor

AI web scraper: name the fields you want in plain words and get structured JSON back from any web page. No CSS selectors or code, LLM tokens included. $19 per 1,000 pages, failed pages are free.

- **URL**: https://apify.com/bulkleads/ai-web-scraper.md
- **Developed by:** [bulkleads](https://apify.com/bulkleads) (community)
- **Categories:** AI, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

$19.00 / 1,000 page extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

**AI Web Scraper** reads any web page and returns the fields you ask for as JSON. Name the fields (prices, plan names, phone, team members), add plain-English instructions if you like, and the AI pulls them from the page. AI tokens are included in the price: **$19 per 1,000 pages**, and pages that fail to load are free.

### How to use it

1. Add **Page URLs**, one per line.
2. List the **Fields to extract**, for example `plan_names`, `monthly_prices`, `phone`.
3. Optionally add **Instructions**: "prices in USD only", "list every person on the team page with their role".
4. Click **Start**. Each page becomes one item with its fields.

### Output example

```json
{
  "url": "https://example.com/pricing",
  "title": "Pricing",
  "ok": true,
  "data": {
    "plan_names": ["Starter", "Pro", "Business"],
    "monthly_prices": ["$19", "$49", "$199"]
  }
}
```

### What can it extract?

Anything printed on the page: prices, product specs, contact details, people and roles, opening hours, job listings, event dates. A field the page doesn't show comes back null, so the AI never fills a gap with a guess.

### Run it from the API or an AI agent

Start runs from your own code with the Apify API, from Make, Zapier or n8n through their Apify integrations, or as a tool in the Apify MCP server so Claude, ChatGPT or Cursor can call it. The input is the same everywhere: `startUrls`, `fields` and `instructions`. Each page comes back as one dataset item with `url`, `title`, `ok`, `error` and `data`.

### How much does it cost?

**$19 per 1,000 pages**, AI included. No OpenAI key needed. A page that can't be loaded is not charged.

### FAQ

**Does it follow links?** It reads the pages you give it. Pair it with [Web Search for AI Agents](https://apify.com/bulkleads/web-search-for-ai-agents) to find pages first.

**Can it read pages behind a login?** No, only public pages.

### Related actors

Get emails and phones from websites with [Contact Details Scraper](https://apify.com/bulkleads/website-contact-scraper).

# Actor input Schema

## `startUrls` (type: `array`):

Pages to read, one per line.

## `fields` (type: `array`):

Field names, e.g. plan_names, prices, phone.

## `instructions` (type: `string`):

Anything else the AI should know, in plain words.

## Actor input object example

```json
{
  "startUrls": [
    "https://en.wikipedia.org/wiki/Web_scraping"
  ],
  "fields": [
    "topic",
    "one_sentence_summary",
    "techniques"
  ]
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://en.wikipedia.org/wiki/Web_scraping"
    ],
    "fields": [
        "topic",
        "one_sentence_summary",
        "techniques"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("bulkleads/ai-web-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://en.wikipedia.org/wiki/Web_scraping"],
    "fields": [
        "topic",
        "one_sentence_summary",
        "techniques",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("bulkleads/ai-web-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://en.wikipedia.org/wiki/Web_scraping"
  ],
  "fields": [
    "topic",
    "one_sentence_summary",
    "techniques"
  ]
}' |
apify call bulkleads/ai-web-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,bulkleads/ai-web-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ovzWprUzDynGyCOth/builds/tY1qhQFW4c1GW2UwV/openapi.json