AI Web Scraper: any website to structured JSON with a prompt (products, prices, listings, contacts)

Describe the data you want in plain words, or list the fields, and get it from any web page as clean JSON rows: products and prices, listings, directories, contacts, articles, tables. No selectors, no code: the AI reads the page the way a person does.

Pages built by JavaScript open in a real Chrome when their HTML is empty. Crawl mode follows the links of a site that best match your prompt. With no prompt, every page comes back as clean Markdown for your own LLM.

What you get

Field	What it holds
Item rows	the fields your prompt asks for (or the ones you list), with url, pageTitle and itemIndex; numbers as numbers, prices with a separate currency, links as full URLs
One row per page (optional)	items (all items of the page) and itemCount
Markdown (optional)	markdown: the page's main content, for your own model
SUMMARY	pages and items per start URL, notes, the model used, pages opened in Chrome, what was charged

What people use it for

• Product and price lists: every product on a category page with price, rating and stock
• Directories and listings: companies, jobs, properties or events from any list page
• Research tables: the facts you name from articles, reports and documentation
• Feeding an LLM: pages as clean Markdown, without menus or cookie banners

How to use it

1. Add one or more start URLs.
2. Write the prompt, e.g. Every product: name, price, rating and link, or list the fields.
3. Pick single pages or crawl (the most relevant pages of each site), then click Start.

Input

Field	Example	Notes
Start URLs	https://www.example.com/products	Pages to read (crawl mode starts there)
Prompt	Every product: name, price, rating and link	Empty: each page as Markdown
Fields	name, price, rating	Optional: the exact columns
Mode	single or crawl	Crawl follows the links that match the prompt
Most pages per site	10	Crawl mode
Model	fast	Gemini Flash; smart (GPT) and claude read harder pages
JavaScript pages	auto	Chrome only when a page's HTML is empty
One row per item	on	Off: one row per page with an items list

Output

Each result is one row in the run's dataset (download it as JSON, CSV, Excel, XML or HTML, or read it through the API). An example row (values shortened):

{
  "url": "https://shop.example.com/catalogue/category/travel/",
  "pageTitle": "Travel | Example Bookshop",
  "itemIndex": 1,
  "title": "It's Only the Himalayas",
  "price": 45.17,
  "currency": "£",
  "starRating": "Two",
  "availability": "In stock",
  "link": "https://shop.example.com/catalogue/its-only-the-himalayas/",
  "scrapedAt": "2026-10-17T19:13:00Z"
}

From a run on 17 October 2026 on a public demo bookshop (address replaced): two category pages gave 11 and 19 books with title, price, currency, star rating, availability and link, with the same field names on both pages, for $0.015 of AI usage. A crawl of a software project's site found its latest release, the release date and the link to the release notes in 4 pages. A record named SUMMARY in the run's key-value store lists what each start URL returned and what was charged.

Pricing

• $0.008 per page read and extracted with your prompt (or returned as Markdown)
• Free: pages that fail to load or have no readable content
• The AI model that reads the page (Google's Gemini Flash by default) is billed by Apify's AI gateway at its exact cost: about $0.006 to $0.010 a page on catalogue pages of 11 to 19 items (measured 17 October 2026); GPT and Claude cost more; Apify charges free-plan accounts 10 times the AI usage

What typical jobs cost at the list price:

Job	Cost
One catalogue page	$0.008 + about $0.008 AI
100 product pages	$0.80 + about $0.80 AI
Crawl a site for 10 pages	$0.08 + about $0.08 AI
1,000 pages as Markdown (no prompt)	$8, no AI usage

On Apify's Scale and Business plans the per-event prices are lower (Store discounts). Set Maximum cost per run and the Actor stops cleanly when it is reached: you never pay more than that.

Tips

• List the fields when you need the same columns on every run; without them the first page's field names are reused on the next pages.
• Say what one item is (every product, each job ad) and the AI returns one row per item.
• In crawl mode the words of your prompt choose which links are followed: name what you are looking for.
• Leave the prompt empty to get pages as Markdown for your own model, or use the LLM Website Crawler for whole sites at $1 per 1,000 pages.

Use it through the API

Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("zaiq/ai-web-scraper").call(run_input={"startUrls": ["https://www.example.com/products"], "prompt": "Every product: name, price, rating and link"})
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(row)

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('zaiq/ai-web-scraper').call({"startUrls": ["https://www.example.com/products"], "prompt": "Every product: name, price, rating and link"});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

HTTP (one call, results back)

curl -X POST 'https://api.apify.com/v2/acts/zaiq~ai-web-scraper/run-sync-get-dataset-items?token=<YOUR_APIFY_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{"startUrls": ["https://www.example.com/products"], "prompt": "Every product: name, price, rating and link"}'

Integrations and schedules

• Schedules: run it every hour, day or week from the Apify Console (Schedules) and keep a history of results.
• Make, Zapier, n8n, Google Sheets, Slack, Airtable: connect it through Apify's integrations, or send each finished run to a webhook.
• Datasets: results stay in the run's dataset; export them or read them by API whenever you need them.

Use it with AI agents (MCP)

Add Apify's MCP server (https://mcp.apify.com) to Claude, ChatGPT, Cursor, VS Code or any MCP client. The agent can find this Actor and run it for you: ask it for "get every product with its price and rating from this page". AI agents can also pay for runs directly through Apify's agentic payments.

Limits

• Long pages are read in pieces of 20,000 characters, up to six (120,000 characters); items beyond that are not read.
• The AI reads what the page shows: a value that is only in an image, or behind a click, is not seen.
• Pages behind a login, CAPTCHAs and paywalls are not read.
• Like any AI, it can misread a value: check a sample before relying on a new site.

FAQ

Do I need an OpenAI or Google key?

No. Apify's AI gateway runs the model and bills its exact cost to your Apify account.

Can it read JavaScript sites?

Yes: a page whose HTML has no content opens in Chrome (JavaScript pages: auto), or always with that setting.

Will the columns be the same every time?

List the fields and they are. Without fields, the AI chooses short camelCase names and keeps the first page's names for the rest of the run.

Can it follow pagination?

Crawl mode follows the links that match your prompt; add each page you need as a start URL for full control.

Related Actors from Zaiq

• llm-website-crawler: whole sites as Markdown for LLMs
• bulk-ai-runner: an AI prompt on every row of the results
• company-research-agent: company profiles from websites
• ai-answers-api: answers from five AI assistants

Support

Questions, a site that changed, or a field you need? Open an issue on this Actor's Issues tab: we answer there, usually within a day.

---

Also found as: ai web scraper, llm scraper, gpt scraper, chatgpt scraper, scrape website with ai, website to json, ai data extraction, prompt scraper, no code web scraper.