# AI Extract: Structured Data from Any Web Page (`swillz/ai-extract`) Actor Extract structured JSON from any public web page with a plain-English prompt or a JSON schema. Input: URLs + what to extract. Output: one validated row per page (products, prices, articles, events, listings). Optional crawling. AI included, no API key needed. - **URL**: https://apify.com/swillz/ai-extract.md - **Developed by:** [William Jin](https://apify.com/swillz) (community) - **Categories:** AI, Developer tools, Open source - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $8.00 / 1,000 page extracted (ai included)s This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README **AI Extract** turns **any public web page into structured JSON** using a **plain-English prompt or a JSON schema**. Paste product, article, event, job or listing URLs (or crawl a whole section of a site), describe the fields you want, and get one clean row per page: `{"title": …, "price": 19.99, "inStock": true, …}`. Export to **CSV, JSON or Excel**, or call it from your app, Make, n8n, Zapier or an AI agent. **The AI is included, so you don't need an API key.** Each page is cleaned into compact markdown, sent to a fast, accurate model with **strict schema-constrained output**, and checked against your schema. AI Extract **respects robots.txt** and costs **$8 per 1,000 pages extracted**, or **$2 per 1,000 if you bring your own AI key**. Pages that fail are free. **When to use:** you need structured JSON from web pages that no dedicated scraper covers (products, prices, events, listings, articles). **Input:** URLs plus what to extract, e.g. `{"startUrls":[{"url":"https://example.com/product/1"}],"instructions":"name, price, currency, inStock"}`. **Output:** one row per page with the extracted JSON in `data`. AI included, no API key needed. From $8 per 1,000 pages ($2 with your own AI key). ### What can AI Extract do? - 🧠 **Prompt or schema:** write "product name, price as a number, currency and stock status", give a JSON Schema, or both. A shorthand such as `{"title":"string","price":"number","tags":"string[]"}` also works, including nested lists like `{"products":[{"name":"string","price":"number"}]}`. - βœ… **Validated output:** every answer is checked against your schema. If it doesn't match, AI Extract retries with the errors. Pages that still don't match are returned as `invalid_output` and **not charged**. - πŸ•ΈοΈ **URLs or crawl:** process a list of URLs, or start from a category page and follow links up to a crawl depth, with a page limit, same-site restriction and include/exclude URL patterns. - ⭐ **Reads what's hidden in HTML:** star ratings shown as icons or CSS classes, schema.org microdata (`ratingValue`, `priceCurrency`, `availability`) and icon-link labels are turned into text, so the AI can return them. - 🧹 **Clean markdown, fewer tokens:** scripts, navigation, footers, cookie banners and ads are stripped. You can narrow extraction to a CSS selector (`main`, `.product`) and get the markdown back for RAG. - πŸ”‘ **Bring your own key (optional):** OpenAI, Anthropic Claude, Google Gemini, DeepSeek, Groq, OpenRouter, Together or any OpenAI-compatible API. Your key is stored encrypted, and you can cap your AI spend per run. - πŸ›‘οΈ **Safe defaults:** robots.txt and crawl-delay are honored, there are no logins, and requests to the same site are spaced out. The AI is told to use only what's on the page, return `null` instead of guessing, and ignore instructions hidden in page text. ### What data can you extract? Anything that's visible in the page text. Typical examples: | Page type | Example fields | |---|---| | Product pages | name, brand, price, currency, availability, rating, SKU, specs | | Category / listing pages | `items: [{name, price, url}]` for every product on the page | | Articles and blogs | headline, author byline, published date, summary, tags | | Events | name, date, venue, city, price, ticket URL | | Job posts | title, company, location, salary range, remote, requirements | | Company pages | company name, what they do, pricing plans, integrations | | Documentation | API endpoints, parameters, version numbers | Each row has `url`, `status`, `title`, `data` (your fields), `error`, `truncated`, `inputTokens`, `outputTokens`, `model` and `extractedAt`. ### How to extract data from a website with AI (step by step) 1. Click **Try for free** and sign in to Apify. The free plan needs no card. 2. Paste your page URLs into **Start URLs**. 3. Describe what you want in **Instructions**, and/or paste a **JSON schema**. 4. Optional: set **Crawl depth** to 1+ to follow links, and use **Only follow URLs matching** (e.g. `/product/`) to stay on relevant pages. 5. Click **Start**, then open the **Overview** table or download CSV, Excel or JSON. ### How much does it cost? AI Extract uses **pay-per-result pricing**. You pay only for pages where data was successfully extracted and validated, plus a tiny $0.00005 per run start. The AI cost is included. | Apify plan | Built-in AI, per 1,000 pages | Your own AI key, per 1,000 pages | |---|---|---| | Free | $12 | $2 (+ your provider's tokens) | | Starter | $12 | $2 | | Scale | $10 | $2 | | Business and up | $8 | $2 | Prices drop on bigger Apify plans (Starter = Bronze tier, Scale = Silver, Business and up = Gold). Example: 10,000 pages with the built-in AI cost $120 on Starter, $100 on Scale or $80 on Business. Pages that fail to load, are blocked by robots.txt, are empty, fail AI extraction or don't match your schema are returned with a status and **not charged**. Apify's **free plan includes $5 of monthly credit**. On the free plan the built-in AI processes up to **25 pages per run** (add your own AI key, or use any paid Apify plan, for bigger runs). You can set a maximum cost per run, and the actor stops cleanly when it's reached. ### Input example ```json { "startUrls": [{ "url": "https://books.toscrape.com/catalogue/category/books/travel_2/index.html" }], "maxCrawlDepth": 1, "maxPages": 20, "includeUrlPatterns": ["/catalogue/[^/]+_\\d+/index\\.html"], "instructions": "Extract the book title, price as a number, currency code, whether it is in stock and the star rating (1-5).", "jsonSchema": { "title": "string", "price": "number", "currency": "string", "inStock": "boolean", "rating": "integer" } } ``` ### Output example ```json { "url": "https://books.toscrape.com/catalogue/its-only-the-himalayas_981/index.html", "loadedUrl": "https://books.toscrape.com/catalogue/its-only-the-himalayas_981/index.html", "depth": 1, "status": "ok", "title": "It's Only the Himalayas | Books to Scrape - Sandbox", "data": { "title": "It's Only the Himalayas", "price": 45.17, "currency": "GBP", "inStock": true, "rating": 2 }, "error": null, "truncated": false, "inputTokens": 1873, "outputTokens": 41, "model": "gpt-6-luna", "aiKey": "built-in", "extractedAt": "2026-10-23T18:05:12.000Z" } ``` A run summary is saved to the `OUTPUT` record in the key-value store. It includes pages extracted, status counts, tokens and AI cost. ### Use cases - **Ecommerce monitoring:** prices, stock and ratings from any store without writing a scraper per site. - **Lead and market research:** pricing plans, features and integrations from competitor and vendor pages. - **Content and RAG pipelines:** clean markdown plus structured metadata (title, date, summary) for every article. - **Directories and listings:** turn a listing page into `items: [...]` rows in one call. - **One-off data jobs:** when a site changes layout, your prompt still works and no selectors break. ### Use with AI agents (MCP, ChatGPT, Claude, Cursor, n8n) AI agents can find and run this Actor through the [Apify MCP server](https://mcp.apify.com). Add it to Claude Desktop, Cursor, VS Code or any MCP client: ```json {"mcpServers":{"apify":{"url":"https://mcp.apify.com?tools=swillz/ai-extract"}}} ``` Or run it locally: `npx -y @apify/actors-mcp-server --tools swillz/ai-extract` with `APIFY_TOKEN` set. In n8n or Make, use the Apify node with Actor `swillz/ai-extract`. Example prompts: - "Get the price and stock status of these 5 product pages as JSON" - "Extract all events (name, date, venue) from this page" - "Turn this pricing page into a table of plans and prices" Minimal input: ```json {"startUrls": [{"url": "https://example.com/product/1"}], "instructions": "product name, price, currency and whether it is in stock", "maxPages": 5} ``` **Keep it fast:** Keep `maxPages` at 10 or lower for answers within ~45 seconds; pass a `jsonSchema` when you need exact field names. The free plan processes up to 25 pages per run with the built-in AI, and says so in the run status. **Cap the cost:** pass `maxTotalChargeUsd` ("Maximum cost per run") when you call the Actor. The run stops cleanly at the cap, keeps the results so far, and says why in the status message and the OUTPUT record. ### Integrations and API - **Make, n8n and Zapier:** use the official Apify app or node, choose *Run Actor* and pick `swillz/ai-extract`. - **API:** `POST https://api.apify.com/v2/acts/swillz~ai-extract/run-sync-get-dataset-items?token=YOUR_TOKEN` with the input JSON returns the rows directly. JavaScript and Python clients are available. - **AI agents (MCP):** connect `https://mcp.apify.com?tools=swillz/ai-extract` to Claude, Cursor or any MCP client and ask, for example, "get the price and stock status from these 5 product pages". ### AI Extract vs alternatives | | AI Extract | Typical AI scraper actors | Hand-written scrapers | |---|---|---|---| | Price per 1,000 pages | $8-12 by plan (AI included), $2 with your key | $20-30 by plan | Your dev time | | Schema-validated JSON | βœ… with automatic retry | Varies | βœ… | | Works when layout changes | βœ… | βœ… | ❌ selectors break | | Bring your own AI key | βœ… 8 providers | Rarely | n/a | | Failed pages charged | ❌ free | Varies | n/a | ### FAQ **Do I need an OpenAI or other AI API key?** No. The built-in AI is included in the $8-12 per 1,000 pages price (lower on bigger Apify plans). Add your own key only if you want the lower $2 per 1,000 price or a specific model. **How accurate is it?** On clean product, article and event pages it is very accurate, because the model sees only the page's main content and answers are schema-constrained and validated. Fields that aren't on the page come back as `null`; the AI is instructed never to invent values. Pages longer than **Max input tokens per page** are trimmed (`truncated: true`), so use a **Content CSS selector** for very long pages. **Does it work on JavaScript-heavy sites?** AI Extract loads pages over fast HTTP without a browser. Sites that render everything in the browser can come back as `empty_page`; those pages are not charged. **Is it legal?** AI Extract reads only public, logged-out pages and respects robots.txt. Please don't use it to collect personal data about private individuals; the AI is also instructed to skip personal contact details. Make sure your use of the data complies with the site's terms and laws like GDPR and copyright law. **What do the statuses mean?** - `ok`: extracted and charged. - `http_error`: the page returned an error. A 403 usually means bot protection, so try enabling a proxy. - `fetch_error`: the page could not be reached. - `blocked_by_robots_txt`: the site disallows automated access. - `not_html` or `empty_page`: there was nothing to read. - `ai_error` or `invalid_output`: the AI failed or its answer didn't match your schema. - `skipped_cost_cap`: your spending limit was reached. Only `ok` rows are charged. **Found a bug or need a feature?** Open an issue on the **Issues** tab. We usually reply within 24 hours. ### Related tools More tools from the same developer, built to work together: - [Career Site Jobs Scraper](https://apify.com/swillz/career-site-jobs-scraper): jobs from Greenhouse, Lever, Ashby, Workable and Recruitee boards, with salary fields, as a jobs API or CSV. - [Workday Jobs Scraper](https://apify.com/swillz/workday-jobs-scraper): every job from any myworkdayjobs.com career site, past the 2,000-result cap. - [Company Website Enrichment](https://apify.com/swillz/company-website-enrichment): turn a list of domains into company name, emails, phones, social links and tech stack. - [Google Sheets Import & Export](https://apify.com/swillz/google-sheets-import-export): send any Apify dataset to a Google Sheet (append, replace or dedupe) or read a sheet as JSON. - [XML Sitemap URL Extractor & Sitemap to CSV](https://apify.com/swillz/sitemap-url-extractor): every URL from a website's sitemap, with lastmod, HTTP status and new-page monitoring. - [AI Brand Visibility Tracker](https://apify.com/swillz/ai-brand-visibility-tracker): track whether ChatGPT mentions, ranks and cites your brand vs competitors (GEO). - [Email Provider Detector](https://apify.com/swillz/email-provider-detector): bulk check whether domains use Google Workspace, Microsoft 365, Proofpoint or Mimecast, plus SPF, DMARC and DKIM. - [Hiring Signals by Company Domain](https://apify.com/swillz/hiring-signals-by-domain): from a list of company domains, find which ATS each uses and how many open roles, by department and location. - [EU VAT Number Validator (VIES)](https://apify.com/swillz/eu-vat-validator): bulk check EU VAT numbers against official VIES, with company name and address. - [EORI Number Validator](https://apify.com/swillz/eori-validator): bulk check EU and UK EORI numbers for customs, with trader name where published. - [LEI Lookup (GLEIF)](https://apify.com/swillz/lei-lookup): find a company's Legal Entity Identifier, legal name, address, status and parent from official GLEIF data. - [Sanctions Screener](https://apify.com/swillz/sanctions-screener): screen company and person names against official OFAC, UN, EU and UK sanctions lists with fuzzy matching. **Popular workflows** - **Catalog or content extraction:** list every product or article URL with [XML Sitemap URL Extractor & Sitemap to CSV](https://apify.com/swillz/sitemap-url-extractor), turn each page into JSON with this Actor, and export with [Google Sheets Import & Export](https://apify.com/swillz/google-sheets-import-export). Source code: https://github.com/williamjin02/apify-actors/tree/main/actors/ai-extract # Actor input Schema ## `startUrls` (type: `array`): Required. Array of objects, e.g. \[{"url":"https://example.com/product/1"}]. Each page is read and turned into one JSON row by AI. ## `instructions` (type: `string`): What to extract, in plain words, e.g. "product name, price, currency and whether it is in stock". Give instructions, a jsonSchema, or both; the run stops with a clear error if both are empty. ## `jsonSchema` (type: `object`): Optional. A JSON Schema for the 'data' field of each row, or a shorthand like {"title":"string","price":"number","tags":"string\[]"}. Output is validated against it; pages that don't match are retried, then returned as invalid_output and not charged. ## `maxCrawlDepth` (type: `integer`): 0 = only the start URLs. 1 = also pages linked from them, and so on. ## `maxPages` (type: `integer`): Stop after this many pages (default 20). AI agents: keep ≀10 for answers within ~45s. The free plan with the built-in AI processes up to 25 pages per run. ## `sameDomainOnly` (type: `boolean`): Only follow links to the start URLs' domains. ## `includeUrlPatterns` (type: `array`): Regular expressions; when set, crawled links must match at least one (e.g. /catalogue/). Start URLs are always processed. ## `excludeUrlPatterns` (type: `array`): Regular expressions for links to skip (e.g. /login|/cart). ## `contentSelector` (type: `string`): Optional. Send only this part of the page to the AI (e.g. main, #content, .product). Saves tokens and improves accuracy. ## `removeSelectors` (type: `string`): Optional comma-separated selectors to strip before extraction (e.g. .reviews, .related-products). Navigation, footers, scripts, cookie banners and ads are removed automatically. ## `keepHeaderFooter` (type: `boolean`): Turn on if the data you need lives in the header or footer (e.g. company address). ## `includeMarkdown` (type: `boolean`): Add the cleaned markdown sent to the AI to each row (useful for RAG or debugging). ## `maxInputTokensPerPage` (type: `integer`): Longer pages are trimmed to this size (β‰ˆ4 characters per token). Built-in AI: up to 16,000. ## `maxOutputTokensPerPage` (type: `integer`): Upper limit on the AI answer per page. Raise for pages with long lists. Built-in AI: up to 4,000. ## `llmApiKey` (type: `string`): Leave empty to use the built-in AI (included in the price). Add your own OpenAI, Anthropic, Gemini, DeepSeek, Groq, OpenRouter or Together key to pay the provider directly and get the lower own-key price. Stored encrypted. ## `llmProvider` (type: `string`): Provider of your own llmApiKey (ignored when using the built-in AI): openai, anthropic, gemini, deepseek, groq, openrouter, together, or openai-compatible with llmBaseUrl. ## `llmModel` (type: `string`): Optional. Defaults: OpenAI gpt-6-luna, Anthropic claude-haiku-5-5, Gemini gemini-3.1-flash-lite, DeepSeek deepseek-flash, Groq openai/gpt-oss-20b. ## `llmBaseUrl` (type: `string`): Only for 'Other OpenAI-compatible API', e.g. https://api.example.com/v1 ## `maxAiCostUsd` (type: `number`): Optional. Stops the run once the estimated cost on your own key reaches this amount. ## `maxConcurrency` (type: `integer`): Pages processed at the same time. Requests to one website are spaced out politely regardless. ## `maxRetries` (type: `integer`): Retries for failed page loads and AI calls. ## `requestTimeoutSecs` (type: `integer`): Seconds to wait for each page to load before giving up (default 30). ## `proxyConfiguration` (type: `object`): Optional. Use Apify Proxy if a site blocks datacenter requests. ## Actor input object example ```json { "startUrls": [ { "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" } ], "instructions": "Extract the book title, price as a number, currency code, whether it is in stock, star rating (1-5) and the product description.", "jsonSchema": { "title": "string", "price": "number", "currency": "string", "inStock": "boolean", "rating": "integer", "description": "string" }, "maxCrawlDepth": 0, "maxPages": 20, "sameDomainOnly": true, "includeUrlPatterns": [], "excludeUrlPatterns": [], "keepHeaderFooter": false, "includeMarkdown": false, "maxInputTokensPerPage": 8000, "maxOutputTokensPerPage": 1500, "llmProvider": "openai", "maxConcurrency": 5, "maxRetries": 2, "requestTimeoutSecs": 30, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `results` (type: `string`): Example: {"url":"https://example.com/product/1","status":"ok","title":"Blue Mug","data":{"name":"Blue Mug","price":12.5,"currency":"USD","inStock":true}}. Failed pages are free rows with status (fetch_error, ai_error…) and error. ## `summary` (type: `string`): JSON: pagesProcessed, extracted, charged, byStatus, truncated, freePlanLimited, notice, zeroResultsReason, stoppedBecause and token/AI-cost stats. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" } ], "instructions": "Extract the book title, price as a number, currency code, whether it is in stock, star rating (1-5) and the product description.", "jsonSchema": { "title": "string", "price": "number", "currency": "string", "inStock": "boolean", "rating": "integer", "description": "string" } }; // Run the Actor and wait for it to finish const run = await client.actor("swillz/ai-extract").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" }], "instructions": "Extract the book title, price as a number, currency code, whether it is in stock, star rating (1-5) and the product description.", "jsonSchema": { "title": "string", "price": "number", "currency": "string", "inStock": "boolean", "rating": "integer", "description": "string", }, } # Run the Actor and wait for it to finish run = client.actor("swillz/ai-extract").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" } ], "instructions": "Extract the book title, price as a number, currency code, whether it is in stock, star rating (1-5) and the product description.", "jsonSchema": { "title": "string", "price": "number", "currency": "string", "inStock": "boolean", "rating": "integer", "description": "string" } }' | apify call swillz/ai-extract --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,swillz/ai-extract" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/snpojAsSzzCiuSkOO/builds/dVITQBj29Al5WT7fm/openapi.json