AI Web Scraper: JSON Schema Extractor
Pricing
from $2.10 / 1,000 page extracted (your own llm key)s
AI web scraper with strict JSON Schema extraction: give URLs and a schema, get validated JSON per page. HTTP-first, so far cheaper than browser scrapers. Bring your own LLM key at $0.003 per page, failed pages are never charged.
You can access the AI Web Scraper: JSON Schema Extractor programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.
{ "openapi": "3.0.1", "info": { "version": "0.0", "x-build-id": "becTbvPBJDQ9c8Ntk" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/tiger19935~schema-ai-extractor/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-tiger19935-schema-ai-extractor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/tiger19935~schema-ai-extractor/runs": { "post": { "operationId": "runs-sync-tiger19935-schema-ai-extractor", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/tiger19935~schema-ai-extractor/run-sync": { "post": { "operationId": "run-sync-tiger19935-schema-ai-extractor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "urls": { "title": "Page URLs", "type": "array", "description": "Pages to extract data from. Static or server-rendered HTML pages work; pages that need JavaScript to show content are reported as 'needs-javascript'. If 'Follow links' is on, these are listing pages and detail pages are extracted instead.", "items": { "type": "string" } }, "schema": { "title": "JSON Schema", "type": "object", "description": "JSON Schema (draft-07 style) of the object to extract, with type 'object' at the top. Property descriptions guide the model. Output is validated against it. Leave empty to use 'Fields' instead." }, "fields": { "title": "Fields (simple alternative to a schema)", "type": "array", "description": "Used only when 'JSON Schema' is empty. One field per line: 'name', 'name: type' (string, number, integer, boolean, string[], number[]) and optionally ' - description'. Example: 'price: number - price without currency symbol'. All fields are required but nullable.", "items": { "type": "string" } }, "instructions": { "title": "Extra instructions", "type": "string", "description": "Optional plain-language hints for the model, for example 'Prices are in EUR; ignore sponsored items'. Max 2,000 characters." }, "llmProvider": { "title": "LLM provider", "enum": [ "openai-compatible", "anthropic", "gemini" ], "type": "string", "description": "Whose API your key belongs to. 'OpenAI-compatible' covers OpenAI, OpenRouter (set Base URL to https://openrouter.ai/api/v1), Groq, Together, Mistral and self-hosted servers. Ignored when billing mode is 'inclusive'.", "default": "openai-compatible" }, "apiKey": { "title": "API key (your own, BYOK)", "type": "string", "description": "Your LLM provider API key. Tokens are billed by your provider, not by this Actor. Stored as a secret and never logged. Empty = preview mode (no LLM call, no charge)." }, "baseUrl": { "title": "Base URL (OpenAI-compatible only)", "type": "string", "description": "API base URL. Leave empty for OpenAI (https://api.openai.com/v1). For OpenRouter use https://openrouter.ai/api/v1. A custom Base URL needs an explicit 'Model'." }, "modelTier": { "title": "Model tier", "enum": [ "fast", "accurate" ], "type": "string", "description": "'fast' = cheapest small model (default: gpt-4.1-mini, claude-haiku-4-5, gemini-2.5-flash-lite). 'accurate' = stronger model (gpt-4.1, claude-sonnet-4-5, gemini-2.5-pro) for messy pages and complex schemas. Overridden by 'Model'.", "default": "fast" }, "model": { "title": "Model (optional override)", "type": "string", "description": "Exact model name to use instead of the tier default, for example 'google/gemini-2.5-flash' on OpenRouter. Required with a custom Base URL." }, "billingMode": { "title": "Billing mode", "enum": [ "byok", "inclusive" ], "type": "string", "description": "'byok' (default): you use your own API key; this Actor charges the lower 'page-passthrough' event and your provider bills the tokens. 'inclusive': no key needed, LLM cost included in the higher 'page-inclusive' event; uses the fast tier only and is available only if the Actor owner enabled it.", "default": "byok" }, "maxRetries": { "title": "Validation retries", "minimum": 0, "maximum": 3, "type": "integer", "description": "How many times to send the validation errors back to the model when its JSON does not match the schema (0 to 3). Each retry costs tokens. Pages that still fail are not charged.", "default": 2 }, "maxItems": { "title": "Max extracted pages", "minimum": 1, "maximum": 100000, "type": "integer", "description": "Stop after this many successfully extracted pages. Also the upper bound of what the run can charge.", "default": 20 }, "followLinks": { "title": "Follow links (listing to detail pages)", "type": "boolean", "description": "Treat each URL as a listing page: find links to detail pages and extract those instead (the listing itself is not extracted or charged).", "default": false }, "detailLinkSelector": { "title": "Detail link CSS selector", "type": "string", "description": "Only with 'Follow links'. CSS selector of the links (or of containers holding them), e.g. 'article.product_pod h3 a'. Empty = automatic detection of the main group of similar links." }, "detailUrlPattern": { "title": "Detail URL pattern (regex)", "type": "string", "description": "Only with 'Follow links'. Regular expression a detail URL must match, e.g. '/catalogue/.+_\\d+/'. Empty = no filter." }, "maxDetailPagesPerListing": { "title": "Max detail pages per listing", "minimum": 1, "maximum": 500, "type": "integer", "description": "Hard limit of detail pages followed from each listing page.", "default": 10 }, "maxPages": { "title": "Max pages fetched in total", "minimum": 1, "maximum": 100000, "type": "integer", "description": "Hard limit of all HTTP page fetches in the run (listings and detail pages), including pages that fail.", "default": 50 }, "contentSelector": { "title": "Content CSS selector (optional)", "type": "string", "description": "Restrict the text sent to the model to this part of the page, e.g. 'main' or '#product'. Smaller input = fewer tokens and better accuracy. Empty = whole page without scripts, navigation and footer." }, "includeImages": { "title": "Keep image URLs", "type": "boolean", "description": "Keep image links in the Markdown. Turned on automatically when the schema mentions image, photo, thumbnail, picture or logo.", "default": false }, "includeMarkdown": { "title": "Include page Markdown in output", "type": "boolean", "description": "Add the cleaned Markdown sent to the model to each dataset item (useful for debugging, makes items larger).", "default": false }, "maxInputChars": { "title": "Max page characters sent to the model", "minimum": 4000, "maximum": 400000, "type": "integer", "description": "Page Markdown longer than this is cut (roughly 4 characters per token). Lower = cheaper. Inclusive mode caps this at 40,000.", "default": 60000 }, "maxConcurrency": { "title": "Parallel pages", "minimum": 1, "maximum": 10, "type": "integer", "description": "Pages processed at the same time. Lower it if your LLM provider rate-limits you.", "default": 3 }, "requestTimeoutSecs": { "title": "Page request timeout (seconds)", "minimum": 5, "maximum": 120, "type": "integer", "description": "Timeout for each page download.", "default": 30 }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Proxy for downloading pages. Default is Apify datacenter proxy. Sites that block datacenter IPs may need the residential fallback below.", "default": { "useApifyProxy": true } }, "useResidentialFallback": { "title": "Residential proxy fallback", "type": "boolean", "description": "Retry blocked pages (HTTP 403/429) through Apify residential proxy. Needs residential proxy access on your Apify plan and costs extra proxy traffic.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 }, "PROXY_UNBLOCKER_UNITS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 }, "PROXY_UNBLOCKER_UNITS": { "type": "integer", "example": 0 } } } } } } } } }}OpenAPI is a standard for designing and describing RESTful APIs, allowing developers to define API structure, endpoints, and data formats in a machine-readable way. It simplifies API development, integration, and documentation.
OpenAPI is effective when used with AI agents and GPTs by standardizing how these systems interact with various APIs, for reliable integrations and efficient communication.
By defining machine-readable API specifications, OpenAPI allows AI models like GPTs to understand and use varied data sources, improving accuracy. This accelerates development, reduces errors, and provides context-aware responses, making OpenAPI a core component for AI applications.
You can download the OpenAPI definitions for AI Web Scraper: JSON Schema Extractor from the options below:
If you’d like to learn more about how OpenAPI powers GPTs, read our blog post.
You can also check out our other API clients: