AI Web Scraper — Extract Structured JSON from Web Pages ($4 / 1,000 Pages)

Extract structured JSON from any list of web page URLs using a plain-English prompt and an optional JSON Schema.

For each URL in startUrls, the actor fetches the HTML over plain HTTP, strips navigation, scripts and boilerplate while preserving headings, lists, links and tables, sends the cleaned page text to an AI language model, validates the returned JSON object against your schema, and writes one row per URL to the default dataset.

It is designed as a low-cost alternative to browser-heavy AI scrapers ($4 per 1,000 extracted pages vs. $30 per 1,000 pages), and you only pay for pages that return validated JSON.

Pricing (Pay Per Event)

Event	Price	When it is charged
page-extracted	$0.004 ($4 per 1,000 pages)	One web page fetched and turned into a validated JSON object in the dataset.
apify-actor-start	$0.00005 per GB	Platform start event charged by Apify at run start.

Failed pages are free. You are never charged page-extracted when:

• A URL is malformed, times out, or returns an HTTP error (such as 403, 404, 429 or 5xx).
• A URL points to non-HTML content (such as a PDF, image or JSON file).
• The model output fails JSON parsing or outputSchema validation after a repair retry.

To cap total spend on any run, set Apify's Maximum total charge (maxTotalChargeUsd) in the run options. Before each page's AI call, the actor checks that your remaining budget covers another page-extracted event and stops cleanly with stopReason: "budget_reached" in RUN_SUMMARY when the cap is reached.

Input

Field	Type	Required	Default	Description
startUrls	string[]	Yes	2 books.toscrape.com URLs	List of http:// or https:// web page URLs (max 1,000).
prompt	string	Yes	"Extract the book title, price and number in stock"	Plain-English instructions describing what to extract.
outputSchema	object	No	—	Optional JSON Schema the extracted JSON must follow. Checked keywords: type, required, properties, additionalProperties, items, enum, const, minimum/maximum (and exclusive), minLength/maxLength (Unicode characters), minItems/maxItems. Other keywords, including pattern (not evaluated, to avoid regex blow-ups), are ignored. Output that fails is retried once, then returned as a free error row.
maxPages	integer	No	100	Maximum number of URLs from startUrls to process (1–1000).
maxConcurrency	integer	No	2	Concurrent AI extraction requests (1–3).

Output

One row per processed URL is written to the default dataset:

{
  "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
  "finalUrl": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
  "status": "ok",
  "data": {
    "title": "A Light in the Attic",
    "price": "£51.77",
    "inStock": 22
  },
  "model": "glm-4.5-flash",
  "truncated": false,
  "error": null,
  "fetchedAt": "2026-10-24T12:00:00.000Z"
}

Dataset views:

• results — url, status, data, model
• errors — url, error

Run summary (RUN_SUMMARY in the default key-value store):

{
  "ok": 2,
  "error": 0,
  "errorByReason": {},
  "chargedEvents": 2,
  "modelsUsed": { "glm-4.5-flash": 2 },
  "retries429": 0,
  "budgetReached": false,
  "stopReason": null,
  "complete": true
}

Honest Limits & Speed

• Plain HTTP only: Pages are fetched with a plain HTTP GET request (20 s timeout, 5 MB max response body) and an honest ApifyAIWebScraper/0.1 User-Agent. There is no JavaScript rendering (no headless browser), no logins/cookies, and no CAPTCHA or bot-protection bypass. Sites that require client-side rendering or block automated HTTP requests will return an uncharged status: "error" row.
• 24,000-character page cap: Cleaned page text (headings, paragraphs, lists, link text and |-separated table rows) is truncated to 24,000 characters before calling the model; when truncated, the row sets "truncated": true.
• Speed and rate limits: Extraction runs on free-tier GLM flash models (glm-4.5-flash with automatic fallback to glm-4.7-flash when busy) at a concurrency of 1–3. Expect roughly 5–10 seconds per page. Under heavy upstream load, a small share of pages may still fail after exponential-backoff retries; those pages are reported as status: "error" and are not charged.
• AI accuracy: Extracted data is generated by an AI language model (glm-4.5-flash / glm-4.7-flash) and may contain mistakes or omissions. Always validate critical fields downstream.

Data handling

• Where your pages go: each page's cleaned text and your prompt are sent to Z.ai's API (GLM models, operated by a Singapore company). Z.ai's API data processing terms state that it does not store the content customers send or generate, and that data is generally processed in Singapore. If you need a different provider or data location, don't use this actor for that data.
• What the actor keeps: only your own run's dataset and run summary in your Apify account. Pages are fetched with an honest user agent, without logins, proxies or bot-protection bypass, and private or internal network addresses are refused.
• Model terms you inherit: Z.ai's terms prohibit using model output to train or improve competing AI models. Don't use this actor's output for that purpose.