Pricing

from $2.49 / 1,000 results

Try for free

Go to Apify Store

AI Web Scraper (Any Website)

Try for free

AI Web Scraper (Any Website) SD - AI Web Scraper (Any Website) is an AI data extraction tool that extracts structured JSON from any public web page with a plain-English prompt: one row per page or per item, pagination and crawling, JS pages, Markdown and RAG chunks. Free LLM or your own key.

Pricing

from $2.49 / 1,000 results

Rating

0.0

(0)

Developer

Neuro Scraper

Actor stats

Bookmarked

Total users

Monthly active users

11 days ago

Last modified

Categories

Automation

Developer tools

Start URLs

startUrls

Required

Pages to extract data from, e.g. https://books.toscrape.com/ or a product, article or listing page. Any public http(s) page; no login.

Type:array

Page extraction prompt

prompt

Optional

What to extract, in plain English, e.g. "Extract every product: name, price in USD, rating and product URL. Use null for fields the page does not show." Runs on every page. Empty prompt and no schema = the page is returned as Markdown (no AI call).

Type:string

Instructions (same as the prompt)

instructions

Optional

Alternative name for 'Page extraction prompt' (used by GPT scraper tasks). When both are given they are joined.

Type:string

Output

outputMode

Optional

'One row per item' turns every product / post / job / row the prompt describes into its own result row. 'Free-text answer' returns the model's text in 'answer'. Markdown and chunks need no AI model.

Type:string

Default:page

Options:

pageitemsanswermarkdownchunks

Output schema (optional)

outputSchema

Optional

A JSON Schema the answer must follow, e.g. {"type": "object", "properties": {"name": {"type": "string"}, "price": {"type": "number"}}}. For 'One row per item' give the schema of ONE item. OpenAI and OpenRouter get it as structured output; other models get it in the prompt.

Type:object

Output fields (optional)

outputFields

Optional

Simpler than a schema: the keys every answer must use, e.g. title, price, availability, url.

Type:array

Field notes (optional)

schemaDescription

Optional

Extra notes about the fields, e.g. "price without currency symbol; date as YYYY-MM-DD".

Type:string

Extraction mode

extractionMode

Optional

Single: each start URL plus its 'next page' links up to 'Max pages'. Crawl: follows links of the start URL's site (link selector, globs, depth). Scout / Agentic: reads the site's sitemap (or the start page's links), lets the AI pick the URLs that match the prompt, then returns their Markdown (scout) or extracts data from them (agentic).

Type:string

Default:single

Options:

singlecrawlscoutagentic

Max pages per start URL

maxPages

Optional

Pages read per start URL: the start page and its next pages in single mode (the page's own 'next' link is followed), pages extracted in crawl / scout / agentic mode.

Type:integer

Minimum:1

Maximum:10000

Default:1

Max results

maxItems

Optional

Stops after this many result rows in total (pages, items or chunks).

Type:integer

Minimum:1

Maximum:1000000

Default:100

Max crawl depth

maxCrawlDepth

Optional

Crawl mode: how many links away from the start URL the crawler goes (0 = start page only).

Type:integer

Minimum:0

Maximum:100

Default:1

Next-page selector (optional)

paginationSelector

Optional

CSS selector of the 'next page' link, e.g. li.next a. Empty = found automatically (rel=next, or a link labelled Next / › / »).

Type:string

Link selector (crawl)

linkSelector

Optional

Crawl mode: CSS selector of the links to follow, e.g. article h3 a. Empty = every link on the page inside the start URL's site.

Type:string

Include URLs (globs)

includeUrlGlobs

Optional

Crawl / scout / agentic: only follow URLs matching one of these globs, e.g. https://books.toscrape.com/catalogue/**. ** = anything, * = anything but /.

Type:array

Exclude URLs (globs)

excludeUrlGlobs

Optional

Never follow URLs matching these globs, e.g. /login, /cart.

Type:array

Pages without AI (globs)

skipLlmGlobs

Optional

Crawl mode: pages matching these globs are only used to find links - no AI call, no result (and no charge), e.g. category pages /category/.

Type:array

Content selector (optional)

targetSelector

Optional

CSS selector of the part of the page sent to the AI, e.g. main, #content or .product-list. Empty = the whole page minus navigation, header, footer and cookie banners.

Type:string

Remove elements (CSS selector)

removeElementsCssSelector

Optional

Elements removed before the AI reads the page, e.g. .ads, .related-posts, #comments.

Type:string

Page content

contentScope

Optional

'Main content only' uses an article extractor (trafilatura) and may drop product grids; keep 'Whole page' for listings.

Type:string

Default:page

Options:

pagemain

Page format sent to the AI

pageFormatInRequest

Optional

Markdown is compact; cleaned HTML keeps short class names and attributes some values live in.

Type:string

Default:markdown

Options:

markdownhtmltext

Remove link URLs

removeLinkUrls

Optional

Drops link URLs from the Markdown sent to the AI (fewer tokens, but the AI can no longer return item URLs).

Type:boolean

Default:false

Max page characters sent to the AI

maxContentChars

Optional

Longer pages are cut at this length (about 4 characters per token). Raise it for long pages and large-context models.

Type:integer

Minimum:1000

Maximum:2000000

Default:40000

Add the page Markdown to each row

includeMarkdown

Optional

Also saves the page content that was sent to the AI in 'markdown' (always on in Markdown and Scout output).

Type:boolean

Default:false

JavaScript rendering

renderJavaScript

Optional

Auto loads each page over HTTP first and opens it in headless Chromium only when the HTML has (almost) no text, or when a browser-only option below is set.

Type:string

Default:auto

Options:

autoalwaysnever

Wait for dynamic content (seconds)

dynamicContentWaitSecs

Optional

Browser: extra seconds to wait after the page loads.

Type:integer

Minimum:0

Maximum:60

Default:0

Wait for selector (optional)

waitForSelector

Optional

Browser: wait until this CSS selector appears (max 20 s), e.g. .product-card.

Type:string

Infinite scroll

hasInfiniteScroll

Optional

Browser: scroll to the bottom until no new content loads (up to 'Max scrolls') before reading the page.

Type:boolean

Default:false

Max scrolls

maxScrolls

Optional

Browser: upper limit of scrolls per page when 'Infinite scroll' is on.

Type:integer

Minimum:1

Maximum:200

Default:10

Accept cookie banners

handleCookieConsent

Optional

Browser: clicks a visible 'Accept' / 'Accept all' / 'I agree' button of a cookie banner. In HTTP mode cookie banners are removed from the content.

Type:boolean

Default:true

Device

deviceType

Optional

Browser: desktop or a phone (Playwright's device emulation: screen, user agent, touch). Mobile pages are often lighter.

Type:string

Default:desktop

Options:

desktopmobile

Phone model (mobile)

mobileDeviceModel

Optional

Browser, mobile: a Playwright device name, e.g. iPhone 13, Pixel 7, Galaxy S9+. Empty = the newest iPhone in Playwright's list.

Type:string

Viewport width (optional)

viewportWidth

Optional

Browser: custom window width in pixels (default 1366 desktop / the phone's own).

Type:integer

Minimum:200

Maximum:4000

Viewport height (optional)

viewportHeight

Optional

Browser: custom window height in pixels (default 900 desktop / the phone's own).

Type:integer

Minimum:200

Maximum:4000

Save screenshots

saveScreenshots

Optional

Browser: saves a full-page PNG of every page in the key-value store; the link is in 'screenshotUrl'. Turns the browser on.

Type:boolean

Default:false

AI model provider

llmProvider

Optional

Free uses Pollinations.ai's anonymous text API: no key, a small open model (GPT-OSS 20B on 2026-10-09), about one AI call every 20-30 seconds (its anonymous limit), answers up to about 1,500 tokens (about 20-30 items per page); the page text is sent to that service. With your own key the run is much faster, handles long pages, and the AI cost is billed by that provider to you.

Type:string

Default:free

Options:

freeopenaianthropicgeminiopenroutercustom

AI API key

llmApiKey

Optional

Your API key for the chosen provider (OpenAI sk-..., Anthropic sk-ant-..., Gemini AIza..., OpenRouter sk-or-...). Not needed for Free.

Type:string

Model (optional)

model

Optional

Model id, e.g. gpt-4.1-mini, claude-haiku-4-5, gemini-2.5-flash, openai/gpt-4o-mini. Empty = picked at run time from the provider's own model list (a fast, low-cost model).

Type:string

Base URL (OpenAI-compatible)

llmBaseUrl

Optional

Only for 'Any OpenAI-compatible API': the base URL ending in /v1, e.g. https://api.groq.com/openai/v1, https://api.deepseek.com/v1, https://api.mistral.ai/v1.

Type:string

Temperature

temperature

Optional

0 = most deterministic (best for extraction). Models that do not take it get the request without it.

Type:string

Default:0

Top P (optional)

topP

Optional

Nucleus sampling, e.g. 1. Empty = the model's default.

Type:string

Frequency penalty (optional)

frequencyPenalty

Optional

OpenAI-format models only, e.g. 0. Empty = the model's default.

Type:string

Presence penalty (optional)

presencePenalty

Optional

OpenAI-format models only, e.g. 0. Empty = the model's default.

Type:string

Max answer tokens (optional)

maxOutputTokens

Optional

Upper limit of the answer length per page. Empty = the model's default (Anthropic: 4096).

Type:integer

Minimum:16

Maximum:200000

Reasoning effort (reasoning models)

reasoningEffort

Optional

OpenAI-format reasoning models (gpt-5, o-series, GPT-OSS): how long the model thinks before answering. Extraction rarely needs more than low. Models that do not take it get the request without it.

Type:string

Default:

Options:

minimallowmediumhigh

Chunk size (tokens)

chunkSize

Optional

Chunks output: target chunk size (approx. tokens = characters / 4).

Type:integer

Minimum:50

Maximum:8000

Default:512

Chunk overlap (tokens)

chunkOverlap

Optional

Chunks output: tokens repeated from the end of one chunk at the start of the next.

Type:integer

Minimum:0

Maximum:2000

Default:64

Respect robots.txt

respectRobotsTxt

Optional

Skips pages the site's robots.txt disallows for all crawlers.

Type:boolean

Default:true

Start URLs in parallel

maxConcurrency

Optional

How many start URLs are processed at the same time (pages of one site are read one after another, politely).

Type:integer

Minimum:1

Maximum:10

Default:2

Proxy configuration

proxyConfiguration

Optional

Residential proxies (the default) get through to most sites; pick a country when a site shows region-specific content. The AI calls do not use the proxy.

Type:object

Default:

{
  "useApifyProxy": true,
  "apifyProxyGroups": [
    "RESIDENTIAL"
  ]
}

AI Web Scraper - Powered by Crawl4AI

raizen/ai-web-scraper

A blazing-fast AI web scraper powered by Crawl4AI. Perfect for LLMs, AI agents, AI automation, model training, sentiment analysis, and content generation. Supports deep crawling, multiple extraction strategies and flexible output (Markdown/JSON). Seamlessly integrates with Make.com, n8n, and Zapier.

Raizen Technology

357

1.0

(1)

📧 Email & Contact Extractor From Any Web

easyapi/email-contact-extractor-from-any-web

A powerful web scraper that automatically extracts emails, phone numbers, and social media profiles from any website. Features deep crawling, smart retry mechanism, and advanced pattern recognition. Perfect for lead generation and contact discovery.

EasyApi

330

Bilibili Scraper

neuro-scraper/bilibili-scraper

Bilibili Scraper SD - Bilibili Scraper is a social media data tool that extracts public Bilibili (哔哩哔哩) videos from search, video details, full comment threads, creator videos and profiles, popular and zone rankings and hot searches, with views, likes, coins, danmaku and tags. No login.

Neuro Scraper

AutoTrader Cars Scraper

neuro-scraper/autotrader-cars-scraper

AutoTrader Cars Scraper SD - AutoTrader Cars Scraper is a logged-out tool that extracts car listings from AutoTrader UK, USA (VIN, KBB price), Canada and South Africa: price, price rating, mileage, specs, history, images and dealer details.

Neuro Scraper

OpenAI Web Scraper

dtrungtin/openai-web-scraper

Crawl web pages and extract structured information using OpenAI

Tin

E-commerce Product Scraper (Any Store)

neuro-scraper/e-commerce-product-scraper

E-commerce Product Scraper (Any Store) SD - E-commerce Product Scraper (Any Store) is a product data tool that extracts title, price, currency, availability, brand, SKU, GTIN, rating, reviews, images, breadcrumbs, specs and variants from any shop's product, category or search pages.

Neuro Scraper

Bayt Jobs Scraper

neuro-scraper/bayt-jobs-scraper

Bayt Jobs Scraper SD - Bayt Jobs Scraper is a job data tool that extracts bayt.com jobs in UAE, Saudi Arabia, Egypt and MENA: title, company, salary, career level, job role, industry, skills, degree, experience, description and apply link. No login.

Neuro Scraper

Instagram Engagement Metrics Scraper

neuro-scraper/instagram-engagement-metrics-scraper

Instagram Stats Scraper SD - Instagram Stats Scraper is an analytics tool that extracts engagement statistics of public Instagram accounts: engagement rate, average and median likes and comments, reel plays, posts per week, top hashtags and the best post.

Neuro Scraper

Bien'ici Scraper [$1/1K💰] | French Real Estate & DPE Data

ahmed_jasarevic/bienici-scraper

Scrape bienici.com French property listings: price, price per m², surface, rooms, DPE/GES energy classes, agency and coordinates. Ideal for real-estate market research, valuation and price monitoring in France.

Ahmed Jasarevic

BigCommerce Store Email and Phone Number Scraper

neuro-scraper/bigcommerce-store-email-and-phone-number-scraper

BigCommerce Store Email and Phone Number Scraper SD - BigCommerce Store Email and Phone Number Scraper is a lead generation tool that extracts leads with public contact emails and phone numbers from BigCommerce Store by keyword, location, email domain and country.

Neuro Scraper