AI Web Search & Read: Google results as clean Markdown avatar

AI Web Search & Read: Google results as clean Markdown

Pricing

from $3.00 / 1,000 results

Try for free
Go to Apify Store
AI Web Search & Read: Google results as clean Markdown

AI Web Search & Read: Google results as clean Markdown

Try for free

AI Web Search runs a Google search, fetches the top organic results and returns clean Markdown per result — one call turns a question into LLM-ready context for agents, RAG and MCP.

Pricing

from $3.00 / 1,000 results

Rating

0.0

(0)

Developer

Murat Uzun

Murat Uzun

Maintained by Community

Actor stats

0

Bookmarked

1

Total users

0

Monthly active users

2 days ago

Last modified

Categories

AI

Agents

SEO tools

Share

What is AI Web Search & Read?

AI Web Search & Read gives your agent the web in one call: send it a question, and it runs a Google search, fetches the top organic result pages, strips navigation/ads/boilerplate and returns clean Markdown per result — ready to drop straight into an LLM prompt, a RAG pipeline or a tool-calling agent. It also accepts a list of urls to skip the search step entirely and just convert specific pages to Markdown, and it can run in Standby mode as a live HTTP API instead of a batch run. Try it from the Input tab, or call it from code, MCP or the Apify API.

Why use AI Web Search & Read?

Most "give my agent web access" setups need two or three separate tools: a SERP API, a scraper, and a Readability/boilerplate remover. This Actor collapses all three into one call and one price. Typical users:

  • AI agent builders wiring a web_search tool into a LangChain, LlamaIndex, OpenAI-function or MCP agent, without standing up their own SERP + scraping infrastructure.
  • RAG pipelines that need fresh, non-indexed web content as retrieval context rather than a stale vector store.
  • Researchers and analysts who want 3-5 clean summaries of "what does the web say about X" without opening ten tabs.
  • Anyone already using urls-only mode as a lightweight, dependency-free "webpage to Markdown" reader.

How to use AI Web Search & Read

  1. Enter a Search query (or a list of Search queries, or a list of URLs to read directly, skipping search).
  2. Set Max results per query (default 3) and Output format (Markdown, plain text, or both).
  3. Click Start. Each organic result becomes one dataset row with the page's title, URL, snippet and clean Markdown.
  4. Export as JSON, CSV, Excel or HTML, or read the results back over the API/MCP.

Example input

{
    "query": "what is web scraping",
    "maxResults": 3,
    "outputFormat": "markdown"
}

Or read specific pages directly, no search:

{
    "urls": ["https://docs.apify.com/platform"]
}

Example output

{
    "query": "what is web scraping",
    "position": 1,
    "title": "What Is Web Scraping? - Apify",
    "url": "https://apify.com/web-scraping",
    "displayedUrl": "apify.com › web-scraping",
    "snippet": "Web scraping is the automated process of extracting data from websites...",
    "markdown": "# What Is Web Scraping?\n\nWeb scraping is the automated process of...",
    "text": null,
    "wordCount": 842,
    "charCount": 6213,
    "truncated": false,
    "lang": "en",
    "publishedAt": "2025-03-11T00:00:00Z",
    "author": null,
    "siteName": "apify.com",
    "statusCode": 200,
    "fetchedAt": "2026-09-13T10:00:04.000Z",
    "error": null,
    "scrapedAt": "2026-09-13T10:00:03.500Z"
}

You can download the dataset in JSON, CSV, Excel, HTML, XML or RSS.

Data table

FieldTypeDescription
query, positionstring, numberThe query this row came from (null for direct urls) and its SERP rank
title, url, displayedUrl, snippetstringGoogle's result title/URL/breadcrumb/snippet
markdown, textstringClean Markdown / plain text of the page's main content
wordCount, charCount, truncatednumber, booleanSize of the extracted content and whether it was cut to fit
lang, publishedAt, author, siteNamestringMetadata read from JSON-LD or meta tags, when present
statusCode, fetchedAtnumber, stringHTTP status of the page fetch and when it happened
error, scrapedAtstringSet when the search or page fetch failed; always-present timestamp

Input parameters

ParameterTypeDefaultDescription
querystringwhat is web scrapingA single search query
queriesarray–Multiple queries in one run; overrides query
urlsarray–Specific pages to read, skipping search entirely
maxResultsinteger3Organic results read per query (max 20)
outputFormatenummarkdownmarkdown, text, or both
maxCharsPerResultinteger8000Truncate long pages for LLM context control (max 100000)
countryCode, languageCodestringus, enGoogle localisation (gl/hl)
includeSnippetOnlybooleanfalseCheap mode: return SERP rows without fetching pages
maxConcurrencyinteger5Parallel page fetches

Pricing

AI Web Search & Read is priced at $0.005 per result (pay-per-event), with volume discounts at higher usage tiers. A default 3-result run costs about $0.015. One SERP fetch (shared across up to 20 results for that query) uses roughly $0.0025 of Apify Proxy traffic internally — page fetches themselves are free, direct HTTP with no proxy. That works out to ~$5 per 1,000 results, in line with the median price of the top actors on Apify Store, and typically cheaper than stacking a separate SERP API + scraper + Readability service.

AI Web Search & Read vs. apify/rag-web-browser

Both turn a query into LLM-ready Markdown. This Actor additionally offers: a pure urls-only reader mode (no search needed), a dedicated includeSnippetOnly cheap mode, both markdown+text output, and JSON-LD-aware metadata (author, publishedAt, siteName) on every row — useful for citing sources back to the LLM.

Using AI Web Search & Read with AI agents

Batch (Apify API / MCP tool call):

curl -s -X POST "https://api.apify.com/v2/acts/webdatatools~ai-web-search/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query": "best crm for startups", "maxResults": 5}'

This Actor is also discoverable through the Apify MCP Server — an MCP-connected agent (Claude, Cursor, etc.) can call it directly as a tool once added.

LangChain (Python):

from langchain_apify import ApifyActorsTool


search_tool = ApifyActorsTool("webdatatools/ai-web-search")
result = search_tool.invoke({"query": "what is retrieval augmented generation", "maxResults": 3})

LlamaIndex:

from llama_index.tools.apify.tool import ApifyWrapper


apify = ApifyWrapper()
dataset = apify.call_actor(
    actor_id="webdatatools/ai-web-search",
    run_input={"query": "what is retrieval augmented generation", "maxResults": 3},
    dataset_mapping_function=lambda item: {"content": item["markdown"], "url": item["url"]},
)

Standby mode: call it like a live API

This Actor supports Actor Standby — start it once and call GET /search as a normal HTTP endpoint instead of starting a new batch run every time:

GET https://ai-web-search.webdatatools.apify.actor/search?query=apify+web+scraping&maxResults=3

Response is a plain JSON array of the same rows described above. GET / returns a readiness payload for health checks.

FAQ

Is this legal? Scraping public Google search results and public web pages for personal or internal use is generally accepted practice, but you are responsible for complying with the target sites' terms of service and applicable law in your jurisdiction.

Why is markdown sometimes short or empty? JavaScript-rendered pages (heavy React/Vue SPAs that inject content client-side) return little or no text to a plain HTTP fetch — this Actor does not run a browser. For those sites, wordCount will be low and text/markdown sparse; this is an honest limitation, not a bug.

Why did I get an error row instead of content? A page fetch can fail (404, timeout, blocked) or the SERP fetch itself can be rate-limited — the row still comes back with error set and every other field intact where possible, so your agent always gets a response instead of a broken run.

Does this work with n8n / Make / Zapier? Yes — use the official Apify integration in any of those tools, or call the REST API directly.

Related Actors: Google Search Results Scraper (raw SERP data), Website to Markdown Crawler (full-site crawling), Article Extractor (single-article metadata).

Found a bug or have a feature request? Use the Issues tab on this Actor's page — we read every report.

Part of the webdatatools web-intelligence suite — every Actor is pay-per-event, reads public data without a login, and returns one clean row per entity:

Browse the whole suite at webdatatools, or call ten of these Actors straight from Claude, Cursor or Cline with the webdatatools MCP server.

Website & domain intelligence

Content for AI, LLMs and RAG

Search, video and social

Leads, jobs and company data

Developer, app and research data

You might also like

⚡ Web to LLM Clean Markdown & RAG Crawler avatar

⚡ Web to LLM Clean Markdown & RAG Crawler

unrivaled_fortress/llm-markdown-clean-web-crawler

Convert any webpage into clean, LLM-ready markdown. Strips ads, navbars, and cookie banners to optimize AI context tokens for ChatGPT, Claude, and RAG.

User avatar

David A

3

AI Web Search Scraper - SERP & Page Content for LLM RAG avatar

AI Web Search Scraper - SERP & Page Content for LLM RAG

get_anything/ai-web-search-scraper

Web search API for LLMs: query -> clean results (title, URL, snippet, position, domain) and OPTIONALLY each page's full content as LLM-ready Markdown. Ground ChatGPT/Claude and RAG pipelines with live web results. Google (browser-rendered) or DuckDuckGo engine. Export JSON, CSV or Markdown.

User avatar

Get Anything

3

OpenClaw Web Search Provider avatar

OpenClaw Web Search Provider

automation-lab/openclaw-web-search-provider

Run DuckDuckGo-backed web queries for OpenClaw agents and export ranked titles, resolved URLs, domains, snippets, query, rank, and fetch provenance.

User avatar

Automation Lab

3

AI Search Scraper – Google AI Overview Answers & Citations avatar

AI Search Scraper – Google AI Overview Answers & Citations

inovaflow/ai-search-scraper

Scrape the real Google AI Overview answer people see in search results — the answer text, its cited sources with resolved URLs, and the page-1 organic results — for any query, per country and language. Present / no-answer flag per row, no API keys, no login. Dataset-only, MCP-ready.

User avatar

inovaflow

2

CC

Clean Content Extractor for RAG & AI Agents

hadster/clean-content-extractor-for-rag-ai-agents

Turn any article into clean markdown for LLMs. Renders JS-heavy pages, strips ads and navigation, pay only for successful extractions. Best for blog posts, news articles, and documentation pages — not homepage or listing pages.

User avatar

Hadi Fadlallah

2

WP

Web Page to Markdown

alleserojje/web-page-to-markdown

Convert any web page into clean, readable Markdown or plain text — ads, navigation and boilerplate stripped out. Built for feeding AI agents, LLMs and RAG pipelines. Pay only per page.

User avatar

Pedro Resende

2

LW

LLM Web Scraper - Website to Markdown for RAG & ChatGPT

get_anything/llm-web-scraper

Turn any URL or website into clean, LLM-ready Markdown or text - boilerplate, ads and nav stripped - with metadata and token count. Feed ChatGPT/Claude context or build RAG pipelines. HTTP-fast with a hardened-browser fallback for JS-heavy pages. Export JSON, CSV or Markdown.

User avatar

Get Anything

2

Google Search Results Scraper — SERP API avatar

Google Search Results Scraper — SERP API

webdatatools/google-search-scraper

Google Search Results Scraper returns organic results (title, real URL, snippet, position) for any query through the Apify GOOGLE_SERP proxy — one row per SERP page, priced per page not per result.

User avatar

Murat Uzun

6

HW

HAW Web to Markdown & JSON

martinhida/haw-web-to-markdown-json

Turn public web pages into clean Markdown, text, links, headings, and metadata.

User avatar

Hidalgo Martin

2

Web Page to Markdown for AI: $1/1K Pages, No Login avatar

Web Page to Markdown for AI: $1/1K Pages, No Login

conserving_celerytop/web-page-to-markdown

Turn any list of web pages into clean Markdown for LLMs, RAG and AI agents: main content without menus and ads, tables kept, plus title, description, language, dates and links. Plain HTTP first, JavaScript only when a page needs it. $1 per 1,000 pages, failures free.

User avatar

Don Mangu

2