AI Agent Web Fetcher

Pricing

Pay per event

AI Agent Web Fetcher

An advanced web fetcher that can fetch almost all websites and convert them to LLM-friendly Markdown format. Perfect for AI agents, RAG systems, and integration with search actors.

Pricing

Pay per event

Rating

0.0

(0)

Developer

AbotAPI

Maintained by Community

Actor stats

Bookmarked

Total users

Monthly active users

16 minutes ago

Last modified

Features

🌍 Universal Fetching: Capable of fetching content from almost any website
🛡️ Advanced Anti-Blocking: Automatically handles complex challenges and protections
📝 Markdown Output: Clean, LLM-friendly format
🔗 Integration Ready: Seamlessly integrates with AI Web Search Tool and other search actors
🎯 Dual Mode: Single URL or batch processing from datasets

Usage

Single URL Mode

{
  "url": "https://example.com"
}

Integration Mode (with AI Web Search Tool)

When triggered by the AI Web Search Tool, the actor automatically:

Receives the search results dataset
Extracts URLs from the link field
Fetches and converts each page to Markdown

No configuration needed - the integration is automatic!

You can also manually specify a dataset:

{
  "datasetId": "abc123",
  "urlField": "link",
  "maxUrls": 10
}

Input Parameters

Parameter	Type	Description	Default
`url`	string	Single URL to fetch	-
`datasetId`	string	Dataset ID containing URLs (integration mode)	-
`urlField`	string	Field name containing URLs in dataset	`link`
`maxUrls`	integer	Max URLs to fetch from dataset (1-100)	`10`
`proxy`	object	Proxy configuration	-
`waitUntil`	string	Navigation wait condition	`domcontentloaded`
`debug`	boolean	Save HTML content in output	`false`

Output Format

Each fetched URL produces:

{
  "url": "https://example.com",
  "markdown": "# Page Title\n\nContent...",
  "metadata": {
    "title": "Page Title",
    "description": "Page description",
    "author": "Author name"
  }
}

Integration with AI Web Search Tool

Run the AI Web Search Tool
Add this actor as an integration
The actor automatically fetches and converts all search results to Markdown

Perfect for building AI agents that need to:

Research topics with real-time web data
Extract content from search results
Build RAG systems with fresh web content

Markdownify MCP Server

crawlerbros/markdownify-mcp-server

Convert any webpage to clean, formatted Markdown perfect for AI consumption. Ideal for building knowledge bases, documentation scrapers, and content migration tools.

Crawler Bros

5.0

(5)

AI Real Estate Agent

harvestlabs/ai-real-estate-agent

Given a user's input the agent extracts key criteria, searches relevant real estate listings, and outputs a multi-format report.

harvest-org

199

RAG Web Browser

apify/rag-web-browser

Web browser for OpenAI Assistants, RAG pipelines, or AI agents, similar to a web browser in ChatGPT. It queries Google Search, scrapes the top N pages, and returns their content as Markdown for further processing by an LLM. It can also scrape individual URLs.

Apify

7.5K

4.9

(14)

Similarweb scraper

curious_coder/similarweb-scraper

Scrape similarweb to get insights for websites in bulk including website traffic, pages per visit, time spent on website, bounce rate, rank, etc

Curious Coder

1.5K

5.0

(1)

Mastra.ai MCP Agent

jakub.kopecky/actor-mastra-mcp-agent

🤖 AI agent using mastra.ai with Apify MCP Server. 🚀 Runs queries via OpenAI models, taps Apify Actors for web data, and outputs to datasets. 🛠️

Jakub Kopecký

Google Search Unlimited Scraper $5 Only 💸

igolaizola/google-search-scraper

Fast & Low Cost Google Search scraper that returns clean, structured SERP data. Get position, title, URL and snippet. Control region, language and domain, with built-in Apify proxy support for reliable, geo-accurate results.

Iñigo Garcia Olaizola

Google Search Scraper ($0.15 per 1000 results)

igolaizola/google-search-scraper-ppe

Iñigo Garcia Olaizola

5.0

(1)

MCP Nexus Universal AI Tool Bridge

tuguidragos/mcp-nexus-universal-ai-tool-bridge

Connect AI agents to real data. MCP Nexus runs tools that fetch, extract, summarize, classify and crawl web content with caching, multi LLM support, HMAC webhooks, circuit breakers and full observability in a stateless production ready Apify actor.

Țugui Dragoș

5.0

(1)

Finance Monitoring Agent

jakub.kopecky/finance-monitoring-agent

The Finance Monitoring AI Agent 📊💹 analyzes specific tickers, gathering, and processing data to generate insightful reports 📈📉. Designed for investors and analysts, this agent provides detailed performance analysis and trends. The agent is built using LangGraph and Python.

Jakub Kopecký

Skool Members Scraper

sajjadalikalhoro13/Skool-members-scraper

The Skool Members Scraper is an Apify Actor that extracts member data from any public Skool community. Given a Skool group URL (e.g. https://www.skool.com/ai-agents-mastery), it automatically