Pricing

Pay per usage

Try for free

Go to Apify Store

RAG Web Browser

Try for free

Developed by

Apify

Web browser for OpenAI Assistants, RAG pipelines, or AI agents, similar to a web browser in ChatGPT. It queries Google Search, scrapes the top N pages, and returns their content as Markdown for further processing by an LLM. It can also scrape individual URLs. Supports Model Context Protocol (MCP).

4.4 (10)

Pricing

Pay per usage

Last modified

23 days ago

Open source

This changelog summarizes all changes of the RAG Web Browser

1.0.15 (2025-03-27)

🐛 Bug Fixes

Cancel requests only in standby mode

1.0.13 (2025-03-27)

🐛 Bug Fixes

Cancel crawling requests from timed-out search queries

1.0.12 (2025-03-24)

🐛 Bug Fixes

Updated selector for organic search results and places

1.0.11 (2025-03-21)

🐛 Bug Fixes

Selector for organic search results

1.0.10 (2025-03-19)

🚀 Features

Handle all query parameters in the standby mode (including proxy)

1.0.9 (2025-03-14)

🚀 Features

Change default value for scrapingTool from 'browser-playwright' to 'raw-http' to improve latency.

1.0.8 (2025-03-07)

🚀 Features

Add a new scrapingTool input to allow users to choose between Browser scraper and raw HTTP scraper

1.0.7 (2025-02-20)

🚀 Features

Update Readme.md to include information about MCP

1.0.6 (2025-02-04)

🚀 Features

Handle double encoding of URLs

1.0.5 (2025-01-17)

🐛 Bug Fixes

Change default value of input query
Retry search if no results are found

1.0.4 (2025-01-04)

🚀 Features

Include Model Context Protocol in Standby Mode

1.0.3 (2024-11-13)

🚀 Features

Improve README.md and simplify configuration
Add an AWS Lambda function
Hide variables initialConcurrency, minConcurrency, and maxConcurrency in the Actor input and remove them from README.md
Remove requestTimeoutContentCrawlSecs and use only requestTimeoutSecs
Ensure there is enough time left to wait for dynamic content before the Actor timeout (normal mode)
Rename googleSearchResults to searchResults and searchProxyGroup to serpProxyGroup
Implement input validation

0.1.4 (2024-11-08)

🚀 Features

Add functionality to extract content from a specific URL
Update README.md to include new functionality and provide examples

0.0.32 (2024-10-17)

🚀 Features

Handle errors when request is added to Playwright queue. This will prevent the Cheerio crawler from repeating the same request multiple times.
Silence error: Could not parse CSS stylesheet as there is no way to fix it at our end
Set logLevel to INFO (debug level can be set using the debugMode=true input)

2024-10-11

🚀 Features

Increase the maximum number of results (maxResults) from 50 to 100
Explain better how to search a specific website using "llm site:apify.com"

2024-10-07

🚀 Features

Add a short description how to create a custom action

2024-09-24

🚀 Features

Updated README.md to include tips on improving latency
Set initialConcurrency to 5
Set minConcurrency to 3

2024-09-20

🐛 Bug Fixes

Fix response format when crawler fails

2024-09-24

🚀 Features

Add ability to create new crawlers using query parameters
Update Dockerfile to node version 22

🐛 Bug Fixes

Fix playwright key creation

2024-09-11

🚀 Features

Initial version of the RAG Web Browser

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

68K

4.3

Web Scraper

apify/web-scraper

Crawls arbitrary websites using a web browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

Apify

93K

4.4

AI Web Scraper - Powered by Crawl4AI

raizen/ai-web-scraper

A blazing-fast AI web scraper powered by Crawl4AI. Perfect for LLMs, AI agents, AI automation, model training, sentiment analysis, and content generation. Supports deep crawling, multiple extraction strategies and flexible output (Markdown/JSON). Seamlessly integrates with Make.com, n8n, and Zapier.

Raizen Technology

192

1.0

Browser Use Apify

lexis-solutions/browser-use-apify

Open-source AI-powered browser automation based on Browser Use and hosted on Apify. Run any task using Apify's platform and LLMs like ChatGPT, Claude, etc. Easy, scalable, resilient, and hosted solution for web-enabled AI agents.

Lexis Solutions

190

5.0

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

EasyApi

100

5.0

🔥fireScraper AI Prompt Website Content Markdown Scraper

mohamedgb00714/fireScraper-AI-prompt-Website-Content-Markdown-Scraper

fireScrape AI is an advanced web scraper built with Crawlee and Puppeteer. It crawls websites, extracts meaningful content, converts it into Markdown, then runs your custom prompt on the extracted text—ideal for generating enriched datasets, summaries or analyses for LLMs and AI pipelines

mohamed el hadi msaid

5.0

AI-Powered Web Content & Link Extractor

scrapercoder/ai-powered-web-content-link-extractor

Crawls websites to extract clean, structured content for AI/LLM use, ideal for training datasets, knowledge bases, and RAG systems. Json output includes: * text: Normalized page content * links: Extracted sub-URLs

wallnut.ai

112

Actors MCP Server

apify/actors-mcp-server

⚠️ Legacy: This Actor is outdated. For the latest features and full documentation, visit https://mcp.apify.com. Easily connect any Apify Actor to AI agents using Anthropic’s Model Context Protocol (MCP) with our actively maintained MCP server.

Apify

2.1K

4.9

🔥 FireScrape AI Website Content Markdown Scraper

mohamedgb00714/fireScraper-AI-Website-Content-Markdown-Scraper

Advanced web scraper powered by Crawlee and Puppeteer — extracts website content, converts it to Markdown, and structures it for LLM training datasets.

mohamed el hadi msaid

118

3.8

Extended GPT Scraper

drobnikj/extended-gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.