Pricing

Pay per usage

Go to Store

RAG Web Browser

Try for free

Developed by

Apify

Web browser for OpenAI Assistants, RAG pipelines, or AI agents, similar to a web browser in ChatGPT. It queries Google Search, scrapes the top N pages from the results, and returns their cleaned content as Markdown for further processing by an LLM. It can also scrape individual URLs. Supports MCP.

4.4 (11)

Pricing

Pay per usage

Monthly users

847

Runs succeeded

>99%

Response time

2.6 days

Last modified

18 days ago

Open source

Back to issues Create new issue

Doesnt seem to remove images from returned markdown

Open

zognotadog opened this issue

The crawler seems to be outputting image data in the markdown even though its meant to strip it. This was confirmed with a few runs on our own site an apify.com

Has something gone wrong? The Website Content Crawler does not have this issue.

-------- RAG Web Browser Result ---------------

Apify: Full-stack web scraping and data extraction platformStar apify/crawlee on GitHubRib StichBack ButtonSearch IconFilter Icon Skip to content Contact sales Log in Get started # Your full‑stack platform for web scraping Apify is the largest ecosystem where developers build, deploy, and publish data extraction and web automation tools

-------- Website Content Cralwer Result for Apify.com --------

Full-stack web scraping and data extraction platform Apify is the largest ecosystem where developers build, deploy, and publish data extraction and web automation tools. We call them Actors. TikTok Data Extractor clockworks/free-tiktok-scraper Extract data about videos, users, and channels based on hashtags or...

Jiří Spilka (jiri.spilka)

Hi, thank you for using RAG Web Browser.
I appreciate your detailed explanation.

The RAG Web Browser has a slightly different configuration. To keep settings simple, it outputs raw page content without transformation, unlike the Website Content Crawler, which uses the readableText option. This option can sometimes remove content and isn’t 100% reliable. Instead, in RAG Web Browser, we let the LLM determine what content is useful by setting "htmlTransformer": "none".

When I run Website Content Crawler with "htmlTransformer": "none", I receive similar output to the RAG Web Browser.

RAG Web Browser: run
"Apify: Full-stack web scraping and data extraction platformStar apify/crawlee on GitHubRib StichBack ButtonSearch IconFilter Icon\n\nSkip to content
Website Content Crawler: run
"Apify: Full-stack web scraping and data extraction platform\n\nSkip to content

Interestingly, there is a bit more processing Website Content Crawler is doing. If you want both Actors to produce identical output, it should be possible. However, I encountered an issue when testing this and couldn't quickly figure out the cause.

Would you like to use RAG Web Browser with this configuration? If so, I can look into that further.

Apologies for any inconvenience. Jiri

Add comment

Pricing

Pricing model

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.

Actors MCP Server

apify/actors-mcp-server

Connect Apify Actors to AI applications (AI Agents) using Anthropic's Model Context Protocol (MCP). This MCP server provides integration of any Apify Actor through the MCP framework.

Apify

648

Mastra.ai MCP Agent

jakub.kopecky/actor-mastra-mcp-agent

🤖 AI agent using mastra.ai with Apify MCP Server. 🚀 Runs queries via OpenAI models, taps Apify Actors for web data, and outputs to datasets. 🛠️

Jakub Kopecký

Playwright MCP

jiri.spilka/playwright-mcp

A Model Context Protocol (MCP) server that provides browser automation capabilities using Playwright

Jiří Spilka

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

46k

Web Scraper

apify/web-scraper

Crawls arbitrary websites using a web browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

Apify

81k

AI Web Scraper - Powered by Crawl4AI

raizen/ai-web-scraper

A blazing-fast AI web scraper powered by Crawl4AI. Perfect for LLMs, AI agents, AI automation, model training, sentiment analysis, and content generation. Supports deep crawling, multiple extraction strategies and flexible output (Markdown/JSON). Seamlessly integrates with Make.com, n8n, and Zapier.

Raizen Technology

GPT Browser

anchor/gpt-browser

A GPT browser to use OpenAI prompt on any website. Put a list of URLs and a prompt, then the GPT agent will give you the answer you need. Fast, easy, and not limited with OpenAI ChatGPT restrictions. The best way to search and use GPT on large number of websites. Upload Excel or CSV. Screenshots 📸

Anchor

AI Web Agent

apify/ai-web-agent

Use natural language prompts to browse the web, click on elements, fill and submit forms, extract data, and take screenshots using the OpenAI API.

Apify

1.2k

Tester MCP Client

jiri.spilka/tester-mcp-client

A model context protocol (MCP) client that connects to any MCP server using Server-Sent Events (SSE) and displays the conversation in a chat-like UI. It is a standalone Actor server designed for testing MCP servers over SSE.

Jiří Spilka

227

ChatGPT

pertosh/chatgpt

You can use this Actor to transform scraped results, such as reviews from restaurants, by rephrasing the sentences. Additionally, translation is also supported. You can also use it to generate new website descriptions, keywords, and other similar metadata.