🤖 RAG Web Browser

🚀 Give your LLM live web access in seconds. Search the web or fetch specific URLs and return clean Markdown with 17 metadata fields per page. No API key, no registration, no manual content cleaning.

The RAG Web Browser is built for retrieval-augmented generation pipelines, autonomous agents, and any workflow where an LLM needs grounded, up-to-date web content. Send a search query to get the top N results, or pass a list of URLs to fetch them directly. Every page is stripped of navigation, ads, and boilerplate, then converted to clean Markdown that feeds directly into embedding pipelines and vector databases.

Each record ships with rich metadata including title, description, author, published time, modified time, site name, Open Graph image, language, word count, and estimated reading time. Search results include a rankFromSearch field so you can weight retrieval by original engine position. Concurrent fetching keeps 10 URLs flying in parallel, so research agents stay snappy and RAG refreshes finish while your coffee is still hot.

🎯 Target Audience	💡 Primary Use Cases
AI engineers, RAG builders, research agent developers, LLM app teams, content researchers, data scientists	Live RAG context, agent web browsing, knowledge base refresh, competitive intelligence, fact-grounding

📋 What the RAG Web Browser does

Five content workflows in a single run:

• 🔎 Search mode. Pass a text query and get the top N results from DuckDuckGo with clean content for each.
• 🎯 URL mode. Provide specific URLs and the scraper fetches them in parallel.
• 📝 Clean Markdown. Strips navigation, footers, sidebars, scripts, and ads. Preserves headings, lists, blockquotes, and code blocks.
• 📊 Rich metadata. Title, description, author, publishedTime, modifiedTime, siteName, og:image, language, word count, reading time.
• 🏆 Search rank preserved. When searching, every result keeps its rank position so you can weight retrieval accordingly.

Output comes as markdown, plain text, or raw HTML. You can also request an outbound-link dump when you need to follow references.

💡 Why it matters: LLMs trained on data older than six months cannot answer questions about today's news, pricing, or product documentation. This Actor gives them a live window on the web without you having to build browser automation, proxies, or content cleaners.

📊 Data fields

Each record includes: author, description, fetchedAt, httpStatus, imageUrl, language, markdown, publishedTime, rankFromSearch, readingTimeMinutes, responseTimeMs, results, siteName, title, url, wordCount. These field names come straight from the actor's dataset schema, so what you see here is what lands in your dataset.

⚠️ Good to Know: single-page apps with heavy client-side rendering sometimes return thin content because the scraper fetches server-rendered HTML. For JavaScript-heavy sites (Notion, Gitbook, some app dashboards), pair this Actor with Website Content Crawler and its browser rendering mode.

🚀 How to use

1. 📝 Sign up. Create a free account with $5 credit (takes 2 minutes).
2. 🌐 Open the Actor. Go to the RAG Web Browser page on the Apify Store.
3. 🎯 Pick a mode. Enter a search query OR a list of URLs, set maxItems, and choose output formats.
4. 🚀 Run it. Click Start and let the Actor collect your content.
5. 📥 Download. Grab your results in the Dataset tab as CSV, Excel, JSON, or XML.

⏱️ Total time from signup to downloaded dataset: 3-5 minutes. No coding required.

🔗 Recommended Actors

• 🕸️ Website Content Crawler - Deep-crawl a domain with depth and JS rendering
• 📰 Smart Article Extractor - Extract clean article text from news sites
• 🔍 Google Search Scraper - SERP results with rank and description
• 📧 Contact Info Scraper - Emails, phones, and socials from URLs
• 📸 URL Screenshot Tool - Full-page screenshots as PNG, JPEG, or PDF

💡 Pro Tip: browse the complete ParseForge collection for more AI-ready web tools.

⚠️ Disclaimer: this Actor is an independent tool and is not affiliated with any search engine or website. Only publicly accessible web content is fetched. Respect the robots.txt and terms of service of every site you add to the input.

🆘 Need Help?

If you hit a bug, have questions about setup, or need a scraper we haven't built yet, open our contact form or write to parseforge@protonmail.com. We also take on paid custom data projects.

For faster answers, join our Discord. It's the best place to get support and suggest new actors.