Pricing

from $2.00 / 1,000 page reads

Try for free

Go to Apify Store

Agent Web Reader: Search + URL to Markdown for LLMs

Try for free

Web search or URLs in, clean markdown out, one JSON row per page with source URL, title and status. $0.002 per page, $0.003 per search; failed URLs never charged. Token budget per result, HTML and PDF, errors returned as rows instead of crashing the call. Built for agents.

Pricing

from $2.00 / 1,000 page reads

Rating

0.0

(0)

Developer

Catalyst Labs

Actor stats

Bookmarked

Total users

Monthly active users

6 hours ago

Last modified

Agent Web Reader: Web Search & URL to Markdown API for LLMs API through CLI

The Apify CLI is the official tool that allows you to use Agent Web Reader: Search + URL to Markdown for LLMs locally, providing convenience functions and automatic retries on errors.

Install Apify CLI

Using installation script (macOS/Linux):

$curl -fsSL https://apify.com/install-cli.sh | bash

Using installation script (Windows):

$irm https://apify.com/install-cli.ps1 | iex

Using Homebrew:

$brew install apify-cli

Using npm:

$npm install -g apify-cli

Other API clients include:

Agent Web Reader: Search + URL to Markdown for LLMs API in Python

Agent Web Reader: Search + URL to Markdown for LLMs API in JavaScript

Agent Web Reader: Search + URL to Markdown for LLMs OpenAPI definition

Agent Web Reader: Search + URL to Markdown for LLMs API

Web Page to Markdown & Text - URL Reader for LLMs & RAG

entranced_gelato/ai-web-page-reader

Read any web page as clean text + Markdown for LLMs and automations. Strips ads, nav, and scripts; returns the main content, metadata (title, author, date, word count), and an optional AI TL;DR + key points. The web-reading primitive for AI agents, RAG pipelines, and no-code flows.

AIDevs

Universal Web Scraper - Extract Any URL

lazymac/web-scraper-toolkit

Pay-per-result web scraper with JS rendering, CSS selector / XPath / regex extraction, schema validation, retry on failure. Use for product catalogs, competitor pricing, news aggregation, lead generation. Fast (<2s/page), respects robots.txt by default.

2x lazymac

Rag Web Browser

opalescent_quintet/rag-web-browser

A specialized crawler designed exclusively to feed LLMs. It visits a website and extracts core content into clean, token-optimized Markdown, stripping all "junk" (navs, footers, ads, cookie banners).

Aryan

Website Content Fetcher | $0.25/1K Pages

litescrape/web-fetch

Fetch any public web page as clean Markdown, text or HTML with title and metadata. Optional selectors and link or image summaries. One page per request. $0.25 per 1k requests, run usage included.

Lite Scraper

Web Page to Markdown for LLMs

incisive_ovation/web-to-markdown

Turn web pages into clean, LLM-ready Markdown: main content only, without menus, ads or cookie banners. Keeps headings, lists, links, code blocks and tables. Pay only per converted page.

pey vodu

Podcast Transcript API - RSS, Apple Podcasts & Audio to Text

catalyst-labs/podcast-transcript-api

Get podcast and audio transcripts as text, timestamped segments, SRT/VTT and Markdown. Input: RSS feeds, Apple Podcasts links, episode pages or audio/video URLs. Uses the publisher's own transcript when one exists ($0.01), Whisper speech-to-text otherwise.

Catalyst Labs

EU GDPR / DSGVO, Cookie & Google Fonts Auditor

cobalt-data-labs/eu-gdpr-cookie-privacy-compliance-auditor

Audit websites for EU GDPR and German DSGVO compliance: detect Cookie Consent Managers (Usercentrics, Cookiebot, Borlabs), remote Google Fonts IP leaks, 3rd-party trackers, Datenschutz URLs, and DPO contacts.

Cobalt Data Labs

Web Search MCP Server: Web, News, Answers for AI Agents

mrlarryjohnson/web-search-mcp-server

MCP server giving AI agents (Claude, Cursor, n8n, Make) multi-engine web search, news search, and DuckDuckGo instant answers. No API keys, pay per tool call, hosted — connect in 30 seconds.

Larry Johnson

Bilibili Video Scraper — Search & Trending (哔哩哔哩)

technicaldost/bilibili-video-scraper-search-trending

Scrape Bilibili (哔哩哔哩) videos by keyword or from the trending (热门) list: title, views, likes, coins, favorites, danmaku, comments, shares, duration, tags, category, author and publish date. No login. $2 per 1,000 videos.

Technical Dost Solutions

Fast Google Search for AI Agents, LLMs & RAG

knightwwang/fast-google-search-for-ai-agents

Lightning-fast, lightweight Google search scraper. Extracts organic results, answers, and Markdown snippets without heavy browser overhead. Ideal for LLM tool calling.