Web Content & URL Tools MCP Server

Search the web with Bing and get titles, URLs, and snippets. MCP server for AI agents.

Website Content Crawler — Text, Markdown & AI Chunks

agency-shift/web-content-crawler

Extract text or clean Markdown from public HTML websites. Create source-linked sections for AI knowledge bases while keeping one row per page. Preserve headings, code and tables; inspect a crawl report for failed URLs and truncated content. No JavaScript rendering.

Web Article Content Extractor

vulnv/web-article-content-extractor

Extract clean, readable content from news articles, blog posts, and web pages. Batch process multiple URLs, download images, bypass bot protection with proxy support. Perfect for content curation, research, and data analysis.

Website Content Crawler Pro

datascoutapi/website-content-crawler-pro

Crawl websites and extract clean, structured content in Markdown, JSON, or plain text for AI models, LLMs, vector DBs, or RAG pipelines. Fast, reliable, and stealthy, with bulk processing, advanced metadata extraction, and seamless integration with LangChain, LlamaIndex, and AI workflows.

Web Fetch

apify/web-fetch

Download any web page by URL and convert it to clean Markdown, plain text, HTML, and other formats. Automatically renders JavaScript and works around bot blocking. The Actor provides a real-time API for quick response times.

Tavily MCP Server

agentify/tavily-mcp-server

Advanced web search and data extraction capabilities through the Tavily API, providing real-time web search, intelligent data extraction, website mapping, and web crawling tools.

Gemini AI MCP SERVER

bhansalisoft/gemini-ai-mcp-server

Gemini AI MCP SERVER unique tool for Gamini AI functionality integration with apify and other AI tool.

Web Content Extractor API — URL to JSON

george.the.developer/web-content-extractor-api

Extract structured JSON from any webpage. Articles, products, recipes, jobs. Auto-detects content type. Returns metadata, headings, images, links. For AI agents and RAG.

Web Crawler & Semantic Schema-Enhanced Extractor

devil_port369-owner/web-crawler

Depth-controlled web crawler that transforms websites into structured analytics-ready data. Starting from one or more URLs, it crawls internal links up to a configurable depth and outputs detailed JSON records per page

SEO & Web Analysis MCP — Crawl, Tech & DNS for Agents

nexgendata/seo-web-analysis-mcp-server

MCP server exposing on-page crawl, tech-stack detection, DNS, SSL and WHOIS as agent tools. Connect Claude, Cursor, n8n or the OpenAI Agents SDK.