MCP server for AI assistants to scrape websites, extract structured content, crawl pages, and parse HTML. Works with Claude, Cursor, and any MCP-compatible AI client.

🔑 Features

• Runs in the Apify cloud — no local setup, no maintenance
• Structured JSON output ready for analytics, CRMs, or databases
• Batch-friendly — process many inputs in one run
• Integrates with Zapier, Make.com, n8n, Google Sheets, Slack, and direct API calls
• Pay-per-use pricing — no subscription, no monthly minimum

💼 Common Use Cases

• Market research and competitive intelligence
• Data enrichment and lead generation
• Content monitoring and automated alerts
• Pipeline and integration workflows
• Dashboards and reporting

Web Scraping & Search MCP Server

MCP server for general web scraping: crawl any URL, Google search, Wikipedia lookup, and website content extraction.

Connect via MCP: https://nexgendata--web-scraping-mcp-server.apify.actor/mcp

💻 Code Example — Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("nexgendata/web-scraping-mcp-server").call(run_input={
    # Fill in the input shape from the actor's input_schema
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

🌐 Code Example — cURL

curl -X POST "https://api.apify.com/v2/acts/nexgendata~web-scraping-mcp-server/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ /* input schema */ }'

❓ FAQ

Q: How do I get started? Sign up at apify.com, grab your API token from Settings → Integrations, and run the actor via the Apify console, API, Python SDK, or any integration (Zapier, Make.com, n8n).

Q: What's the typical cost per run? See the pricing section below. Most runs finish under $0.10 for typical batches.

Q: Is this actor maintained? Yes. NexGenData maintains 140+ Apify actors and ships updates regularly. Bug reports via the Apify console issues tab get responses within 24 hours.

Q: Can I use the output commercially? Yes — you own the output data. Check the target site's Terms of Service for any usage restrictions on the scraped content itself.

Q: How do I handle rate limits? Apify manages concurrency and retries automatically. For very large batches (10K+ items), run multiple smaller jobs in parallel instead of one mega-job for better reliability.

💰 Pricing

Pay-per-event pricing — you only pay for what you actually extract.

• Actor Start: $0.0001
• result: $0.0020

🔗 Related NexGenData Actors

• RAG Web Browser
• AI Web Scraper
• Hacker News Scraper

🚀 Apify Affiliate Program

New to Apify? Sign up with our referral link — you get free platform credits on signup, and you help fund the maintenance of this actor fleet.

Built and maintained by NexGenData — 140+ actors covering scraping, enrichment, MCP servers, and automation.