🕷️ Website Crawler — Full-Site Scraping for AI
Pricing
from $20.00 / 1,000 results
🕷️ Website Crawler — Full-Site Scraping for AI
Crawl entire websites for clean text, markdown or HTML. Perfect for RAG pipelines, AI training & content analysis. Handles JS-rendered pages. Alternative to Firecrawl & Jina. Pay per page.
Pricing
from $20.00 / 1,000 results
Rating
0.0
(0)
Developer
NexGenData
Maintained by CommunityActor stats
1
Bookmarked
23
Total users
4
Monthly active users
9 days ago
Last modified
Share
Website Crawler
Full-Site Scraping for AI
Public web content, for research and RAG.
📊 What you get
Clean JSON output (7 fields):
url— Urltitle— Titletext— Textmarkdown— Markdownhtml— Htmlmetadata— MetadatacrawledAt— Crawled At
Pricing: $0.020 per page (Pay-Per-Event) — about 50 pages per $1.
🤖 Use with AI agents
Point Claude, the OpenAI Agents SDK, an n8n flow or any MCP-aware client at it.
Sample agent prompt:
Run Website Crawler on my input and return the structured results.
Agentic payments (x402): Supports agentic payment via x402 — call this actor with USDC, no API key required.
🔗 Related tools — Web crawling for AI/RAG
A NexGenData utility actor.