Pricing

from $2.00 / 1,000 results

Website RAG & Lead Intelligence Crawler

Convert public websites to clean Markdown for RAG and AI. Crawl pages and extract emails, phones, social profiles, metadata, technologies, headings and links.

Pricing

from $2.00 / 1,000 results

Rating

0.0

(0)

Developer

Samuel Huirau Atutahi

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

What it extracts

For every successfully crawled page:

Clean Markdown for RAG, LLM, and knowledge-base workflows
Page title and meta description
Canonical URL
H1/H2/H3 headings
Publicly displayed email addresses
Publicly displayed phone numbers
Linked social profiles
Basic website technology detection
Internal and external link counts
HTTP status
Crawl depth
Word and character counts
Fetch timestamp

Useful for

RAG and AI agents

Convert website content into compact Markdown suitable for embeddings, retrieval systems, AI agents, summarization, and knowledge bases.

Business research

Collect publicly displayed business contact information, website metadata, social links, and technology signals alongside page content.

Website intelligence

Discover page structure, technologies, canonical URLs, headings, and internal-link relationships.

Data pipelines

Export results as JSON, CSV, Excel, XML, JSONL, or retrieve them through the Apify API.

Crawl controls

Configure:

Multiple start URLs
Maximum page count
Crawl depth
Same-domain restriction
robots.txt compliance
Include URL patterns
Exclude URL patterns
Concurrency
Request timeout
Maximum Markdown size per page

Output

Each dataset item represents one page.

Typical fields include:

url, final_url, title, description, markdown, emails, phones, social_profiles, technologies, headings, word_count, internal_link_count, and external_link_count.

A run-level SUMMARY record also contains aggregate crawl statistics and unique discovered contacts and technologies.

Responsible use

This Actor is intended for publicly accessible website content.

respect_robots is enabled by default. Users are responsible for ensuring their use complies with applicable website terms, permissions, privacy rules, and laws.

The Actor does not log into websites, bypass access controls, solve CAPTCHAs, or access private content.

Cost-efficient architecture

The Actor uses lightweight HTTP requests rather than a browser by default, keeping runs fast and inexpensive.

DataVault Labs

Built by DataVault Labs for practical automation, structured data, RAG, and AI-agent workflows.

Website to Markdown Crawler – Clean RAG Content

signal_lab/website-to-markdown-crawler

Website-to-Markdown crawler for RAG and LLM datasets. Extract clean content, SEO metadata, links, JSON-LD, sitemaps, and robots.txt-compliant pages.

Signal Lab

AI-Ready Website Crawler

optimus-fulcria/ai-ready-website-crawler

Crawl websites and convert to clean markdown for AI/RAG, LLM fine-tuning, and document pipelines.

Fulcria Labs

Website to RAG Markdown Crawler

knotted_tussock/rag-markdown-crawler

Crawl any website or docs site and export clean Markdown plus JSONL-style chunks for RAG, LLM apps, and AI agents.

Ralph T

Website to Markdown Crawler for LLM & RAG

logiover/website-text-markdown-crawler

Crawl any website to clean Markdown and plain text for LLM training and RAG. HTML to Markdown, no API or login. Export website text to CSV or JSON.

Logiover

Web-to-Markdown Generator for AI & RAG Pipelines

profitstack/web-to-markdown-generator-for-ai-rag-pipelines

Convert any website into clean, heading-based chunking, LLM-ready Markdown for RAG and AI agents.

Manas Mantri

Website Contact Crawler

competent_clarinet/website-contact-crawler

Crawls websites to extract emails, phones, and social links.

Man Mohit verma

5.0

Website Contact Finder

ezzatox/website-contact-finder

Crawl public websites for emails, phones, social profiles, contact pages, and company metadata.

Ezzat Ghazi

Site Crawler: Website → Markdown Corpus for LLM/RAG

boxbox10/site-crawler

Crawl a whole website or docs site and get one clean, LLM-ready Markdown + JSON record per page (title, headings, content, links, token count). Built for RAG ingestion and AI knowledge bases.

Marvin Eguilos

Website to RAG Dataset

sebastian-actors/website-to-rag-dataset

Convert public websites, docs, blogs, and XML sitemaps into clean Markdown, structured metadata, and stable chunks for RAG pipelines and vector databases.