📇 Website Contact & Email Crawler

Crawl real websites and extract emails, phone numbers, and social-media profiles into one clean, aggregated row per domain. Give it a list of website URLs or domains; it fetches each site, follows internal links (prioritizing contact, about, team, impressum and support pages), and pulls the contact data out of the actual page HTML.

This is a genuine website crawler, not a search-snippet scraper. It downloads and parses each page you point it at.

What it extracts

Per domain (one dataset row):

Field	Description
domain	Registrable domain (e.g. apify.com).
companyName	Display name derived from the domain.
emails	Real emails scraped from the pages (deduped, junk-filtered).
roleEmails / personalEmails	Emails split into shared/role inboxes (info@, sales@, support@…) vs. personal.
phones	Phone numbers normalized to E.164 (e.g. +14155550100).
socials	{platform: [handles]} across Facebook, Instagram, X/Twitter, LinkedIn, YouTube, TikTok, Pinterest, GitHub, Telegram, WhatsApp and more.
socialProfiles	Flat list of full social-profile URLs.
sourceUrls	Map of email → the page it was first found on.
pagesScanned / pagesScannedUrls	How many (and which) pages were fetched.
contactPagesFound	The contact/about/team pages that were prioritized.
inferredEmails	Separate, low-confidence emails from Google-snippet discovery — never mixed with scraped emails.
emailMxHeuristic	Optional: per-email syntax flag + whether the domain has an MX record (heuristic only).
scrapedAt	ISO-8601 timestamp.

How email extraction stays honest

• Scraped emails come only from the real page HTML: mailto: links (highest confidence), Cloudflare data-cfemail XOR-decoded addresses, /cdn-cgi/l/email-protection links, de-obfuscated text (name [at] domain [dot] com, HTML entities, zero-width chars), and plain visible text.
• Inferred emails (from the optional Google-search discovery layer's result snippets) are stored in a separate inferredEmails field, clearly labeled low-confidence — they are parsed from Google's text, not from the page, and are never treated as scraped contacts.
• No guessing. The actor never fabricates first.last@domain patterns and never claims deliverability. The optional MX/syntax check is a labeled heuristic (DNS MX lookup, no SMTP handshake).
• Placeholder/junk addresses (noreply@, example.com, logo@2x.png, sentry hashes, etc.) are filtered out. Genuinely-absent values are left empty — never faked.

Input

• startUrls — website URLs or bare domains to crawl (primary mode).
• maxDepth — link-hops to follow (default 2).
• maxPagesPerDomain — page cap per domain (default 20).
• prioritizeContactPages — visit contact/about/team/impressum first (default true).
• respectRobotsTxt — obey robots.txt (default true).
• useBrowser — headless-Chromium re-render for JS-only SPA pages (default false).
• checkMx — add the labeled MX/syntax heuristic (default false).
• discoveryMode — off (use your URLs) or google_serp (find candidate sites via a Google search of keywords, then crawl them).
• proxyConfiguration — honored; website fetches are routed through it.
• aiEnhancement / aiModel / aiApiKey — optional company/industry classification using your own LLM key (off by default; never used to invent contacts).

Output & billing

One aggregated row per domain, viewable in the dataset Contact Profiles table. Billing is pay-per-event: domain_result is charged once per domain that yields at least one real contact (email, phone, or social). Empty domains are still returned but not charged.

Techniques

Fetching uses curl_cffi with Chrome TLS impersonation (to pass modern anti-bot/JA3 fingerprinting), BeautifulSoup HTML parsing, phonenumbers for E.164 normalization, and an optional Playwright browser fallback for SPAs.

Keywords

website email scraper, contact scraper, email finder, phone number extractor, social media profile scraper, lead generation, B2B leads, contact crawler, mailto extractor, Cloudflare email decode, business contact data, domain email lookup, website crawler, email harvester alternative, company contact enrichment.