Site Contact, Social, Tech Stack & AI Markdown Extractor

Website Contact, Tech Stack & AI RAG Markdown Extractor

A lightweight, zero headless browser website scraper, B2B contact finder, and lead enrichment Apify Actor. Acts as a fast, deterministic BuiltWith alternative and AI RAG Markdown converter—crawling public sites respectfully (robots.txt-compliant, SSRF-safe, zero anti-bot evasion) in pure HTTP/Python.

Key Capabilities

• B2B Contact Finder & Lead Enrichment: Deduplicated emails (filtering asset filenames and placeholder domains) plus normalized phone numbers (tel:, international +60/E.164, local formats).
• Messaging & Social Profiles: WhatsApp (wa.me, api.whatsapp.com), Telegram (t.me), Line, LinkedIn, X/Twitter, Instagram, Facebook, TikTok, YouTube, and GitHub.
• Tech Stack Detection (BuiltWith Alternative): Detects Shopify, WooCommerce, WordPress, Webflow, Wix, Next.js, Astro, Stripe, Google Tag Manager, Google Analytics, Meta Pixel, and Cloudflare without launching Chromium.
• Structured Metadata: application/ld+json Schema.org blocks, <title>, meta description, and <html lang>.
• Clean AI RAG Markdown: Strips <script>, <style>, <nav>, <footer>, and <aside> while preserving #/##/### headings, lists, and links for LLM/RAG ingestion.

Pricing & Value

Transparent Pay-Per-Event (PPE) pricing with zero headless browser compute overhead:

• $0.005 per Actor run start
• $0.003 per domain extracted

Sample Output JSON

{
  "domain": "mycalc-hub.pages.dev",
  "url": "https://mycalc-hub.pages.dev",
  "title": "MyCalc Hub — Free Financial & Lifestyle Calculators",
  "description": "Fast, privacy-first calculators for tax, loan, and savings planning.",
  "language": "en",
  "pagesCrawled": 5,
  "emails": ["hello@mycalc-hub.pages.dev"],
  "phones": [],
  "socials": {
    "github": ["https://github.com/mycalc-hub"]
  },
  "messaging": {},
  "techStack": ["Cloudflare"],
  "jsonLd": [
    {
      "@context": "https://schema.org",
      "@type": "WebSite",
      "name": "MyCalc Hub"
    }
  ],
  "pages": [
    {
      "url": "https://mycalc-hub.pages.dev",
      "title": "MyCalc Hub — Free Financial & Lifestyle Calculators",
      "emails": ["hello@mycalc-hub.pages.dev"],
      "phones": [],
      "markdown": "# MyCalc Hub\n\nFast, privacy-first calculators..."
    }
  ],
  "errors": []
}

API & n8n Automation Integration

curl Sync Run

curl -X POST "https://api.apify.com/v2/acts/YOUR_USERNAME~site-intel-markdown-extractor/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "startUrls": [{ "url": "https://mycalc-hub.pages.dev" }],
    "maxPagesPerDomain": 5,
    "includeMarkdown": true
  }'

n8n HTTP Request Node

• Method: POST
• URL: https://api.apify.com/v2/acts/YOUR_USERNAME~site-intel-markdown-extractor/run-sync-get-dataset-items
• Authentication: Query Auth (token = YOUR_APIFY_TOKEN)
• Send Body (JSON):

{
  "startUrls": [{ "url": "{{ $json.websiteUrl }}" }],
  "maxPagesPerDomain": 5,
  "includeMarkdown": true,
  "respectRobotsTxt": true
}

Local Testing

$python3 -m unittest discover -s tests -v

Run locally with a sample input.json:

echo '{"startUrls": [{"url": "https://mycalc-hub.pages.dev"}], "maxPagesPerDomain": 3}' > input.json
python3 -m src.main