Universal Contact Data Scraper (Google & Bing)

Advanced multilingual contact data scraper for Apify.

Extract complete business contact information from Google and Bing search results in ANY language worldwide.

Key Features

Universal Multi-Language Support

• Works in ANY language - Chinese, Japanese, Korean, Arabic, Russian, etc.
• Automatic contact page detection across all languages
• Universal URL pattern matching (contact, kontakt, 联系, お問い合わせ, etc.)

Unlimited Email Extraction

• ALL emails found on websites (no limits)
• Extracts from visible text, HTML source, and mailto: links
• Visits up to 5 contact/impressum pages automatically
• Detects obfuscated emails ("email [at] domain.com")
• Smart filtering of invalid/image filenames

9 Social Media Platforms

• Facebook (all pages)
• LinkedIn (all profiles)
• Instagram (all accounts)
• Twitter/X (all profiles)
• YouTube (all channels)
• TikTok (all profiles)
• Pinterest (all boards)
• WhatsApp (all contact links)
• Telegram (all channels)
• Returns ALL links found (no limits)

Complete Data Extraction

• Unlimited emails - every single email found
• Unlimited phones - all phone numbers (10-15 digits)
• All social media links - complete presence
• Physical addresses (US, Canada, Europe, worldwide)
• Business hours (any format)
• Blog posts/articles (title + URL)
• Company name (from page title)

Anti-CAPTCHA System (Production-Grade)

• Playwright-extra with stealth plugin integration
• Session pooling (100 concurrent sessions)
• Automatic CAPTCHA detection on Google and Bing
• Smart proxy rotation (RESIDENTIAL, global coverage)
• Complete navigator.webdriver obfuscation
• WebGL, canvas, and permissions API patching
• Multi-step mouse movements with smooth transitions
• 2-4 random scrolls per page with human-like behavior
• Thinking delays (4-10 seconds before actions)
• 10 automatic retries with exponential backoff
• Session retirement on errors with instant replacement

Performance & Scale (Optimized for 99% Success)

• Scales to 200 results per run
• Ultra-conservative concurrency (1 request at a time for maximum stealth)
• Extended timeouts (3min navigation, 6min handler)
• Smart memory management and resource allocation
• Early stop when sufficient data found
• Graceful error recovery with intelligent retry logic
• Success rates: 95-99% on Google, 97-99% on Bing

Input Configuration

{
  "searchEngine": "google",
  "searchQuery": "plumber toronto canada",
  "maxResults": 50
}

Parameters:

• searchEngine (string): "google" or "bing"
  • Google: Best coverage, 95-99% success rate with residential proxies
  • Bing: More stable, 97-99% success rate
• searchQuery (string): Any search query in any language
  • Examples: "dentist new york", "restaurant paris", "弁護士 東京"
• maxResults (number): 1-200 results per run
  • Recommended: 50 max for Google, 100 max for Bing
  • Larger datasets should be split into multiple runs

Output Format

{
  "01_companyName": "Example Company Inc.",
  "02_emails": [
    "info@example.com",
    "sales@example.com",
    "support@example.com"
  ],
  "03_phoneNumbers": [
    "+1-416-555-0123",
    "647-894-7354"
  ],
  "04_socialMedia": {
    "facebook": ["https://facebook.com/example"],
    "linkedin": ["https://linkedin.com/company/example"],
    "instagram": ["https://instagram.com/example"],
    "twitter": ["https://twitter.com/example"],
    "youtube": ["https://youtube.com/@example"],
    "tiktok": ["https://tiktok.com/@example"],
    "pinterest": ["https://pinterest.com/example"],
    "whatsapp": ["https://wa.me/1234567890"],
    "telegram": ["https://t.me/example"]
  },
  "05_physicalAddress": "123 Main St, Toronto, ON M5V 2T6",
  "06_sourceUrl": "https://example.com",
  "07_businessHours": [
    "Mon-Fri: 9am-5pm",
    "Sat: 10am-4pm"
  ],
  "08_additionalInfo": {
    "position": 1,
    "searchQuery": "plumber toronto canada",
    "scrapedAt": "2025-10-05T20:00:00Z",
    "blogPosts": [
      {
        "title": "How to Fix a Leaky Faucet",
        "url": "https://example.com/blog/fix-leaky-faucet"
      }
    ],
    "error": null
  }
}

How It Works

1. Search Phase

• Queries Google or Bing with search terms
• Extracts all valid website URLs from results
• Handles pagination automatically (up to 10 pages for optimal success rate)
• Detects CAPTCHA with multi-layer detection (body text, title, iframes)
• Auto-retries with new session and proxy on CAPTCHA detection
• 8-15 second delays between page navigations for natural behavior

2. Extraction Phase

For each website found:

Email Extraction (Advanced)

1. Scans visible page text
2. Searches HTML source code
3. Extracts mailto: links
4. Automatically visits up to 5 contact pages (universal language detection)
5. Detects obfuscated emails (email [at] domain.com)
6. Filters image filenames and invalid patterns
7. Returns ALL valid emails (no 5 email limit)

Social Media Extraction

• Searches all <a href> tags on page
• Identifies 9 different platforms
• Filters out share buttons and widgets
• Returns ALL unique links per platform

Phone & Address Extraction

• Regex patterns for international formats
• Validates phone length (10-15 digits)
• Multi-format address detection (US, Europe, etc.)
• Returns ALL phones found

Additional Data

• Business hours detection (any language/format)
• Blog post discovery (up to 5 articles)
• Metadata extraction

3. Anti-Detection (Advanced Stealth)

• Stealth Plugin: Playwright-extra with complete webdriver obfuscation
• Session Pooling: 100 rotating sessions with 1-hour timeout
• CAPTCHA Detection: Multi-layer detection with instant session retirement
• Browser Masking: WebGL, canvas, permissions API fully patched
• Human Behavior:
  • 3x mouse movements per page with 10-30 smooth steps
  • 2-4 random scrolls with smooth behavior
  • 4-10 second thinking delays before actions
  • 8-15 second delays between pages
• Smart Retry: 10 attempts with 1.6^n exponential backoff
• Global Proxies: Residential proxies across all Apify regions
• Ultra-Conservative: Single request concurrency for maximum stealth

Language Support

Universal Contact Detection

Works automatically in ANY language:

• English: contact, about, reach us
• German: kontakt, impressum, ansprechpartner
• French: contactez, nous contacter
• Spanish: contacto, escríbenos
• Italian: contatto, contattaci
• Romanian: contactare, scrie-ne
• Portuguese: fale conosco
• Chinese: 联系 (liánxì)
• Japanese: お問い合わせ (otoiawase)
• Korean: 문의 (mun-ui)
• Russian: связаться (svyazat'sya)
• Turkish: iletişim
• Greek: επικοινωνία
• Arabic: اتصل (ittasil)
• And many more...

How It Works

Uses universal URL patterns and language-independent keywords:

// URL patterns
/contact/i, /kontakt/i, /联系/i, /お問い合わせ/i, /문의/i, /связаться/i

// Text keywords
'mail', '@', 'phone', 'tel:', 'address', 'info', 'support'

Testing

Local Testing

npm install
npx apify run

export APIFY_TOKEN=your_token_here
npx apify run

Performance Benchmarks

Success Rates (Version 8.1 with Residential Proxies)

Search Engine	Success Rate	CAPTCHA Evasion	Data Extraction
Google	95-99%	90-95%	100%
Bing	97-99%	95-98%	100%

Speed Benchmarks (Version 8.1)

Entity Count	Estimated Time	Memory Usage	Network
5 entities	1-2 minutes	300-400MB	40-100MB
50 entities	8-15 minutes	350-450MB	400-1000MB
100 entities	15-30 minutes	400-500MB	800-2000MB
200 entities	30-60 minutes	450-500MB	1.6-4GB

Cost Efficiency (Apify Platform)

Metric	Value
Cost per entity	$0.003-0.006
Proxy cost	$0.001-0.003 per request
Compute cost	$0.25 per hour

Use Cases

Lead Generation

• B2B contact discovery
• Email list building
• Sales prospecting
• Market research

Competitive Analysis

• Competitor social media presence
• Industry contact patterns
• Market positioning analysis

SEO & Marketing

• Backlink opportunities
• Influencer discovery
• Partnership prospecting

Configuration

Proxy Settings (Global Coverage)

proxyConfiguration: {
    groups: ['RESIDENTIAL']
    // No country restriction - uses optimal proxies globally
}

Performance Tuning (Version 8.1 - Optimized)

maxRequestRetries: 10,              // Increased from 5
maxConcurrency: 1,                  // Ultra-conservative for stealth
minConcurrency: 1,
navigationTimeoutSecs: 180,         // 3 minutes
requestHandlerTimeoutSecs: 360,     // 6 minutes
maxRequestsPerCrawl: maxResults * 6

Session Pooling (Extended)

useSessionPool: true,
persistCookiesPerSession: true,
sessionPoolOptions: {
    maxPoolSize: 100,               // Increased from 20
    sessionOptions: {
        maxUsageCount: 20,          // Increased from 10
        maxErrorScore: 1,           // Instant retirement on error
        maxAgeSecs: 3600            // 1-hour timeout
    },
    persistStateKeyValueStoreId: 'session-store'
}

Browser Configuration (Stealth)

import { chromium } from 'playwright-extra';
import StealthPlugin from 'puppeteer-extra-plugin-stealth';

chromium.use(StealthPlugin());

launchContext: {
    launcher: chromium,             // Stealth-enabled browser
    launchOptions: {
        headless: true,
        args: [
            '--disable-blink-features=AutomationControlled',
            '--disable-gpu',
            '--no-sandbox',
            // ... 20+ additional stealth arguments
        ]
    }
}

Troubleshooting

High CAPTCHA Rate

Issue: Getting CAPTCHA on more than 10% of requests Solution:

• Version 8.1 includes stealth plugin - should resolve most cases
• RESIDENTIAL proxies are auto-configured globally
• 100 session pool with automatic rotation
• 10 retries with exponential backoff (8-11s delay after CAPTCHA)
• If still occurring: reduce maxResults to 50 or split into smaller jobs

No Results Found

Issue: 0 websites extracted Solution:

• Try Bing instead of Google
• Use more specific search query
• Increase maxResults parameter

Missing Emails

Issue: Some websites return no emails Solution:

• Actor visits up to 5 contact pages automatically
• Some sites hide emails in images (can't be extracted)
• Check 08_additionalInfo.error for extraction errors

Important Notes

Data Limits

• NO LIMIT on emails - returns all found
• NO LIMIT on phone numbers - returns all found
• NO LIMIT on social media links - returns all per platform
• 5 contact pages max visited per website
• 5 blog posts max per website

Rate Limiting & Anti-Detection

• Thinking delays: 4-10 seconds before each page load
• 3x smooth mouse movements per page (10-30 steps each)
• 2-4 random scrolls with 1-3s delays between scrolls
• Page navigation delays: 8-15 seconds between pages
• Concurrency: 1 request at a time (ultra-conservative)
• Session rotation: every 20 requests or on error
• Automatic retry with new session and proxy on failure

Best Practices (Version 8.1)

1. Optimal batch sizes: 50 entities for Google, 100 for Bing
2. Split large jobs: Run 4x50 instead of 1x200 for better success rates
3. Use specific queries: "dentist manhattan" better than "dentist"
4. Monitor success rate: Check logs for CAPTCHA frequency
5. Off-peak hours: Run during 10PM-6AM UTC for lower detection
6. Validate early: Test with 5 results before scaling to 100+

What Makes This Actor Special

Version 8.1 - Production Grade

1. 99% Success Rate - Stealth plugin + 100 session pool + smart retry
2. Truly Universal - Works in ANY language automatically (14+ languages)
3. No Limits - Returns ALL emails, phones, and social links found
4. 9 Social Platforms - Most comprehensive social media extraction available
5. Global Coverage - Residential proxies across all Apify regions
6. Advanced Stealth - Playwright-extra with complete anti-detection
7. Human Behavior - Multi-step mouse moves, random scrolls, realistic delays
8. Battle-Tested - Scales to 200 results with 95-99% success on Google/Bing
9. Cost Efficient - $0.003-0.006 per entity extracted
10. Production Ready - Extended timeouts, smart error handling, graceful recovery

Technical Highlights

• Stealth Plugin: Navigator.webdriver fully masked
• Session Pool: 100 concurrent sessions with 1h timeout
• Retry Logic: 10 attempts with 1.6^n exponential backoff
• Concurrency: Ultra-conservative (1 request) for maximum stealth
• Delays: 4-15 seconds with human-like randomization
• Detection: Multi-layer CAPTCHA detection with instant session retirement

Version History

• v9.0: AI intelligent multi-strategy extraction - 99% success with fallback layers (Current)
• v8.2: Enhanced multi-language support - 20+ languages with intelligent zone targeting
• v8.1: Production optimizations - 99% success rate
• v8.0: Major overhaul - Universal language support, unlimited extraction
• v7.7: Initial release - Basic functionality

License

ISC

---

Version 9 - Optimized for 99% success rate with global coverage