Google Ads Transparency Scraper

This Apify actor scrapes Google's Ad Transparency Center to check if domains are running ads and extracts ad creatives with OCR text extraction.

Features

• Checks if domains are running Google Ads
• Extracts ad creatives (images and videos)
• Performs OCR on image ads to extract text (capped per domain, configurable)
• Handles YouTube video thumbnails and links
• Provides detailed statistics and progress tracking
• Configurable concurrency for faster processing
• Robust error handling and retries

Input

The actor accepts the following input parameters:

{
    "domains": [
        "example.com",
        "example.org"
    ],
    "maxConcurrency": 1, // Optional, default: 1, max: 10
    "timePeriod": "Last 30 days", // Optional, default: "Last 30 days"
    "startDate": "2018-06-08", // Optional, required if timePeriod is "Custom" (format: YYYY-MM-DD)
    "endDate": "2018-06-15", // Optional, required if timePeriod is "Custom" (format: YYYY-MM-DD)
    "platform": "All", // Optional, default: "All" - filter by Google platform
    "ocrMaxImages": 10, // Optional, default: 10, 0 disables OCR, max 100
    "healthCheckOnly": false // Optional, default: false - skip scraping, just verify the fast-path API against canary domains
}

• domains: Array of domains to check for ads (required)
• maxConcurrency: Maximum number of domains to process concurrently (optional)
• timePeriod: Time period to check ads for (optional). Options:
  • "Today" - Check ads from today
  • "Yesterday" - Check ads from yesterday
  • "Last 7 days" - Check ads from the last 7 days
  • "Last 30 days" - Check ads from the last 30 days (default)
  • "Custom" - Use custom date range (requires startDate and endDate)
• startDate: Start date for custom date range (format: YYYY-MM-DD). Required if timePeriod is "Custom"
• endDate: End date for custom date range (format: YYYY-MM-DD). Required if timePeriod is "Custom"
• ocrMaxImages: Maximum number of image ads per domain to extract text from with OCR (optional, default 10, 0 disables OCR, max 100). OCR is the slowest and costliest step, so it is capped; ad_texts therefore holds text for at most this many images and can be shorter than creatives. Raise it if you need the text of every image.
• platform: Google platform to filter ads by (optional). Options:
  • "All" - Check ads across all platforms (default)
  • "SEARCH" - Google Search ads only
  • "YOUTUBE" - YouTube ads only
  • "SHOPPING" - Google Shopping ads only
  • "MAPS" - Google Maps ads only
  • "PLAY" - Google Play ads only

Output

The actor saves results to its default dataset. Each item contains:

{
    "domain": "example.com",
    "ads_running": true,
    "creatives": [
        {
            "type": "image",
            "url": "https://..."
        },
        {
            "type": "video",
            "url": "https://..."
        }
    ],
    "ad_texts": [
        "Extracted text from image 1",
        "Extracted text from image 2"
    ],
    "time_period": "Last 30 days",
    "start_date": null,
    "end_date": null,
    "platform": "All",
    "fetch_method": "fast_api", // "fast_api" (direct API, low cost) or "selenium" (browser fallback)
    "error": null, // Error message if scraping failed
    "timestamp": "2024-03-21T12:34:56.789Z"
}

Fast path and health checks

For the default settings (platform: "All", timePeriod: "Last 30 days"), the actor calls Google's Ads Transparency Center API directly instead of driving a full Chrome browser, cutting compute cost roughly 10x. Any other combination of filters, or any failure of that direct call, automatically falls back to the original Selenium-based scraping -- so behavior is unchanged, only the cost for the common case drops.

Because this relies on an undocumented Google endpoint, it's worth monitoring. Set input healthCheckOnly: true to skip scraping and instead verify the fast path against known canary domains, writing a pass/fail status to the dataset and failing the run (so Apify's run-failure notifications fire) if it's down. Set this up as a separate Apify Schedule (e.g. every few hours) running the same actor with {"domains": ["example.com"], "healthCheckOnly": true} as input (domains is still required by the input schema but is ignored in health-check mode).

Google occasionally rate-limits (HTTP 429) shared datacenter IPs. The health check retries across several spaced-out rounds and treats pure rate-limiting as a transient warning (dataset status blocked, run still succeeds, since real runs fall back to Selenium). It fails the run only when the endpoint answers in an unexpected shape or is gone (status degraded).

Usage

1. Create a new task for the actor
2. Provide input:

   {
       "domains": ["example.com"],
       "maxConcurrency": 1,
       "timePeriod": "Last 7 days",
       "platform": "YOUTUBE"
   }
   

3. Run the task
4. Get results from the dataset

Performance and Limits

• Memory: 4096 MB
• Timeout: 4 hours
• Concurrency: 1-10 domains in parallel
• Rate limiting: 2 second delay between requests

Dependencies

• Python 3.9
• Chrome browser
• Tesseract OCR
• Key Python packages:
  • selenium
  • pytesseract
  • aiohttp
  • Pillow
  • apify-client

Error Handling

The actor implements robust error handling:

• Automatic retries for transient errors
• Graceful degradation for OCR failures
• Detailed error reporting in output
• Progress tracking and statistics

Development

1. Install dependencies:

   $pip install -r requirements.txt
   

2. Install system dependencies:

   $apt-get install tesseract-ocr
   

3. Run locally:

   $python main.py
   

License

Apache 2.0