Answer The Public Scraper

Extract keyword insights from AnswerThePublic.com at scale. This Apify Actor automatically scrapes search data from multiple providers including Google, Bing, YouTube, TikTok, Instagram, Amazon, and ChatGPT.

Features

• Multi-Provider Data: Get keyword data from Google, Bing, YouTube, TikTok, Instagram, Amazon, and ChatGPT in a single search
• Cloudflare Bypass: Built-in Playwright automation with residential proxies to bypass Cloudflare protection
• Smart Account Management: Automatically creates and rotates accounts with 48-hour cooldown cycles
• Auto-Scaling: Intelligently calculates accounts needed based on your keyword list (1 account per 3 keywords)
• Complete Results: Waits for all providers to finish before returning comprehensive data

Quick Start

1. Add this Actor to your Apify account
2. Configure your keyword list in the input
3. Run the Actor
4. Download your results from the dataset

That's it! The Actor handles everything else automatically.

Input

Keywords (Required)

List of keywords to search. The Actor automatically creates the right number of accounts needed (1 account handles 3 keywords per 48-hour cycle).

Language & Region (Optional)

• Language: en, es, fr, de, it, pt, nl, pl, ru (default: en)
• Region: us, uk, ca, au, in, de, fr, es, it (default: us)

Example

{
  "keywords": [
    "digital marketing",
    "content strategy",
    "SEO optimization",
    "social media management",
    "email marketing",
    "conversion optimization"
  ],
  "language": "en",
  "region": "us"
}

Note: For 6 keywords, the Actor will automatically create 2 accounts (6 ÷ 3 = 2). Each account can perform 3 searches per 48-hour cycle.

Output

Results are saved to your Apify dataset and include comprehensive data from all providers:

{
  "keyword": "digital marketing",
  "language": "en",
  "region": "us",
  "parentSearchId": "uuid-here",
  "completedAt": "2025-01-16T12:00:00.000Z",
  "data": {
    "search_engine": {
      "gweb": {
        "questions": ["what is digital marketing", "how does digital marketing work"],
        "prepositions": ["digital marketing for beginners", "digital marketing in 2024"],
        "comparisons": ["digital marketing vs traditional marketing"],
        "alphabeticals": ["digital marketing agency", "digital marketing basics"]
      },
      "bing": { /* similar structure */ }
    },
    "social_media": {
      "youtube": { /* video-related queries */ },
      "tiktok": { /* trending content ideas */ },
      "instagram": { /* visual content suggestions */ }
    },
    "shopping": {
      "amazon": { /* product-related queries */ }
    },
    "ai": {
      "chatgpt": { /* AI-generated insights */ }
    },
    "max_search_volume": 12000,
    "max_cost_per_click": 8.50
  }
}

Export Formats

Download your data in:

• JSON - Full structured data
• CSV - Flattened for spreadsheets
• Excel - Ready for analysis
• HTML - Human-readable reports

How It Works

1. Smart Scaling: Analyzes your keyword list and calculates how many accounts are needed
2. Account Creation: Automatically creates accounts using Playwright with Apify's residential proxies to bypass Cloudflare
3. 48-Hour Cycles: Each account can perform 3 searches per 48-hour period
4. Persistent Storage: Accounts are stored in Apify's key-value storage and reused across runs
5. Parallel Processing: Processes multiple keywords simultaneously for faster results

Account Lifecycle Example

First Run (10 keywords):

• Calculates: 10 ÷ 3 = 4 accounts needed
• Creates 4 new accounts with unique proxies
• Processes all 10 keywords
• Stores accounts for future use

Second Run (5 keywords, same 48hr cycle):

• Checks existing accounts
• Account 1: used 3/3 ❌
• Account 2: used 3/3 ❌
• Account 3: used 2/3 ✅ (1 slot remaining)
• Account 4: used 2/3 ✅ (1 slot remaining)
• Creates 1 new account for the 5th keyword

Third Run (next 48hr cycle):

• All accounts reset
• Reuses existing 5 accounts

Use Cases

• SEO Research: Find question-based keywords for content creation
• Content Planning: Discover what people are asking about your topic
• Competitive Analysis: Compare keyword trends across regions and languages
• Social Media Strategy: Get insights into viral content ideas from TikTok, YouTube, Instagram
• E-commerce: Find product-related questions from Amazon shoppers
• AI Content: Leverage ChatGPT insights for content ideation

Performance

• Speed: Processes 3-10 keywords per minute (depending on AnswerThePublic's response time)
• Cloudflare Success Rate: >95% with residential proxies
• Concurrent Searches: Multiple keywords processed in parallel

Pricing

This Actor uses a Pay-Per-Event pricing model, which means you only pay for what you use. No platform compute costs - just simple, predictable pricing based on actual results.

Event-Based Pricing

Event Type	Description	Cost per Event
Actor Start	One-time charge per run	$0.01
Account Created	Each new ATP account created	$0.02
Search Query	Each successful keyword search	$0.05
Result Item	Each data point scraped	$0.001

Cost Examples

Small Job (10 keywords, no new accounts needed)

• Actor start: $0.01
• 10 search queries: 10 × $0.05 = $0.50
• ~1,000 results: 1,000 × $0.001 = $1.00
• Total: ~$1.51

Medium Job (50 keywords, 17 new accounts)

• Actor start: $0.01
• 17 new accounts: 17 × $0.02 = $0.34
• 50 search queries: 50 × $0.05 = $2.50
• ~5,000 results: 5,000 × $0.001 = $5.00
• Total: ~$7.85

Large Job (100 keywords, 34 new accounts)

• Actor start: $0.01
• 34 new accounts: 34 × $0.02 = $0.68
• 100 search queries: 100 × $0.05 = $5.00
• ~10,000 results: 10,000 × $0.001 = $10.00
• Total: ~$15.69

Why Pay-Per-Event?

• Fair Pricing: Only pay for successful operations
• Predictable Costs: Know exactly what each search will cost
• No Compute Fees: No platform usage costs for proxies or runtime
• Transparent: Clear breakdown of charges per run
• Scalable: Cost grows linearly with usage

Account Reuse = Lower Costs

After the first run, accounts are stored and reused in future runs. This means:

• 48-hour cycles: Accounts reset every 48 hours (3 searches per account)
• Fewer account creation charges: Subsequent runs reuse existing accounts
• Example: Running 100 keywords today and 100 keywords in 3 days will only create accounts once, saving $0.68 on the second run

Support

Found a bug or have a feature request? Open an issue on GitHub or contact support through Apify.