Competitor Analyzer

Extract and analyze the top organic and paid competitors for any domain. This actor provides comprehensive competitor data including common keyword terms and competitive ranking scores.

Features

• 🔍 Competitor Discovery: Identify top organic and paid competitors for any domain
• 📊 Competitive Analysis: Get common keyword terms and competitive rank scores
• 🎯 Dual Channel Insights: Separate organic (SEO) and paid (PPC) competitor data
• 🚀 Fast Execution: Optimized for quick data extraction (typically completes in 10-20 seconds per domain)
• 🔄 Optional Proxy Support: Can use Apify proxy for IP rotation if needed
• 🌐 Multi-Domain Support: Process multiple domains in a single run
• 🔗 URL Parsing: Automatically extracts domains from full URLs

Input

The actor accepts the following input parameters:

Parameter	Type	Required	Default	Description
domains	Array	Yes	-	Array of domains or URLs to analyze (e.g., ["example.com", "https://www.site.com/page"])
useProxy	Boolean	No	false	Enable Apify proxy rotation for requests (recommended for multiple domains)

URL/Domain Handling

The actor automatically extracts clean domain names from various input formats:

• Full URLs: https://www.example.com/page → example.com
• URLs with protocols: http://example.com → example.com
• Domains with www: www.example.com → example.com
• Clean domains: example.com → example.com
• URLs with ports: example.com:8080 → example.com

Example Input - Multiple Domains

{
  "domains": [
    "domaincoasters.com",
    "https://www.example.com",
    "http://another-site.com/page"
  ],
  "useProxy": true
}

Example Input - Single Domain

{
  "domains": ["domaincoasters.com"],
  "useProxy": false
}

Output

The actor extracts and returns detailed competitor data in the following format:

Competitor Fields

Field	Type	Description
sourceDomain	String	The domain being analyzed
competitorType	String	Type of competitor: "organic" or "paid"
competitorDomain	String	The competitor's domain
commonTerms	Number	Number of common keywords shared with the source domain
rank	Number	Competitive rank score (0-1, higher is more competitive)

Summary Data

The actor also provides a summary record with aggregate statistics:

{
  "_summary": true,
  "sourceDomain": "example.com",
  "totalOrganicCompetitors": 5,
  "totalPaidCompetitors": 5
}

Output Example

[
  {
    "sourceDomain": "domaincoasters.com",
    "competitorType": "organic",
    "competitorDomain": "serpnames.com",
    "commonTerms": 49,
    "rank": 0.031621046
  },
  {
    "sourceDomain": "domaincoasters.com",
    "competitorType": "organic",
    "competitorDomain": "spamzilla.io",
    "commonTerms": 52,
    "rank": 0.028129926
  },
  {
    "sourceDomain": "domaincoasters.com",
    "competitorType": "paid",
    "competitorDomain": "seo.domains",
    "commonTerms": 8,
    "rank": 0.011008541
  },
  {
    "_summary": true,
    "sourceDomain": "domaincoasters.com",
    "totalOrganicCompetitors": 5,
    "totalPaidCompetitors": 5
  }
]

Use Cases

• Competitive Analysis: Identify who you're competing against in both organic and paid search
• Market Research: Understand the competitive landscape in your niche
• SEO Strategy: Find competitors ranking for similar keywords
• PPC Intelligence: Discover competitors bidding on related terms
• Partnership Opportunities: Identify potential partners or acquisition targets
• Market Entry: Assess competition before entering a new market

How It Works

1. URL Processing: Cleans and extracts domain names from provided URLs or domains
2. Multi-Domain Loop: Processes each domain sequentially with a small delay between requests
3. Navigation: Navigates to the platform overview page for each specified domain
4. API Interception: Intercepts the backend API response that contains competitor data
5. Data Extraction: Separates and structures organic and paid competitor information
6. Dataset Storage: Saves all competitor records to the Apify dataset

The actor uses Playwright to load the page and intercept network responses, ensuring accurate and up-to-date data extraction.

Performance

• Speed: Typically completes in 10-20 seconds per domain
• Multi-Domain: Processes domains sequentially with 2-second delay between domains
• Data Volume: Extracts top 5 organic and top 5 paid competitors per domain (default)
• Memory Usage: Low - optimized for efficient execution
• Compute Units: Minimal usage due to fast execution time (scales linearly with number of domains)

Limitations

• Extracts the top competitors available (typically 5 organic + 5 paid)
• Requires the domain to have competitive data available on the platform
• Some domains may not have sufficient competitive data
• Network timeouts may occur for very slow connections

Error Handling

The actor includes robust error handling:

• Invalid Domain: Returns an error if no domain is provided
• No Data Found: Returns a record with error details if no competitors are captured
• Network Issues: Continues execution even if network idle timeout occurs
• Parsing Errors: Logs errors and continues with remaining data

Tips for Best Results

1. Domain Format: Can use any format - full URLs, domains with/without www, with/without protocols - all will be automatically cleaned
2. Proxy Usage: Enable proxy when processing multiple domains to avoid rate limiting
3. Batch Size: For large lists, consider breaking into smaller batches of 10-20 domains per run
4. Data Validation: Check for the _summary record per domain to verify data completeness
5. Retry Logic: If no data is returned for a domain, wait a few minutes and retry
6. Result Filtering: Filter results by sourceDomain to separate results when processing multiple domains

Integration

Using with Apify API

const client = new ApifyClient({
    token: 'YOUR_APIFY_TOKEN',
});

// Process multiple domains
const run = await client.actor('YOUR_ACTOR_ID').call({
    domains: [
        'example.com',
        'https://www.competitor.com',
        'another-site.com'
    ],
    useProxy: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();

// Filter organic competitors for a specific domain
const organicCompetitors = items.filter(item =>
    item.sourceDomain === 'example.com' &&
    item.competitorType === 'organic'
);
console.log(organicCompetitors);

// Filter paid competitors
const paidCompetitors = items.filter(item =>
    item.sourceDomain === 'example.com' &&
    item.competitorType === 'paid'
);
console.log(paidCompetitors);

Using with Apify Console

1. Navigate to the actor in Apify Console
2. Click "Try it"
3. Enter your target domain(s)
4. Click "Start"
5. View results in the Dataset tab with pre-configured views:
   • Overview: All competitors (organic + paid)
   • Organic Competitors: SEO competitors only
   • Paid Competitors: PPC competitors only

Support

For issues, questions, or feature requests, please contact support or create an issue in the actor's repository.

Version History

• 0.0.1 - Initial release with core competitor extraction functionality

License

This actor is licensed under the ISC License.