SEO Competitor Content Analyzer

Analyze competitor web pages for comprehensive SEO content metrics. Built as an Apify Actor.

Features

• Title & Meta Extraction — Page title, meta description, meta keywords, canonical URL, robots meta
• Heading Analysis — All headings (h1–h6) extracted with their levels and text
• Word Count & Frequency — Total word count plus top N most frequent words (stopwords excluded)
• Topic Extraction — Key topics identified from heading text using significant phrases
• Readability Scoring — Simple Flesch Reading Ease–style score based on average sentence length and average word length
• Links & Images — Count of links and images, with up to 50 sample entries
• Deduplication — Duplicate URLs are automatically deduplicated before analysis
• Graceful Abort — Handles platform stop signals cleanly
• Webhook Support — Optional webhook URL notified on completion

Input

Field	Type	Required	Default	Description
urls	array[string]	✅	—	List of competitor or target URLs to analyze
extractTopics	boolean	❌	true	Extract key topics from heading text
analyzeKeywords	boolean	❌	true	Analyze meta keywords and keyword patterns
wordCount	boolean	❌	true	Compute word count and word frequency
maxResults	integer	❌	20	Max top keywords/words per URL
webhookUrl	string	❌	—	Optional webhook to notify on completion

Example input

{
  "urls": ["https://example.com"],
  "extractTopics": true,
  "analyzeKeywords": true,
  "maxResults": 5
}

Output

Each analyzed URL produces a dataset entry with these fields:

Field	Type	Description
timestamp	string	UTC analysis timestamp
url	string	The analyzed URL
title	string	HTML <title> tag content
wordCount	integer	Total words on the page
headings	array	All headings (h1–h6) as {level, text} objects
keywords	array	Extracted keywords (meta + frequent words)
topics	array	Key topics identified from headings
readabilityScore	number	Readability score (0–100, higher = easier)
metaDescription	string	Meta description content
wordFrequency	array	Top N frequent words with counts
canonicalUrl	string	Canonical URL if set
robotsMeta	string	Robots meta tag content
metaKeywords	array	Meta keywords if present
linksCount	integer	Number of links found
imagesCount	integer	Number of images found
statusCode	integer	HTTP status code
contentType	string	Response content-type

Usage

Local development

# Install dependencies
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Run locally
apify run --purge

Deploy to Apify

# Login (first time only)
apify login

# Push and build
apify push

API call

curl -X POST "https://api.apify.com/v2/acts/YOUR_ACTOR_ID/runs" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -d '{
    "urls": ["https://example.com"],
    "extractTopics": true,
    "analyzeKeywords": true,
    "maxResults": 20
  }'

Use cases

• Competitive content audit — Analyze how competitors structure their pages
• SEO gap analysis — Identify topics and keywords your content is missing
• Content benchmarking — Measure word count, readability, and heading usage across competitors
• Technical SEO check — Verify canonical URLs, meta tags, and robots directives

Technical stack

• Apify SDK for Python
• HTTPX — Async HTTP client
• Beautiful Soup with lxml — HTML parsing

License

MIT