SEO Competitor Page Structure Analyzer

Compare the structure of up to 50 public HTML pages you supply in a single run. Export each page's title, main-content word count, ordered H1/H2/H3 headings and collection time, together with a comparison summary.

Price: $0.004 per successfully analyzed page + $0.00005 per run. A run with 10 successful pages costs $0.04005. Failed, blocked, empty and duplicate pages have no page-analysis charge. The startup charge still applies when a run returns no successful pages. Standard runs use this event price; there is no separate search-provider, proxy or LLM fee. Any taxes and your Apify subscription are separate.

What this release does

• Reads the exact authorized URLs you provide, once. The default limit is 10; the maximum is 50.
• Extracts the HTML <title> and an approximate main-content word count.
• Returns H1, H2 and H3 nodes in document order, with their level, text, 1-based index and parent index.
• Separates navigation, sidebars, common footer/feedback modules, scripts and hidden content where the HTML identifies them.
• Records the supplied URL, final URL, HTTP status, timestamp and extraction warnings.
• Returns explicit error rows for inaccessible pages. Their word count is null, not zero.
• Compares successful pages using minimum, median and maximum main-content word counts and per-page heading counts.
• Separately reports search activity, crawl measurements and Actor event fees.

This release requires URLs. Keyword, country and language do not enable search discovery. No search provider with suitable rights has been connected to this release, so searchRank is always null. Input order is returned only as inputPosition.

Real search rankings depend on location, language, personalization and the time of the search. This Actor does not claim to retrieve Google's first ten results or to reproduce anyone's search rankings.

Quick start

1. Paste public URLs into Competitor page URLs.
2. Confirm that automated access and your intended reuse are authorized.
3. Optionally add a comparison keyword, country context and word-segmentation language.
4. Set the maximum page count and an Apify run budget, then start the Actor.
5. Open the Dataset for individual pages and the Comparison summary output for aggregate metrics.

Example input

The following Python documentation pages are used as accessible examples; see the source-license notices below.

{
  "competitorUrls": [
    "https://docs.python.org/3/tutorial/datastructures.html",
    "https://docs.python.org/3/tutorial/controlflow.html"
  ],
  "permissionConfirmed": true,
  "keyword": "Python tutorial",
  "country": "US",
  "language": "en",
  "maxResults": 10,
  "requestTimeoutSecs": 15
}

Input reference

Field	Default	Meaning
competitorUrls	Required	1–50 exact HTTP(S) URLs. Standard ports only; credentials in URLs are rejected.
permissionConfirmed	false	Must be explicitly true after confirming automated access and intended reuse are authorized.
keyword	Empty	Comparison label, up to 200 characters. No search or keyword-ranking inference.
country	Empty	Optional two-letter context code such as US. It does not geolocate requests or filter pages.
language	en	Supported BCP47 locale for Unicode word segmentation. It does not translate or filter pages.
maxResults	10	Maximum unique supplied URLs to process, between 1 and 50.
requestTimeoutSecs	15	Per HTTP request, including body download, between 5 and 30 seconds.

URL fragments are removed and exact normalized duplicates are processed once. Query parameters are preserved because they can change page content. Invalid input, missing permission confirmation and unsupported URLs are rejected before crawling. A keyword-only request cannot run.

Outputs

Dataset: one row per unique attempted URL

Field	Interpretation
sourceUrl, finalUrl	Supplied normalized URL and the destination after permitted redirects.
sourceType	user_url.
inputPosition	1-based position in the normalized input; never a search rank.
searchRank	Always null in this release.
status	success or error.
title	Document title, or null if absent or the page failed.
mainWordCount	Unicode word-like segments in the selected content; null for errors.
headings	Ordered H1/H2/H3 objects: index, level, text, parentIndex, textTruncated.
headingCounts	Full H1/H2/H3 counts in the selected scope, including empty heading nodes.
headingOutline	Readable outline of the retained heading nodes.
crawledAt	UTC collection timestamp.
context	Your keyword, country and language context.
extraction	Content-selection method, effective segmentation locale and truncation limits/flags.
warnings	Heuristic, sparse-content, heading-level or truncation diagnostics.
metrics	Page/robots request counts, consumed response-body bytes and elapsed time.
errorCode, errorMessage	Explicit failure diagnostics when no usable page analysis was produced.

parentIndex points to the nearest preceding heading with a lower level. A missing level is not invented. Empty heading nodes remain visible and produce EMPTY_HEADING. At most 500 headings and 500 characters per heading/title are exported, with explicit truncation flags; full level counts remain available.

Example result

This is a real Python documentation result from the release QA. The excerpt shows the first three of its 14 heading records; the complete example is supplied in the project. Counts and timestamps are observations from that run, and the source can change.

{
  "sourceUrl": "https://docs.python.org/3/tutorial/datastructures.html",
  "searchRank": null,
  "status": "success",
  "title": "5. Data Structures — Python 3.15.0 documentation",
  "mainWordCount": 4029,
  "headingCounts": {
    "h1": 1,
    "h2": 8,
    "h3": 5
  },
  "crawledAt": "2026-10-10T19:59:20.975Z",
  "headings": [
    {
      "index": 1,
      "level": 1,
      "text": "5. Data Structures",
      "parentIndex": null,
      "textTruncated": false
    },
    {
      "index": 2,
      "level": 2,
      "text": "5.1. More on Lists",
      "parentIndex": 1,
      "textTruncated": false
    },
    {
      "index": 3,
      "level": 3,
      "text": "5.1.1. Using Lists as Stacks",
      "parentIndex": 2,
      "textTruncated": false
    }
  ]
}

Key-value outputs

• COMPARISON: successful/failed counts, per-page metrics and minimum/median/maximum word counts. Failed pages are excluded from statistics.
• RUN-SUMMARY: processing status, unprocessed pages, zero search requests/cost, crawl time/bytes, a clearly labeled infrastructure estimate and Actor event fees. Event totals cover the run, including verified earlier results after a restart; crawl measurements identify when they cover only the current process. The final Apify run billing record is authoritative.

Dataset JSON retains the heading arrays and parent relationships. CSV is useful for the overview columns; use JSON when consuming the complete hierarchy.

How main content is selected

The Actor prefers a usable <main>, then [role="main"], then an <article>. When these are absent it uses a largest-content fallback and adds a warning. Block elements create word boundaries, and Intl.Segmenter counts Unicode word-like segments in the selected language. Code examples and tables that belong to the main article can contribute to the count.

Navigation and identifiable supplementary modules are removed. Article headers carrying the page title are retained. Common permalink decorations are removed from headings. The result is an HTML-based estimate, not a rendered-pixel or editorial assessment. External CSS, JavaScript, cookie interactions and lazy-loaded content are not executed. Sparse but readable static pages may succeed with SPARSE_CONTENT; empty or recognized JavaScript-only placeholders fail without a page-analysis fee.

A higher word count or a particular heading pattern does not establish better quality, ranking potential or the right length for your own page.

Access and failure behavior

Use only pages you are authorized to access automatically and analyze. A permissive robots.txt is a technical access signal, not a license to reuse content. Website terms, copyright and any provider agreement remain relevant.

The Actor always checks robots rules, identifies itself, spaces same-origin requests at least one second apart and respects supported crawl-delay values. It does not use authenticated sessions, residential proxies, CAPTCHA solving or block evasion. It does not crawl additional links, monitor changes or schedule later checks.

Supported targets are public HTTP(S) HTML/XHTML pages on standard ports. Private/special-use addresses and redirects to them are rejected. Every page redirect requires safe DNS and destination robots checks. Up to five redirects are supported. Cross-origin robots-file redirects, denied/unavailable robots files and crawl-delay values above 30 seconds fail closed. A robots 404/410 means no robots file is present.

HTML responses are capped at 2 MiB and robots files at 512 KiB, for both received and decoded bytes. Parsed HTML is limited to 30,000 DOM nodes (including text and comment nodes) and depth 128; more complex documents return CONTENT_TOO_COMPLEX. HTTP errors, access challenges, unsupported encodings, non-HTML responses, oversized/overcomplex responses and timeouts produce error rows. There is no page-analysis charge for these rows. Different URLs reaching an already analyzed final URL produce an explicit duplicate error and are uncharged.

Pricing examples

One run	Actor fees
1 successful page	$0.00405
3 successful pages and 1 error	$0.01205
10 successful pages	$0.04005
50 successful pages	$0.20005
0 successful pages	$0.00005 startup only

The synthetic startup event is charged automatically by Apify, once at the supported 256–512 MB memory allocation. There is no automatic per-Dataset-row fee. The Actor checks whether another successful-page event fits your budget before fetching. Once the budget is exhausted, remaining pages are reported as unprocessed, not as zero-word results.

On a container restart, paid runs compare saved successes with Apify's authoritative page-analyzed count before proceeding. A write and its charge are separate operations. If an interruption leaves them inconsistent, the run fails visibly with BILLING_STATE_MISMATCH and does not recharge saved pages. Inspect the existing Dataset and run charges before starting a new run; automatic repair of an uncertain charge is outside this release.

Validation and source notes

The release QA checks three publicly accessible HTML layouts: Python documentation, MDN Web Docs and the USGS Water Science School. Heading tags and parent relationships are reviewed independently from the extraction implementation. The QA package records exact snapshots/timestamps and full expected sequences, plus an actual 404 and controlled robots, timeout, encoding, redirect, private-address, body-limit and billing tests. This is the tested scope; arbitrary site layouts can still require manual inspection.

Source policies and validation evidence are maintained with the project. Python documentation is under PSFv2; MDN heading-only QA samples credit Mozilla Contributors and retain CC BY-SA 2.5-or-later terms; USGS-authored source material is credited to the U.S. Geological Survey. These sources do not endorse this Actor. Their full article bodies are not included in the product's output or distributed examples.

Local development

Requires Node.js 22 or later. Install pinned dependencies with npm ci; run npm test. To run locally, provide storage/key_value_stores/default/INPUT.json and execute npm start. No search or LLM API key is required. Cloud authentication belongs in Apify's environment, never in the input example or source files.

Maintained by AFK Digital. Report reproducible issues through the Actor's Apify Issues tab, with the affected public URL, run ID and error code. Remove sensitive query-string values before sharing a report.