Internal Link Opportunity & Anchor Risk Auditor

A deterministic Apify Actor that crawls one public website and produces a human-review queue of internal-link opportunities. It does not call an LLM, require an SEO subscription, log in to a CMS, or publish links.

All documented analysis features use the same code path for free and paid Apify users. There is no account-tier feature gating. Cost is bounded with input limits; future monetization should use tiered event pricing instead of disabling promised behavior for free users.

What makes the output reviewable

Each suggestion contains:

• source and target URL;
• the exact sentence already present on the source page;
• an exact phrase from that sentence proposed as the anchor;
• a 0–100 score and its numeric components;
• incoming-link count for the target;
• anchor repetition and collision warnings;
• target-specificity warnings when a match relies on a generic page or does not match the URL topic;
• a timestamp and explicit confidence label.

Suggestions are omitted when the source already links to the target. Orphan and underlinked pages are listed in the OUTPUT summary.

Deterministic scoring

The same crawl produces the same ranking. The Actor:

1. extracts visible static main-content text, titles, H1s, descriptions and existing same-site links;
2. normalizes English and Russian terms with a small fixed stop-word list and light suffix normalization;
3. calculates corpus-aware term weights from target title, H1, description and URL slug;
4. searches source sentences for exact anchor spans containing target terms;
5. combines lexical relevance, title/H1 coverage, phrase quality, underlinked-target boost and optional priority-target boost;
6. subtracts a penalty for anchors already repeated across the site and emits separate risk flags.
7. requires a target-specific heading term and discounts matches that do not also support the target URL topic.

There is no hidden prompt, embedding model, random seed or third-party ranking API.

Input

{
  "startUrl": "https://example.com/sitemap.xml",
  "maxPages": 100,
  "maxSuggestionsPerPage": 3,
  "minimumScore": 48,
  "targetUrls": ["https://example.com/services/seo"],
  "excludePatterns": ["/account/", "/cart", "/tag/"],
  "respectRobotsTxt": true
}

startUrl can be a regular page, a sitemap, or a sitemap index. Crawling remains on the original hostname and accepts public HTTPS destinations only. maxPages is a hard limit on attempted HTML pages; supporting traffic is bounded to one robots.txt request and at most 20 sitemap documents.

With no supplied input, the Actor audits at most eight pages from Apify Academy. This bounded default exercises the complete free-user path and returns a non-empty review queue for Store health checks. A one-page site such as https://example.com/ still succeeds with status: "insufficient_pages", an empty review queue, and an explicit explanation in OUTPUT.

Output example

{
  "recordType": "opportunity",
  "sourceUrl": "https://example.com/blog/trail-guide",
  "targetUrl": "https://example.com/catalog/trail-shoes",
  "sourceSentence": "For wet forest routes, trail running shoes with deep grip improve control.",
  "suggestedAnchor": "trail running shoes with deep grip",
  "score": 76.4,
  "confidence": "high",
  "warningFlags": [],
  "targetIncomingInternalLinks": 1,
  "reason": "48 lexical-relevance points; 18 title/H1 coverage points; underlinked target boost +4"
}

Guardrails and honest limitations

• robots.txt is respected by default.
• Private/reserved network addresses and credential-bearing URLs are blocked.
• Only static HTML is supported in v1. JS-only pages fail closed and are reported; content is not guessed.
• Lexical relevance can miss synonyms and meaning expressed with completely different words. Lower-confidence rows must be reviewed.
• No traffic or PageRank claim is made without first-party inputs such as a future GSC/crawl export.
• The Actor never modifies the website.
• A one-page or empty crawl returns status: "insufficient_pages"; at least two readable pages are required to calculate link opportunities.
• maxPages limits attempted HTML pages, including failures, so broken sites cannot create an unbounded request loop.

Local development

npm ci
npm run verify

For a local Actor run, place input in storage/key_value_stores/default/INPUT.json, then run npm start.

Free-user and ADWIC verification

Before every Store release, run the default input and compare its run status, dataset, and OUTPUT record with every claim above. Every promised feature must remain technically reachable by a free Apify user. Expensive workloads are controlled by input limits and tiered event pricing, not feature gates.

Quality gate before Store publication

Build a human gold set on three permitted public sites (small, medium and content-heavy). Review the top ten suggestions per site. Continue only if at least 60% are accepted without changing the source/target pair, all rejected rows expose a useful warning or low score, and no suggestion duplicates an existing source-to-target link.