🌐 Website Migration Link Auditor

Checks an old->new URL migration manifest. Each source URL gets an acceptance receipt: redirect target/chain/loop, 404/410/5xx, fragment survival, canonical convergence, and lingering old-host links on the destination.

Output is one migration_verdict row per result; billing is pay-per-event, the value event being one migration url check (a $0.02 start fee per run, then $0.05 per migration url check).

No login, no API key and no CAPTCHA solving are involved: the source is read logged-out with an identified contact User-Agent.

📊 Sample Output

Real rows from run 30BtdTKoFHECge3R3 on build 0.1.5 (2026-09-17), the same input as the Quick start below — every value is as the source published it (emails masked, long text shortened):

old_url	intended_new_url	final_url	redirect_hops	outcome	input
http://gov.uk	https://www.gov.uk/	https://www.gov.uk/	2	answer	http://gov.uk
https://www.gov.uk/does-not-exist-xyz	https://www.gov.uk/	https://www.gov.uk/does-not-exist-xyz	0	answer	https://www.gov.uk/does-not-exist-xyz
http://localhost/old	http://localhost/new		0	blocked	http://localhost/old

The run finished with the status message: PARTIAL: 2 billable verdict(s), 1 blocked (unbilled status), 0 unreachable, 0 unparseable of 3 target(s) | migration_url_check billable=2 | migration_url_check charged=2

✅ What you get

Each row is flat JSON with these fields (from the dataset schema and the sample run; a field the source does not publish for a given row is null):

• old_url (string/null) — e.g. http://gov.uk
• intended_new_url (string/null) — e.g. https://www.gov.uk/
• final_url (string/null) — e.g. https://www.gov.uk/
• redirect_chain (string/null) — e.g. [{"status":301,"location":"https://gov.uk/"},{"status":301,"location":"https://w
• redirect_hops (integer/string/null) — e.g. 2
• status_class (string/null) — e.g. ok
• http_status (integer/string/null) — e.g. 200
• reaches_intended (boolean/null) — e.g. True
• canonical_url (string/null) — e.g. https://www.gov.uk/
• canonical_converges (boolean/null) — e.g. True
• fragment (string/null) — null in every sample row
• lingering_old_host_links (integer/string/null) — e.g. 0
• verdict (string/null) — e.g. clean
• faults (string/null) — e.g. 404 Not Found; 63 link(s) still point to the old host www.gov.uk
• record_type (string) — e.g. migration_verdict
• source (string/null) — e.g. buyer-submitted old->new URL manifest
• outcome (string/null) — e.g. answer
• input (string/null) — e.g. http://gov.uk
• outcome_reason (string/null) — e.g. robots.txt for http://localhost unavailable/forbidden — cannot confirm permissio
• observed_at (string/null) — null in every sample row
• terminal (string/null) — null in every sample row

Output

One row per target. A complete evidenced verdict carries outcome: answer; blocked / unreachable / unparseable targets carry that status and are not billed.

⚙️ Sample inputs

1. Quick start — the Store example (this is what the sample above came from)

{
  "manifest": [
    "http://gov.uk|https://www.gov.uk/",
    "https://www.gov.uk/does-not-exist-xyz|https://www.gov.uk/",
    "http://localhost/old|http://localhost/new"
  ],
  "userAgent": "Mozilla/5.0 (compatible; NexGenWatchBot/1.0; +https://apify.com/nexgenwatch)"
}

The sample run charged exactly: 1 × $0.02 apify-actor-start + 2 × $0.05 migration_url_check = $0.12 on the Free tier — every delivered row was billed.

2. A smaller, narrowed run

{
  "manifest": [
    "http://gov.uk|https://www.gov.uk/",
    "https://www.gov.uk/does-not-exist-xyz|https://www.gov.uk/",
    "http://localhost/old|http://localhost/new"
  ],
  "userAgent": "Mozilla/5.0 (compatible; NexGenWatchBot/1.0; +https://apify.com/nexgenwatch)"
}

A bounded run for checking the field shape before scheduling. userAgent narrows what the source is asked for.

3. A full-size run

{
  "manifest": [
    "http://gov.uk|https://www.gov.uk/",
    "https://www.gov.uk/does-not-exist-xyz|https://www.gov.uk/",
    "http://localhost/old|http://localhost/new"
  ],
  "userAgent": "Mozilla/5.0 (compatible; NexGenWatchBot/1.0; +https://apify.com/nexgenwatch)"
}

The same input run on a schedule; each run is billed only for what it delivers.

🧾 JSON sample record

One real record from run 30BtdTKoFHECge3R3, exactly as it lands in the dataset (emails masked, long text shortened):

{
  "old_url": "http://gov.uk",
  "intended_new_url": "https://www.gov.uk/",
  "final_url": "https://www.gov.uk/",
  "redirect_chain": "[{\"status\":301,\"location\":\"https://gov.uk/\"},{\"status\":301,\"location\":\"https://www.gov.uk/\"}]",
  "redirect_hops": 2,
  "status_class": "ok",
  "http_status": 200,
  "reaches_intended": true,
  "canonical_url": "https://www.gov.uk/",
  "canonical_converges": true,
  "fragment": null,
  "lingering_old_host_links": 0,
  "verdict": "clean",
  "faults": null,
  "record_type": "migration_verdict",
  "source": "buyer-submitted old->new URL manifest",
  "outcome": "answer",
  "input": "http://gov.uk",
  "outcome_reason": null
}

🔧 How it works

Transport. Plain HTTPS from the Apify platform, no proxy. robots.txt is read first and a disallowed path is never fetched. Every request carries an identified contact User-Agent.

Terminal states. A run ends NORMAL, CAPPED (your cap was reached), PARTIAL (something was withheld and the message says what), GENUINE_EMPTY (the source was read and truly had nothing in scope) or BLOCKED (the source refused or changed shape — the run FAILS loud and bills nothing). A zero-row run is never reported as a silent success.

Charging. Each migration url check is charged at the moment it is pushed (migration_url_check); a row that fails to charge is not delivered, so the dataset count always equals the charged count.

What you submit

manifest — old->new url manifest. A migration manifest: each row is an old URL and (optionally) its intended new URL, as 'old|new' or {"old":..,"new":..}. Per source URL: redirect target/chain/loop, 404/410/5xx, fragment survival, canonical convergence, and lingering old-host links on the destination -> one acceptance receipt row. Two-origin robots gates (old host AND new host). A complete receipt bills; blocked/unreachable is delivered as an unbilled status, strictly separated from a broken (404/redirect-mismatch) verdict.

The runtime source gate (per submitted target)

Because you choose the targets, the source contract is enforced at run time, per origin, before any page is read:

• robots.txt is fetched once per origin and honored for our crawler. No robots / 404 = permitted; a disallowed path or an unavailable robots file = BLOCKED.
• SSRF guard — every host is resolved and must be a public address. Private, loopback, link-local, reserved and cloud-metadata addresses are refused before a socket opens.
• HTTP — bounded body reads; 403 / 429 / 5xx = BLOCKED; DNS / timeout / connection faults = UNREACHABLE.

A BLOCKED or UNREACHABLE target is delivered as an unbilled status row — never a broken-site verdict, and never charged.

Why this and not a generic crawler

no owned actor audits a migration manifest; SEO crawlers exist but this is an old->new acceptance receipt with blocked-vs-broken separated — a distinct deliverable.

What is not done. No login, no cookie or CAPTCHA bypass, no private or personal-account data, no browser automation.

💰 Pricing example

Event	Free	Bronze	Silver	Gold
Actor Start (apify-actor-start)	$0.02	$0.02	$0.02	$0.02
Migration Url Check (migration_url_check)	$0.05	$0.04	$0.04	$0.03

Worked at the live Free-tier price:

• 3 migration url checks: $0.02 start + 3 × $0.05 = $0.17
• 25 migration url checks: $0.02 start + 25 × $0.05 = $1.27
• 100 migration url checks: $0.02 start + 100 × $0.05 = $5.02

A run that delivers zero rows charges the $0.02 start fee only. A BLOCKED run (source refused) fails loud and charges no value event. The start fee is charged once per GB of run memory; the default run memory is 1024 MB.

Yield on the sample run: PARTIAL: 2 billable verdict(s), 1 blocked (unbilled status), 0 unreachable, 0 unparseable of 3 target(s) | migration_url_check billable=2 | migration_url_check charged=2.

⚖️ Legal & ToS

This actor reads public data only. It collects only what the source publishes to any visitor, keeps to the source's robots rules (checked on every run), identifies itself with a contact User-Agent, and does not access accounts, private data or anything behind authentication. Use the output in line with the source's terms and your local law; the intended use is B2B research and monitoring.

❓ FAQ

Q: Do I need an API key or a login?
A: No. the input schema has no key field and the actor carries no secrets.

Q: Why did my run return 0 rows?
A: Read the run's status message. GENUINE_EMPTY means the source was read and had nothing in scope for your input; BLOCKED means the source refused and the run failed without billing a value event — retry later or narrow the input. A zero-row run bills the start fee only.

Q: How fresh is the data?
A: Every run reads the source live at run time; nothing is cached between runs. Put it on a schedule for a continuous feed.

Q: What formats can I export?
A: The dataset downloads as JSON, CSV, Excel, XML or RSS from the run's Dataset tab or the Apify API, and any run can push to a webhook or integration.

Q: How is this different from the other web QA and enrichment tools actors?
A: Same output shape and billing model; this one covers Website Migration Link Auditor. The siblings under Related Actors cover the other sources or slices — run several on one schedule for a combined feed.

Q: Are there rate limits?
A: The actor paces itself against the source and honours its robots rules; there is no per-buyer limit beyond your Apify plan's concurrency.

🆘 Troubleshooting

• Run FAILED with BLOCKED → the source refused the request or changed its page shape → nothing was billed beyond the start fee; retry after a while, and if it persists open an Issue with the run id.
• Fewer rows than expected → the source had fewer items in scope → widen the input.
• Input validation error on start → a field is outside the schema's allowed values → start from the Quick start block and change one field at a time.
• Run TIMED-OUT → a very wide request on a slow day → raise the run timeout in Run options or narrow the input; what was delivered before the timeout is still in the dataset.

🔗 Related Actors

• Tech Stack Enricher — Give it a list of company domains; it returns one CRM-ready record per domain describing
• Website Brand Kit API — Extracts a full brand system from a domain — logos, favicons, declared and pixel-derived colors, font declarations, theme color and social share imag…
• Website Contact Finder — Email, Phone & Social Links — Find business emails, phone numbers, and social profile links published on company websites. Submit domains or URLs and pay only when a contact-beari…
• Website Screenshot Evidence API — Captures a headless-Chromium screenshot of each URL as a hashed evidence bundle — the image plus its final URL, HTTP state, capture timestamp, browse…
• Website Technology Detector — Fingerprints the technologies a website exposes on the pages its robots.txt permits — web server, CDN, CMS, JavaScript frameworks, analytics and mark…
• 🏢 About NexGenData — NexGen Watch is NexGenData's fleet of 256 public monitoring and lookup actors built on official sources, pay-per-result. Browse the catalog at apify.com/nexgenwatch.

⭐ Found this useful?

If this actor saved you a manual check, a quick review on the Apify Store helps other teams find it. Feature request or a source that changed? Open it from the Issues tab — every one is read.