# Broken Link Checker (`hereditary_model/broken-link-checker`) Actor Crawls a website and reports all broken links (4xx/5xx and unreachable), with the pages they appear on. Pay only per page checked. - **URL**: https://apify.com/hereditary\_model/broken-link-checker.md - **Developed by:** [Aaron Marxsen](https://apify.com/hereditary_model) (community) - **Categories:** SEO tools, Automation, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Broken Link Checker Find every broken link on any website in one run. This actor crawls your site page by page, collects all links (internal and external), verifies each one, and returns a clean report of dead links together with the exact pages they appear on. Broken links hurt SEO rankings, waste crawl budget, and frustrate visitors. Catch them before your users or Google do. Built for SEO specialists, web developers, and site owners who need a reliable broken link report. ### Features - Check any website for broken links (404s and unreachable URLs) - Crawl all pages of a site on the same domain - Verify both internal and external links, with external checking optional - See the exact status code or network error for every broken link - See every page each broken link appears on - Run on a schedule for a fresh report daily or weekly - Pay per page checked, no subscription ### Use cases - Audit a site for dead links before a launch or migration - Run a weekly broken link check for every client site - Clean up crawl budget waste for better SEO - Give an AI SEO agent a tool to verify a site's links ### Input | Field | Type | Description | | --- | --- | --- | | `startUrls` | array (required) | Website URL(s) where the crawl starts | | `maxPages` | integer (default 50) | Maximum pages to crawl and scan | | `checkExternalLinks` | boolean (default true) | Also verify links to other domains | ```json { "startUrls": [{ "url": "https://example.com" }], "maxPages": 100 } ```` ### Output One dataset item per broken link: ```json { "brokenUrl": "https://example.com/old-page", "statusCode": 404, "error": null, "isExternal": false, "foundOnPages": ["https://example.com/", "https://example.com/blog/post-1"] } ``` ### Pricing: worked example Pay per event. You are charged `page-scanned` at $0.002 for each page crawled and scanned for links. Checking a 200 page site triggers 200 page events at $0.002 each. Total: $0.40. ### How to use 1. Open the actor and click Try for free. 2. Enter the website URL to check. 3. Set `maxPages`, and choose whether to verify external links. 4. Click Start. 5. Download the broken link report as JSON or CSV from the Storage tab. 6. Fix the dead links on the pages listed, or schedule the actor to re-check weekly. ### FAQ **How do I find broken links on my website?** Enter your site URL and this actor crawls every page, verifies each link, and returns the dead ones with the pages they appear on. **Does it follow redirects?** Yes. A link that redirects to a working page is not reported as broken. **Can it check external links too?** Yes. External link checking is on by default and can be switched off. **Can it handle large sites?** Yes. Raise `maxPages`. Cost scales linearly with pages scanned. **Can an AI agent run this?** Yes. It supports agentic payments, so an SEO agent can check a site and pay per page. ### API and agents Run it from the [Apify API](https://docs.apify.com/api/v2) or let an AI agent call it. Supports agentic payments via x402 and Skyfire (pay per event, limited permissions, no Standby). ```bash curl -X POST "https://api.apify.com/v2/acts/hereditary_model~broken-link-checker/runs?token=YOUR_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "startUrls": [{ "url": "https://example.com" }], "maxPages": 50 }' ``` ### Related actors - [Local SEO Audit](https://apify.com/hereditary_model/local-seo-audit) - [Website to Markdown for RAG and LLMs](https://apify.com/hereditary_model/website-to-markdown) - [Domain Intelligence API](https://apify.com/hereditary_model/domain-intelligence) - [Email and Social Media Extractor](https://apify.com/hereditary_model/contact-details-extractor) - [ATS Job Postings Scraper](https://apify.com/hereditary_model/ats-job-postings-scraper) - [NPI Healthcare Provider Directory](https://apify.com/hereditary_model/healthcare-provider-directory) - [Bandcamp Discography Scraper](https://apify.com/hereditary_model/bandcamp-discography-scraper) # Actor input Schema ## `startUrls` (type: `array`): Website URL(s) where the crawl starts. The checker stays on the same domain while discovering pages, but verifies links pointing anywhere. ## `maxPages` (type: `integer`): Maximum number of pages on the site to crawl and scan for links. Each crawled page counts as one billable event. ## `checkExternalLinks` (type: `boolean`): Also verify links pointing to other domains (social profiles, references, etc.). ## `proxyConfiguration` (type: `object`): Proxy settings used for crawling. Defaults to no proxy. ## Actor input object example ```json { "startUrls": [ { "url": "https://crawlee.dev" } ], "maxPages": 50, "checkExternalLinks": true } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://crawlee.dev" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("hereditary_model/broken-link-checker").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://crawlee.dev" }] } # Run the Actor and wait for it to finish run = client.actor("hereditary_model/broken-link-checker").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://crawlee.dev" } ] }' | apify call hereditary_model/broken-link-checker --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=hereditary_model/broken-link-checker", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Broken Link Checker", "description": "Crawls a website and reports all broken links (4xx/5xx and unreachable), with the pages they appear on. Pay only per page checked.", "version": "0.1", "x-build-id": "d3j56AtUqBpnVZZfd" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/hereditary_model~broken-link-checker/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-hereditary_model-broken-link-checker", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/hereditary_model~broken-link-checker/runs": { "post": { "operationId": "runs-sync-hereditary_model-broken-link-checker", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/hereditary_model~broken-link-checker/run-sync": { "post": { "operationId": "run-sync-hereditary_model-broken-link-checker", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "startUrls" ], "properties": { "startUrls": { "title": "Start URLs", "type": "array", "description": "Website URL(s) where the crawl starts. The checker stays on the same domain while discovering pages, but verifies links pointing anywhere.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "maxPages": { "title": "Max pages to crawl", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Maximum number of pages on the site to crawl and scan for links. Each crawled page counts as one billable event.", "default": 50 }, "checkExternalLinks": { "title": "Check external links", "type": "boolean", "description": "Also verify links pointing to other domains (social profiles, references, etc.).", "default": true }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Proxy settings used for crawling. Defaults to no proxy." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```