# Website Health Checker: Broken Links & SEO (Saúde do Site) (`kman-br/website-health-checker`) Actor HTTP-only audit of a site you own: broken internal and external links, robots.txt, sitemap.xml, llms.txt and per-page title, meta description, canonical, robots meta, Open Graph, h1, lang and JSON-LD. No browser, no personal data. - **URL**: https://apify.com/kman-br/website-health-checker.md - **Developed by:** [Kman](https://apify.com/kman-br) (community) - **Categories:** SEO tools, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.00 / 1,000 url or site report checkeds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Website Health Checker: Broken Links & SEO (Saúde do Site) HTTP-only health audit for a website you own or are authorized to audit. One run checks: - **Broken links**: internal pages are crawled; internal and external links are verified (external sites are never crawled). - **robots.txt**: exists, syntax, `Sitemap:` lines, whether it blocks everything (RFC 9309). - **sitemap.xml**: exists, valid (sitemaps.org protocol), sitemap index, URLs listed vs. URLs found by the crawl. - **llms.txt**: exists and follows the basic llmstxt.org format. - **Per page**: title, meta description, canonical, robots meta, Open Graph basics, h1, `lang`, JSON-LD present, parseable and with `@type`. No browser, no proxy: plain HTTP requests only, 256 MB. > Use only on sites you own or are authorized to audit. ### Pricing Pay-per-event pricing (standard Apify events only): | Event | Price | |---|---| | `apify-actor-start` | US$ 0.001 per run | | `apify-default-dataset-item` | US$ 0.001 per checked URL or site report | One dataset item is written only for a request that was actually made: a checked URL (page or link) or one of the three site reports (robots.txt, sitemap, llms.txt). URLs blocked by robots.txt, refused by the SSRF guard, out of scope or over a limit are not written and not charged. Diagnostics go to the `OUTPUT` record and are free. `ACTOR_MAX_TOTAL_CHARGE_USD` and the platform charge limit are respected; if a charge cannot be confirmed, nothing is written and the run stops. ### Input | Field | Default | Notes | |---|---|---| | `startUrl` | required | `http(s)` home page. Only this host (with or without `www`) is crawled. Other subdomains are treated as external. | | `maxPages` | 50 (max 100) | HTML pages downloaded and audited. | | `maxLinks` | 300 (max 500) | URLs checked per run (each is one charged result). | | `maxDepth` | 2 (max 3) | Clicks from the start URL whose pages are audited. `0` audits only the start page and checks its links. | | `checkExternalLinks` | true | HEAD request with GET fallback; the body is never downloaded. | | `checks` | all | Any of `links`, `meta`, `robots`, `sitemap`, `llms`. | The hard caps cannot be raised by the input. Ports other than 80/443, credentials in the URL and non-HTTP schemes are rejected. ### Output The default dataset has one item per checked request (`kind` tells the type): ```json {"kind":"url","url":"https://example.com/","final_url":"https://example.com/","found_on":null,"link_type":"page","depth":0,"method":"GET","status":200,"result":"ok","error_code":null,"redirects":0,"response_ms":17,"content_type":"text/html","meta":{"title":"Example Domain","title_length":14,"description":null,"canonical":null,"robots":null,"og_title":null,"og_image":null,"h1_count":0,"lang":"en","jsonld_blocks":0,"jsonld_valid":false,"jsonld_types":[]},"issues":["missing_description","missing_canonical","missing_h1"],"checked_at":"2026-10-23T12:54:29.000Z"} {"kind":"robots","url":"https://example.com/robots.txt","status":404,"result":"not_found","found":false,"sitemaps":[],"issues":["not_found"]} {"kind":"sitemap","url":"https://example.com/sitemap.xml","found":false,"valid":false,"urls_listed":0,"listed_not_found_count":0,"found_not_listed_count":1,"issues":["not_found"]} ``` (Examples are abbreviated.) The key-value store record `OUTPUT` has the run summary: `status`, `completo`, `motivo` (stop reason), counts, charged events and a cost estimate. Stop reasons: `prazo_interno`, `teto_cobranca`, `teto_margem`, `limite_links`, `robots_bloqueia`, `robots_inacessivel`, `ssrf_bloqueado`, `input_invalido`. ### How it behaves - **robots.txt (RFC 9309)**: always read first. User-agent `KmanSiteHealthBot/0.1 (+https://apify.com/kman-br/website-health-checker)`. Our group wins over `*`; the longest matching rule wins and `Allow` wins ties; `*` and `$` are supported. `4xx` means no restrictions; `5xx` or an unreachable file means the run stops after reporting it. Disallowed internal URLs, `sitemap.xml` and `llms.txt` are not requested. `Crawl-delay` is honored up to 10 s. - **Courtesy**: at least 500 ms between requests to the same host, one request at a time per host, 4 requests in parallel overall, 8 s timeout, at most 5 redirects, at most 1 MiB read per page. - **`nofollow`**: the robots meta `nofollow` does not stop link following; it is an indexing directive and the site belongs to the user. - **Ignored links**: `mailto:`, `tel:`, `sms:`, `javascript:`, WhatsApp links and images are neither checked nor listed. - **SSRF guard**: only `http`/`https` on ports 80/443; private, loopback, link-local, metadata (`169.254.169.254`) and reserved addresses are refused, including hosts that resolve to them (checked at connection time, on every redirect hop). - **Deadline**: the run stops after about 150 s (internal deadline) with a partial summary (`prazo_interno`) and exit code 0. - **No personal data**: page bodies are never stored. Output fields are a fixed whitelist; e-mail addresses and phone numbers found in titles, descriptions or headings are replaced by `[removido]`, URLs with credentials or e-mail addresses are dropped, and short meta values are truncated. ### Limits 100 pages, 500 checked URLs, depth 3 per run. Large sites are audited partially; use several runs with different `startUrl` values. ### Related Actors More Actors from the same publisher: - [PDF to Text & Markdown Converter (PDF para Texto)](https://apify.com/kman-br/pdf-to-text-markdown): Extracts the text layer of PDFs from public URLs as plain text and simple Markdown, with the page count. - [Workday Jobs Scraper API](https://apify.com/kman-br/workday-jobs): Job postings from public Workday career sites (myworkdayjobs.com), with clean descriptions. - [Greenhouse Jobs Scraper API](https://apify.com/kman-br/greenhouse-jobs): Job postings from public Greenhouse job boards, with departments and clean descriptions. Browse the full catalog: [kman-br.github.io](https://kman-br.github.io/) ([Português](https://kman-br.github.io/pt/)). # Actor input Schema ## `startUrl` (type: `string`): Home page of a site you own or may audit. Only this host (with or without www) is crawled; other hosts are only checked. ## `maxPages` (type: `integer`): HTML pages on the same site that are downloaded and audited. Hard cap 100. ## `maxLinks` (type: `integer`): Total URLs checked per run (crawled pages plus internal and external links). Each checked URL is one charged result. Hard cap 500. ## `maxDepth` (type: `integer`): Clicks away from the start URL whose pages are downloaded and audited. 0 audits only the start page and checks its links. ## `checkExternalLinks` (type: `boolean`): Check links to other sites with a HEAD request (GET fallback, body not downloaded). External sites are never crawled. ## `checks` (type: `array`): Audits to run. robots.txt is always read to obey it, even when its report is not selected. ## Actor input object example ```json { "startUrl": "https://example.com", "maxPages": 50, "maxLinks": 300, "maxDepth": 2, "checkExternalLinks": true, "checks": [ "links", "meta", "robots", "sitemap", "llms" ] } ``` # Actor output Schema ## `results` (type: `string`): No description ## `summary` (type: `string`): OUTPUT: status, reason for stopping (prazo_interno, teto_cobranca, teto_margem, limite_links, robots_bloqueia, robots_inacessivel, ssrf_bloqueado), counts and events charged. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrl": "https://example.com" }; // Run the Actor and wait for it to finish const run = await client.actor("kman-br/website-health-checker").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrl": "https://example.com" } # Run the Actor and wait for it to finish run = client.actor("kman-br/website-health-checker").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrl": "https://example.com" }' | apify call kman-br/website-health-checker --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,kman-br/website-health-checker" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/DyVe4tCSJbvieclH6/builds/6AdXDPn3tMfWS1Et3/openapi.json