Website Health Checker: Broken Links & SEO (Saúde do Site)

HTTP-only health audit for a website you own or are authorized to audit. One run checks:

• Broken links: internal pages are crawled; internal and external links are verified (external sites are never crawled).
• robots.txt: exists, syntax, Sitemap: lines, whether it blocks everything (RFC 9309).
• sitemap.xml: exists, valid (sitemaps.org protocol), sitemap index, URLs listed vs. URLs found by the crawl.
• llms.txt: exists and follows the basic llmstxt.org format.
• Per page: title, meta description, canonical, robots meta, Open Graph basics, h1, lang, JSON-LD present, parseable and with @type.

No browser, no proxy: plain HTTP requests only, 256 MB.

Use only on sites you own or are authorized to audit.

Pricing

Pay-per-event pricing (standard Apify events only):

Event	Price
apify-actor-start	US$ 0.001 per run
apify-default-dataset-item	US$ 0.001 per checked URL or site report

One dataset item is written only for a request that was actually made: a checked URL (page or link) or one of the three site reports (robots.txt, sitemap, llms.txt). URLs blocked by robots.txt, refused by the SSRF guard, out of scope or over a limit are not written and not charged. Diagnostics go to the OUTPUT record and are free. ACTOR_MAX_TOTAL_CHARGE_USD and the platform charge limit are respected; if a charge cannot be confirmed, nothing is written and the run stops.

Input

Field	Default	Notes
startUrl	required	http(s) home page. Only this host (with or without www) is crawled. Other subdomains are treated as external.
maxPages	50 (max 100)	HTML pages downloaded and audited.
maxLinks	300 (max 500)	URLs checked per run (each is one charged result).
maxDepth	2 (max 3)	Clicks from the start URL whose pages are audited. 0 audits only the start page and checks its links.
checkExternalLinks	true	HEAD request with GET fallback; the body is never downloaded.
checks	all	Any of links, meta, robots, sitemap, llms.

The hard caps cannot be raised by the input. Ports other than 80/443, credentials in the URL and non-HTTP schemes are rejected.

Output

The default dataset has one item per checked request (kind tells the type):

{"kind":"url","url":"https://example.com/","final_url":"https://example.com/","found_on":null,"link_type":"page","depth":0,"method":"GET","status":200,"result":"ok","error_code":null,"redirects":0,"response_ms":17,"content_type":"text/html","meta":{"title":"Example Domain","title_length":14,"description":null,"canonical":null,"robots":null,"og_title":null,"og_image":null,"h1_count":0,"lang":"en","jsonld_blocks":0,"jsonld_valid":false,"jsonld_types":[]},"issues":["missing_description","missing_canonical","missing_h1"],"checked_at":"2026-10-23T12:54:29.000Z"}
{"kind":"robots","url":"https://example.com/robots.txt","status":404,"result":"not_found","found":false,"sitemaps":[],"issues":["not_found"]}
{"kind":"sitemap","url":"https://example.com/sitemap.xml","found":false,"valid":false,"urls_listed":0,"listed_not_found_count":0,"found_not_listed_count":1,"issues":["not_found"]}

(Examples are abbreviated.) The key-value store record OUTPUT has the run summary: status, completo, motivo (stop reason), counts, charged events and a cost estimate. Stop reasons: prazo_interno, teto_cobranca, teto_margem, limite_links, robots_bloqueia, robots_inacessivel, ssrf_bloqueado, input_invalido.

How it behaves

• robots.txt (RFC 9309): always read first. User-agent KmanSiteHealthBot/0.1 (+https://apify.com/kman-br/website-health-checker). Our group wins over *; the longest matching rule wins and Allow wins ties; * and $ are supported. 4xx means no restrictions; 5xx or an unreachable file means the run stops after reporting it. Disallowed internal URLs, sitemap.xml and llms.txt are not requested. Crawl-delay is honored up to 10 s.
• Courtesy: at least 500 ms between requests to the same host, one request at a time per host, 4 requests in parallel overall, 8 s timeout, at most 5 redirects, at most 1 MiB read per page.
• nofollow: the robots meta nofollow does not stop link following; it is an indexing directive and the site belongs to the user.
• Ignored links: mailto:, tel:, sms:, javascript:, WhatsApp links and images are neither checked nor listed.
• SSRF guard: only http/https on ports 80/443; private, loopback, link-local, metadata (169.254.169.254) and reserved addresses are refused, including hosts that resolve to them (checked at connection time, on every redirect hop).
• Deadline: the run stops after about 150 s (internal deadline) with a partial summary (prazo_interno) and exit code 0.
• No personal data: page bodies are never stored. Output fields are a fixed whitelist; e-mail addresses and phone numbers found in titles, descriptions or headings are replaced by [removido], URLs with credentials or e-mail addresses are dropped, and short meta values are truncated.

Limits

100 pages, 500 checked URLs, depth 3 per run. Large sites are audited partially; use several runs with different startUrl values.

Related Actors

More Actors from the same publisher:

• PDF to Text & Markdown Converter (PDF para Texto): Extracts the text layer of PDFs from public URLs as plain text and simple Markdown, with the page count.
• Workday Jobs Scraper API: Job postings from public Workday career sites (myworkdayjobs.com), with clean descriptions.
• Greenhouse Jobs Scraper API: Job postings from public Greenhouse job boards, with departments and clean descriptions.

Browse the full catalog: kman-br.github.io (Português).