Web Hygiene MCP — live website checks for AI agents

An MCP server that gives Claude, ChatGPT, Cursor and any MCP-capable agent seven tools that answer from the live web: what a site publishes, whether URLs actually work, what its feeds contain, and whether the sources a text cites really say what is claimed. Every answer is the result of real HTTP requests made at call time — never a cache, never the model's memory — and every tool is bounded so it returns in seconds, mid-conversation.

Built on the same engine as the Sitemap URL Extractor, Broken Link Checker and Feed Monitor Actors; the tools point to them when a job is too big for a chat turn.

Tools

Tool	What the agent gets	Typical prompt
site_overview(site)	robots.txt (sitemaps declared, AI crawlers addressed, blanket disallow), llms.txt presence, sitemaps found with a URL count sample, feeds found	"Before we cite anything from example.com, what does the site publish and allow?"
list_site_urls(site_or_sitemap, max_urls, include_regex, exclude_regex)	URLs from the site's sitemaps with lastmod, filtered, up to 2,000	"List the blog posts on example.com published this year"
check_url(url)	Status, full redirect chain, final URL, response time, content type	"Does this link still work? Where does it redirect?"
check_links(urls | site_or_sitemap, max_urls, problems_only)	Bulk status check: 404s, 5xx, timeouts, SSL errors, redirect chains, slow pages	"Check every link in this document" · "Any broken pages on our site?"
discover_feeds(site)	The site's RSS/Atom/JSON feeds	"Does this blog have a feed?"
read_feed(feed_or_site, max_items, include_content)	Normalised items: title, link, date in UTC, author, plain-text summary, categories, enclosures	"What did they publish this week?"
verify_citations(text | urls | citations, check_archive, max_citations)	Per cited URL: verdict (verified, reachable, mismatch, not-found, redirected-home, paywalled, blocked, unreachable), confidence, reasons, the page's real title/date/author, quote match evidence, Internet Archive copy for dead links	"Check the sources in the answer you just gave me" · "Is this quote really on that page?"

All tools accept a bare domain and discover the rest; all return structured JSON plus a nextSteps / fullSiteTool / batchTool hint naming the Actor to use for whole-site or whole-document work.

Connect

Streamable HTTP endpoint (Standby mode — always on, no run to start):

https://bruco3--web-hygiene-mcp.apify.actor/mcp
Authorization: Bearer <YOUR_APIFY_API_TOKEN>

Claude Desktop / Cursor style configuration:

{
  "mcpServers": {
    "web-hygiene": {
      "url": "https://bruco3--web-hygiene-mcp.apify.actor/mcp",
      "headers": { "Authorization": "Bearer YOUR_APIFY_API_TOKEN" }
    }
  }
}

Or add it through the Apify MCP server, which exposes any Actor as a tool.

Try it — three prompts

1. "Check the sources in the answer you just gave me: does each link exist, and is the quote really on the page?" → verify_citations
2. "Before we cite anything from example.com, what does the site publish and allow — robots.txt, llms.txt, sitemaps, feeds?" → site_overview
3. "Check every link in this README and tell me which are broken or redirected." → check_links

Claude Code users: install the ready-made plugin (two skills + this server, asks for your Apify token once): /plugin marketplace add Luca-003/web-hygiene-claude-plugin then /plugin install web-hygiene@luca-003.

Why an agent needs this

Language models are confident about links and sites they have never fetched. That is how hallucinated citations, dead links in generated content and crawls of forbidden paths happen. These tools give the agent a cheap, deterministic way to look before it claims:

• verify that a URL exists and where it really leads before including it in an answer;
• confirm that a cited page really carries the quoted passage, title and date — the check that turns "the model cited something" into "the citation holds";
• check the site's robots.txt and llms.txt before fetching or quoting;
• read what a site actually publishes (sitemap, feed) instead of guessing page URLs;
• validate every link in a document it just wrote.

Pricing

Pay per event, no subscription, no idle cost to you:

Event	When
Tool call	every invocation of any tool
URL returned	each URL in a list_site_urls answer
URL checked	each URL probed by check_url / check_links
Feed checked / item returned	read_feed
Citation checked	each URL fetched and verified by verify_citations

A check_url costs a fraction of a cent; a 100-URL check_links a few cents; verifying the 10 sources of an answer about two cents. Set a maximum charge on the Standby Actor to cap spending.

Limits (by design)

• list_site_urls ≤ 2,000 URLs, check_links ≤ 300 URLs, read_feed ≤ 100 items, verify_citations ≤ 50 citations, site_overview samples ≤ 5,000 URLs. Whole-site and whole-document jobs belong to the Actors: Sitemap URL Extractor, Broken Link Checker, Feed Monitor, Citation & Link Verifier.
• Never more than 8 concurrent requests per host. Bodies are never downloaded for checks (HEAD, or GET closed after the headers); verify_citations reads cited pages up to 1.5 MB each.
• No JavaScript rendering and no crawling of pages: the tools read what sites publish for machines — sitemaps, feeds, robots.txt — and probe or read the URLs you name.

Legal

The tools request only public files sites publish for automated readers (sitemaps, feeds, robots.txt, llms.txt) and probe public URLs with a clear User-Agent. No personal data is collected or stored.

Feedback

A site the tools misread, or a tool you are missing? Open an issue on the Issues tab.

Changelog

• 0.2 — verify_citations: live verification of cited sources (quote, title, date, author; soft-404s; archive copies).
• 0.1 — initial release: six tools, Streamable HTTP, Standby mode, pay-per-event.