Sitemap Indexability & AI Crawler Audit API

Technical SEO & AI Crawler Audit

taroyamada/technical-seo-portfolio-regression-report

Audit public pages you are authorized to test for robots, noindex, sitemap, canonical, and AI crawler access changes. Deliver source-linked issues and portfolio reports.

AI Crawler Access Checker

zinin/ai-crawler-access-checker

Audit up to 100 sites for 16 AI crawler policies. Get search vs training verdicts, robots.txt and llms.txt evidence, confidence, unknown states, priorities, and next actions for GEO/AEO reviews.

Company Domain Tech, DNS & Security Audit

taroyamada/company-enrichment-intelligence

Turn company domains into structured website, tech stack, DNS, SPF, DMARC, RDAP, TLS certificate, robots.txt, and sitemap audit records. Public, keyless sources only: the company's own website plus public registries. Pay per audited domain.

Website RAG Readiness Audit Report

taroyamada/website-rag-readiness-audit

Turn public website URLs into a decision-ready RAG readiness audit with coverage, chunking risk, retrieval cleanup actions, source URLs, and no user API key requirement.

robots.txt Parser & AI Crawler Block Checker

taroyamada/robotstxt-ai-checker

Audit robots.txt files across thousands of websites to detect specific crawl policies, disallowed paths, and user-agents for GPTBot and ClaudeBot.

Website Lighthouse & WCAG Regression Report

taroyamada/website-lighthouse-accessibility-regression-report

Run Lighthouse and axe-core audits on authorized public websites, then generate performance, accessibility, SEO, and regression reports.

🔎 Crawl websites and turn technical SEO, accessibility, security, performance, broken-link, structured-data, robots.txt, llms.txt, and AI crawler signals into prioritized 0–100 audit scores and client-ready reports.

SEO Audit API — On-Page + Core Web Vitals, Bulk & Scored

fetchbase/website-seo-audit

SEO audit with 0–100 score, real Core Web Vitals (LCP, CLS, FCP) measured in a live browser, and prioritized issues each with a concrete fix. Meta, headings, canonical, OG, JSON-LD, alt text, robots.txt, sitemap, broken links. Single pages or site crawls. Charged only per audited page.

SEO Audit + AI Visibility · Are You Citable by ChatGPT?

siccscha/seo-health-auditor

Technical SEO audit plus the question no other audit answers: can AI systems cite you? Checks which AI crawlers your robots.txt allows (only retrieval agents can cite you), flags pages that are empty without JavaScript, and diffs against your previous run. Plus fixes you can paste.

Test URLs against robots.txt with real Google semantics: longest-match, Allow wins ties, UA group selection (test Googlebot, GPTBot, ClaudeBot...), wildcards. Verdict + the exact rule that decided it + sitemaps. $0.0008 per URL, invalid URLs never charged — vs $0.001-$0.15 measured incumbents.

Website Metadata Extractor

scrapers-hub/website-metadata-extractor

Website metadata extractor to extract titles, descriptions, keywords, and meta tags from any website 🌐📊 Perfect for SEO analysis, auditing, and research. Fast, accurate, and scalable extraction.