AI Website Intelligence Scanner

Bulk AI Readiness Scanner

bmiller1009/bulk-ai-readiness-scanner

Score many domains for AI agent readiness in one run.

AI Crawler Policy Auditor

junipr/ai-crawler-policy-auditor

Audit robots and AI crawler policy signals for AI crawler access, blocking, and site-readiness.

AI Readiness Checker - Website Scanner

alizarin_refrigerator-owner/ai-readiness-checker

Analyze any website for AI optimization readiness. Check robots.txt, llms.txt, structured data, meta tags & content quality. Get actionable recommendations to improve AI crawler accessibility.

Bulk AI Crawler Policy Checker

bmiller1009/bulk-ai-crawler-policy-checker

Check many domains' robots.txt AI crawler policy in one run.

AI Website Content Crawler

ilborso/ai-website-content-crawler

A super fast website crawler for Agentic AI integration

Website Intelligence API

ib4ngz/website-intelligence-api

Analyze website technologies, comments, structured data, robots.txt, and sitemaps in one run.

Audit domains for llms.txt, robots.txt AI bot rules, sitemap hints, and public AI crawler readiness signals.

AI Readiness Auditor

rationalistic_counsel/ai-readiness-auditor

Check how AI-ready any website is. Get an AI Readiness Score (0-100) checking llms.txt, robots.txt AI crawler directives, Schema.org structured data, and meta tags. No API key needed.