llms.txt Generator: Make Any Website AI-Readable

Crawl any website and generate llms.txt and llms-full.txt files following the llms.txt standard. BFS crawler with URL filtering and markdown/plaintext output. For AI readiness, SEO for AI, documentation portals, and archiving.

Crawl a website and generate llms.txt + llms-full.txt for AI/LLM discoverability and RAG pipelines. Batch, API-callable, pay-per-page.

Crawl a website and generate an llms.txt file (the emerging standard that helps LLMs and AI agents understand your site). Optionally build llms-full.txt with full page content. Output saved to the key-value store.

LLMs.txt Generator — Make Any Site AI-Ready | $0.05

devsef/site-to-llms-txt

Generate a spec-compliant llms.txt for any website in one run — make your site discoverable for ChatGPT, Claude & Perplexity. Crawls up to 200 pages, extracts titles and meta descriptions, outputs clean markdown. Flat $0.05 per site.

llms.txt Generator - Make Your Site AI-Discoverable

dltik/llms-txt-generator

Generate a standards-compliant llms.txt file for any website. Crawls your sitemap, extracts page titles & descriptions, groups them into sections, and outputs a clean llms.txt that helps ChatGPT, Claude, Gemini & Perplexity understand your site. Optional AI enrichment.

Generate llms.txt files for AI search visibility. Fetches sitemap, validates structure, checks AI bot access (OAI-SearchBot, PerplexityBot, ClaudeBot, Google-Extended). 2 blue ocean competitors only.

Ai Visibility Suite - Dark Visitors Alternative

alizarin_refrigerator-owner/ai-visibility-suite---dark-visitors-alternative

Comprehensive AI bot monitoring, robots.txt analysis, LLMs.txt generation & AI shopping optimization. Monitor AI crawlers visits, check AI compliance, generate AI-friendly configurations, and optimize for AI shopping agents. AI Bot Directory Robots.txt LLMs.txt AI Shopping Competitor AI Audit

Turn accessibility scan results into developer-ready remediation reports. Plain-English explanations, AI code fixes, cost estimates, fix-by timeline, competitor benchmarking. $5/page, no subscription.

Html to Markdown Converter

antonio_espresso/html-to-markdown-converter

Crawl a target URL and convert its HTML content into clean, structured Markdown with optional heading-based chunking.

Free API: turn any URL into clean link-preview metadata including title, description, Open Graph and Twitter card, favicon, canonical, and language. Powers link unfurling and preview cards. No login.