Website Agent-Readiness & Metadata Audit

Website Metadata Extractor (meta tags, sitemap, robots) 🔎

powerful_bachelor/website-metadata-extractor

🔍 Website Metadata Extractor 🌐 Extract essential website data: meta tags, robots.txt, and sitemap.xml in one scan. 📊 Analyze SEO elements, crawler directives, and site structure. ✅ Perfect for SEO audits, 🔎 competitor research, and 🚀 understanding how search engines view your website.

List of URLs whose HTTP redirect chains will be analyzed

Open Graph & Meta Tag Extractor

bgfc97/opengraph-meta-extractor

Extract link-preview metadata from any URL in bulk: Open Graph, Twitter Card, title, meta description, canonical, favicon and language. Perfect for building preview cards, SEO audits and social sharing tools. No API key, no proxy - just paste URLs.

🔎 Crawl websites and turn technical SEO, accessibility, security, performance, broken-link, structured-data, robots.txt, llms.txt, and AI crawler signals into prioritized 0–100 audit scores and client-ready reports.

Extract Schema.org JSON-LD structured data from any website. Fast, lightweight HTTP-based scraper that pulls all JSON-LD scripts - perfect for SEO analysis, product data extraction, and AI/RAG pipelines. No browser overhead.

URL Metadata Crawler

easyapi/url-metadata-crawler

Extracting comprehensive metadata from web pages. Gather vital information like meta tags, favicons, Open Graph tags, and more, all while enjoying flexible options for customization. Perfect for SEO specialists, developers, and content creators looking to enhance their web presence! 🌐

Page Metadata Extractor API: OpenGraph, JSON-LD, Feeds

tudorlane/page-metadata-extract

Get a web page's title, description, OpenGraph and Twitter card, schema.org JSON-LD, RSS feeds, favicon, canonical URL, contact emails/phones and social links in one call. Respects robots.txt.

GEO Site Audit - AI Readiness Checker

dltik/geo-site-audit

Audit your website for AI crawler accessibility: robots.txt (GPTBot, ClaudeBot, Perplexity), llms.txt, sitemap, Schema.org, meta tags, content extractability, TTFB. Get an AI-readiness score 0-100 with prioritized recommendations.

Gemini Search Scraper: Gemini Answers, Citations & Sources

plainrows/gemini-search-scraper

Gemini scraper and MCP tool: send prompts, get Google Gemini's answers as JSON: markdown text, the sources Gemini cites (URL, domain, title), cited domains, model. No account, no browser. $0.008 per answer, plus $0.001 per run; empty or failed answers are free.

Website SEO audit crawler: on-page issues (title, meta description, H1, canonical, hreflang, noindex), broken internal and external links, schema.org JSON-LD errors, sitemap and robots.txt checks, AI-crawler rules (GPTBot, ClaudeBot…) and llms.txt. One row per page plus a site summary.