Website Metadata Extractor (meta tags, sitemap, robots) 🔎

powerful_bachelor/website-metadata-extractor

🔍 Website Metadata Extractor 🌐 Extract essential website data: meta tags, robots.txt, and sitemap.xml in one scan. 📊 Analyze SEO elements, crawler directives, and site structure. ✅ Perfect for SEO audits, 🔎 competitor research, and 🚀 understanding how search engines view your website.

Extract and parse XML sitemaps from any website. Discovers sitemaps via robots.txt and common paths. Outputs structured URL data with metadata.

Website Metadata Extractor

scrapers-hub/website-metadata-extractor

Website metadata extractor to extract titles, descriptions, keywords, and meta tags from any website 🌐📊 Perfect for SEO analysis, auditing, and research. Fast, accurate, and scalable extraction.

Extract every URL from any website's XML sitemaps. Handles sitemap indexes, robots.txt discovery, URL filtering. Great for SEO audits.

Extract all URLs from any website's sitemap. Auto-discovers sitemaps from robots.txt, supports sitemap index files and .gz compression. Filter by URL pattern, date range.

Crawl any XML sitemap and instantly extract comprehensive SEO metadata (Title, Description, H1s, OG Tags, Canonical) for every page

Audit every URL in a sitemap for SEO metadata, status codes, titles, descriptions, H1s, canonical tags, noindex, word count, and common issues.

Find robots.txt, declared sitemaps, homepage metadata, canonical URL, and basic crawlability signals for a list of websites.

Sitemap Url Extractor

scrapers-hub/sitemap-url-extractor

Sitemap URL extractor to extract all URLs from XML sitemaps quickly and efficiently 🌐📄 Ideal for SEO audits, site analysis, and indexing workflows. Fast, accurate, and easy to use.