Sitemap Analyzer

Robots.txt & Sitemap Analyzer

automation-lab/robots-sitemap-analyzer

This actor fetches and parses robots.txt and sitemap.xml files for any list of websites. It extracts crawl directives (user-agent rules, allowed/disallowed paths, crawl-delay), discovers sitemap URLs, and counts the number of pages listed in each sitemap. Use it for SEO audits, competitive...

Sitemap Structure Analyzer

onescales/sitemap-structure-analyzer

Analyze any website's sitemap in seconds using sitemap.xml data. Get URL counts by type (product, blog, docs), content freshness, URL patterns, and SEO anomalies — no page fetching required.

🗺️ Sitemap Scraper & Analyzer

taroyamada/sitemap-analyzer

Extract thousands of URLs from complex nested XML sitemaps to audit website structure, HTTP status codes, and indexability for technical SEO.

Sitemap Analyzer — Recursive Parse, Health Check, AI Tags

and_krm/sitemap-analyzer

Parse any sitemap.xml recursively, extract all URLs with metadata, check HTTP health status, and optionally cluster URLs by topic using Claude AI. Perfect for SEO audits and site migration.

Sitemap Analyzer — Parse, Validate & Check URLs

accurate_pouch/sitemap-analyzer

Parse XML sitemaps, extract all URLs, validate structure (priority, changefreq, lastmod), optionally check HTTP status of every URL. Supports sitemap indexes.

Domain Age Checker

automation-lab/domain-age-checker

Check domain age and registration dates in bulk using RDAP — the modern WHOIS replacement. Returns registration date, expiration date, registrar, nameservers, and age in days/years. Ideal for SEO analysis, fraud detection, and domain research.

Sitemap URL Extractor

automation-lab/sitemap-url-extractor

This actor parses XML sitemaps and extracts all URLs with their metadata. It handles both regular sitemaps and sitemap indexes (recursively follows child sitemaps up to 3 levels deep). For each URL, it captures the last modified date, change frequency, priority, and whether the entry...

Redirect Chain Analyzer

automation-lab/redirect-chain-analyzer

This actor follows HTTP redirects step-by-step and reports the full chain for any list of URLs. It also tests HTTP-to-HTTPS and www/non-www variants to verify canonical redirects are properly configured. Use it for SEO migration audits, redirect debugging, or monitoring redirect health.

Structured Data Scraper & Validator

taroyamada/structured-data-validator

Crawl websites to extract JSON-LD and Microdata, validate schema markup syntax, and flag missing fields across massive URL lists.

Tech Stack Detector — Bulk Website Technology Lookup

accurate_pouch/tech-stack-detector

Detect frameworks, CMS, analytics, CDN, hosting, chat tools, and more for any list of URLs. No API key needed — reads public HTTP headers and HTML.