Sitemap & Robots SEO Index Auditor

Sitemap Extractor

apify/sitemap-extractor

This Apify Actor extracts all URLs from a website's sitemaps and checks their status codes via lightweight HTTP requests. It provides a clean list of valid links, acting as an ideal pre-processor to ensure your larger crawling projects target only active URLs.

Expired Domains Finder — GoDaddy Auctions with SEO Metrics

hipersoft/godaddy-auctions-scraper

Find expired and auction domains from GoDaddy Auctions with price, bids, age, valuation, Majestic Trust/Citation Flow, backlinks, referring domains, SEMrush Authority Score, keyword search volume and CPC. Filter by TLD, keyword, price, age and SEO metrics. Export CSV/JSON.

Robots.txt & Sitemap Analyzer

automation-lab/robots-sitemap-analyzer

This actor fetches and parses robots.txt and sitemap.xml files for any list of websites. It extracts crawl directives (user-agent rules, allowed/disallowed paths, crawl-delay), discovers sitemap URLs, and counts the number of pages listed in each sitemap. Use it for SEO audits, competitive...

Instagram Profile Metrics Scraper

glowing_glove/instagram-profile-analyzer

Collect lightweight public Instagram profile metrics: username, bio, follower count, following count, and post count.

Extract all URLs from any website's sitemap. Auto-discovers sitemaps from robots.txt, supports sitemap index files and .gz compression. Filter by URL pattern, date range.

XML Sitemap Finder & Extractor API

andok/find-sitemap-from-url

Find and extract all XML sitemaps for any domain. Automatically parses robots.txt, scans HTML tags, and recursively follows indexes. Perfect for SEO & web scraping.

Wikipedia Change Intelligence

glowing_glove/wikipedia-change-intelligence

Watch Wikipedia entities and return summaries, revision timestamps, changed sections, links, and knowledge-base update signals.

AI Bot & LLM Crawler Exposure Auditor

glowing_glove/ai-bot-llm-crawler-exposure-auditor

Audit robots.txt, llms.txt, AI crawler directives, public content exposure, and policy gaps for websites in the AI search era.

Broken Link Checker

jancurn/find-broken-links

Crawls a website and finds broken links. Unlike other similar SEO analysis tools, the actor also reports broken URL #fragments. The results are stored in a JSON and HTML report.

Local Service Lead Gap Finder

glowing_glove/local-service-lead-gap-finder

Find agency sales opportunities on local service websites by scoring booking gaps, review gaps, local SEO gaps, trust signals, and conversion paths.