Pricing

from $4.00 / 1,000 site auditeds

Sitemap & Robots.txt Auditor

Checks robots.txt and XML sitemaps for a site: crawlability issues, sitemap validity, and broken sitemap URLs. Fast, no browser needed.

Pricing

from $4.00 / 1,000 site auditeds

Rating

0.0

(0)

Developer

Zaden

Actor stats

Bookmarked

Total users

Monthly active users

19 hours ago

Last modified

Sitemap & Robots.txt Auditor API through CLI

The Apify CLI is the official tool that allows you to use Sitemap & Robots.txt Auditor locally, providing convenience functions and automatic retries on errors.

Install Apify CLI

Using installation script (macOS/Linux):

$curl -fsSL https://apify.com/install-cli.sh | bash

Using installation script (Windows):

$irm https://apify.com/install-cli.ps1 | iex

Using Homebrew:

$brew install apify-cli

Using npm:

$npm install -g apify-cli

Other API clients include:

Sitemap & Robots.txt Auditor API in Python

Sitemap & Robots.txt Auditor API in JavaScript

Sitemap & Robots.txt Auditor OpenAPI definition

Sitemap & Robots.txt Auditor API

Robots.txt Policy Auditor

junipr/robots-txt-policy-auditor

Fetch and audit robots.txt files, parse crawl directives, test important paths, and report sitemap and policy risks.

junipr

Robots.txt Auditor & Sitemap Finder

andok/robotstxt-auditor

Scan robots.txt files in bulk to extract sitemap URLs and verify crawler directives for technical SEO compliance.

Andok

Robots.txt & Sitemap Auditor

phoenix2810/robots-sitemap-auditor

Audit a website's robots.txt directives and sitemap discovery in one fast API call. Detect blocked paths, missing sitemaps, and crawlability issues. Built for SEO agencies, technical SEO consultants, and site migration QA teams.

Sanskar Jaiswal

Sitemap & Robots SEO Index Auditor

glowing_glove/sitemap-robots-index-auditor

Audit robots.txt, sitemap discovery, indexability signals, canonical tags, and SEO crawl readiness for business websites.

Ushba Khan

Website Crawl Preflight Auditor

sebastian-actors/website-crawl-preflight-auditor

Check robots.txt, sitemaps, static extraction quality, crawl risk, and estimated Apify costs before running a crawler or RAG ingestion job.

Sebastián S

Robots Policy Auditor

frank_conifer_af4/robots-policy-auditor

Audit robots.txt and crawl policy signals for domains before scraping, crawling, SEO checks, or data collection.

Juddroid

SSL/TLS Certificate Expiry Monitor

gp005/tls-cert-monitor

Bulk-check TLS certificates for any HTTPS hosts: days-to-expiry, expiry alerts, and issuer/SAN/fingerprint change detection. Pure protocol, no scraping.

Geo

Lighthouse Batch Auditor — Performance, SEO & Core Web Vitals

accurate_pouch/lighthouse-auditor

Batch Lighthouse audits via Google PageSpeed Insights API. Performance, SEO, accessibility scores + Core Web Vitals (LCP, FID, CLS) for 100s of URLs. 3 audits free per run.

Manchitt Sanan

Website SEO, Health & Broken Link Auditor

fascinating_lentil/website-health-seo-broken-link-auditor

Audit authorized websites for technical SEO, broken links, redirects, sitemap and indexability issues, TLS, security headers, PageSpeed, and prior-run regressions. Returns per-page evidence plus HTML, PDF, JSON, and AI-ready summaries.

Md Jakaria Mirza

Sitemap URL Extractor: XML Sitemap Checker, Auditor & SEO Audit

scrapevanta/sitemap-url-extractor

Extracts every URL from a sitemap or sitemap index, auto-discovers sitemaps from robots.txt, handles gzipped sitemaps, filters by URL pattern and last-modified date, and reports site structure and content freshness analytics.