Pricing

from $4.00 / 1,000 site auditeds

Sitemap & Robots.txt Auditor

Checks robots.txt and XML sitemaps for a site: crawlability issues, sitemap validity, and broken sitemap URLs. Fast, no browser needed.

Pricing

from $4.00 / 1,000 site auditeds

Rating

0.0

(0)

Developer

Zaden

Actor stats

Bookmarked

Total users

Monthly active users

19 hours ago

Last modified

Categories

SEO tools

Sites to audit

startUrls

Required

One or more site URLs (e.g. https://example.com). robots.txt and any referenced sitemaps are fetched from each site's origin.

Type:array

URLs to spot-check per site

maxUrlsToSample

Optional

How many URLs listed in the sitemap(s) to check with a live request to detect broken/dead entries. Set to 0 to skip spot-checking. Checked concurrently within each site.

Type:integer

Minimum:0

Default:15

Max child sitemaps to follow

maxChildSitemaps

Optional

If robots.txt or the sitemap points to a sitemap index, how many child sitemap files to fetch and aggregate (largest sites split into many files).

Type:integer

Minimum:0

Default:5

Request timeout (seconds)

requestTimeoutSecs

Optional

How long to wait for each HTTP request before giving up.

Type:integer

Minimum:1

Default:15

Concurrent site audits

concurrency

Optional

How many sites to audit in parallel. Higher values finish faster at the same cost; lower values are gentler if you're auditing many sites on the same shared host.

Type:integer

Minimum:1

Maximum:15

Default:5

Robots.txt Policy Auditor

junipr/robots-txt-policy-auditor

Fetch and audit robots.txt files, parse crawl directives, test important paths, and report sitemap and policy risks.

junipr

Robots.txt Auditor & Sitemap Finder

andok/robotstxt-auditor

Scan robots.txt files in bulk to extract sitemap URLs and verify crawler directives for technical SEO compliance.

Andok

Robots.txt & Sitemap Auditor

phoenix2810/robots-sitemap-auditor

Audit a website's robots.txt directives and sitemap discovery in one fast API call. Detect blocked paths, missing sitemaps, and crawlability issues. Built for SEO agencies, technical SEO consultants, and site migration QA teams.

Sanskar Jaiswal

Sitemap & Robots SEO Index Auditor

glowing_glove/sitemap-robots-index-auditor

Audit robots.txt, sitemap discovery, indexability signals, canonical tags, and SEO crawl readiness for business websites.

Ushba Khan

Website Crawl Preflight Auditor

sebastian-actors/website-crawl-preflight-auditor

Check robots.txt, sitemaps, static extraction quality, crawl risk, and estimated Apify costs before running a crawler or RAG ingestion job.

Sebastián S

Robots Policy Auditor

frank_conifer_af4/robots-policy-auditor

Audit robots.txt and crawl policy signals for domains before scraping, crawling, SEO checks, or data collection.

Juddroid

SSL/TLS Certificate Expiry Monitor

gp005/tls-cert-monitor

Bulk-check TLS certificates for any HTTPS hosts: days-to-expiry, expiry alerts, and issuer/SAN/fingerprint change detection. Pure protocol, no scraping.

Geo

Lighthouse Batch Auditor — Performance, SEO & Core Web Vitals

accurate_pouch/lighthouse-auditor

Batch Lighthouse audits via Google PageSpeed Insights API. Performance, SEO, accessibility scores + Core Web Vitals (LCP, FID, CLS) for 100s of URLs. 3 audits free per run.

Manchitt Sanan

Website SEO, Health & Broken Link Auditor

fascinating_lentil/website-health-seo-broken-link-auditor

Audit authorized websites for technical SEO, broken links, redirects, sitemap and indexability issues, TLS, security headers, PageSpeed, and prior-run regressions. Returns per-page evidence plus HTML, PDF, JSON, and AI-ready summaries.

Md Jakaria Mirza

Sitemap URL Extractor: XML Sitemap Checker, Auditor & SEO Audit

scrapevanta/sitemap-url-extractor

Extracts every URL from a sitemap or sitemap index, auto-discovers sitemaps from robots.txt, handles gzipped sitemaps, filters by URL pattern and last-modified date, and reports site structure and content freshness analytics.