Public Sitemap & SEO Meta Auditor avatar

Public Sitemap & SEO Meta Auditor

Pricing

$1.00 / 1,000 results

Go to Apify Store
Public Sitemap & SEO Meta Auditor

Public Sitemap & SEO Meta Auditor

Robots-compliant XML sitemap & SEO meta auditor. Checks title/description lengths, canonicals, H1s, and HTTP status codes.

Pricing

$1.00 / 1,000 results

Rating

0.0

(0)

Developer

B2 Apps

B2 Apps

Maintained by Community

Actor stats

0

Bookmarked

1

Total users

0

Monthly active users

8 hours ago

Last modified

Categories

Features

Limited permissions

Share

Public Sitemap & SEO Meta Auditor (Robots-Compliant)

A fast, strictly policy-compliant Apify Actor that audits public websites from their sitemap.xml while honoring robots.txt directives (Disallow, Allow, Crawl-delay, and User-agent matching).

Policy & Compliance

  • Robots-First: Fetches and parses robots.txt before requesting any sitemap or page URL. Disallowed URLs are never fetched and are flagged with BLOCKED_BY_ROBOTS.
  • Public Pages Only: No login-wall circumvention, no CAPTCHA bypassing, no private endpoint probing.
  • Rate-Limit Friendly: Honors Crawl-delay and bounded concurrency.

Input Schema

FieldTypeDefaultDescription
sitemapUrlstring (required)—Public sitemap.xml or sitemap index URL
maxUrlsinteger50Maximum URLs to audit (1–5000)
userAgentstringPublicSitemapSEOAuditor/1.0 (+https://apify.com)User-Agent header & robots token
requestTimeoutMsinteger10000Request timeout in ms
concurrencyinteger3Concurrent page fetches (1–10)
respectCrawlDelaybooleantrueHonor Crawl-delay in robots.txt

Output Example

{
"url": "https://example.com/pricing",
"robotsAllowed": true,
"httpStatus": 200,
"title": "Pricing - Example",
"titleLength": 17,
"metaDescription": "Simple transparent pricing.",
"metaDescriptionLength": 27,
"canonicalUrl": "https://example.com/pricing",
"h1Count": 1,
"h1Texts": ["Pricing"],
"issues": [],
"auditedAt": "2026-11-14T09:00:00.000Z"
}

Local Development & Testing

$npm test