You can access the AI Crawler Policy Auditor programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = {}
9
10# Run the Actor and wait for it to finish
11run = client.actor("junipr/ai-crawler-policy-auditor").call(run_input=run_input)
12
13# Fetch and print Actor results from the run's dataset (if there are any)
14print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
15for item in client.dataset(run["defaultDatasetId"]).iterate_items():
16    print(item)
17
18# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

AI Crawler Policy Auditor — technical SEO, web, and domain API in Python

The Apify API client for Python is the official library that allows you to use AI Crawler Policy Auditor API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

AI Crawler Policy Auditor API in JavaScript

AI Crawler Policy Auditor API through CLI

AI Crawler Policy Auditor OpenAPI definition

AI Crawler Policy Auditor API

Robots.txt & AI Crawler Policy Auditor

merlin-hermes/robots-ai-crawler-policy-auditor

Parses a site's public robots.txt into structured crawler groups, allow/disallow counts, crawl delays, sitemap declarations, and explicit or inherited policies for major AI crawlers.

Sean Burke

LLMS.txt Auditor

junipr/llms-txt-auditor

Check for llms.txt / AI crawler policy guidance and produce structured readiness reports.

junipr

AI Bot & LLM Crawler Exposure Auditor

glowing_glove/ai-bot-llm-crawler-exposure-auditor

Audit robots.txt, llms.txt, AI crawler directives, public content exposure, and policy gaps for websites in the AI search era.

Ushba Khan

GEO / AEO Website Audit — AI Search Readiness

rtworule/ai-search-readiness-auditor

Audit public websites for GEO and AEO readiness: AI crawler access, robots.txt, llms.txt, sitemaps, Schema.org, social metadata, scores, and prioritized fixes.

Kunteper Koyu

Mexico RFC Validator | SAT Taxpayer ID Format Check

parseforge/mexico-rfc-scraper

Validate Mexican RFC taxpayer identifiers in bulk. Check format, embedded date, and homoclave checksum using the SAT modulo-11 algorithm for persona fisica and persona moral. Returns isValid plus per-check details. CSV, Excel, JSON, XML for KYC and CFDI invoice workflows.

ParseForge

Wayback Machine Scraper - Track Website Changes Over Time

ryanclinton/wayback-machine-search

Search the Internet Archive's Wayback Machine for historical snapshots of any website. Retrieve archived page metadata -- including timestamps, URLs, MIME types, HTTP status codes, and content hashes -- for up to 10,000 snapshots per run.

Ryan Clinton

107

SEC EDGAR Scraper

constructive_calm/sec-edgar-scraper

Fetches SEC EDGAR filings, normalized financial statements, XBRL facts, insider trades (Form 4), institutional holdings (13F), activist stakes (13D/G), full-text search hits, and parsed 8-K items via official SEC APIs. Zero anti-bot. Optional AI summaries.

Omar Eldeeb

5.0

(1)

LinkedIn Search Jobs Scraper

api-empire/linkedin-search-jobs-scraper

LinkedIn Search Jobs Scraper extracts job listings from any LinkedIn search query. Capture titles, companies, locations, descriptions, salaries, and posting dates. Ideal for market research, hiring, lead generation, and workflows needing structured LinkedIn job search data.

API Empire

Schema Markup Scraper & SEO Auditor

autofacts/metadata-scraper

Extract JSON-LD, Microdata, RDFa, Open Graph & Twitter Cards. Runs a 0-100 SEO audit — checks canonical, hreflang, headings, image alt, EEAT author signals. Detects 80+ schema.org types including LocalBusiness with NAP, geo coordinates, and Google Place IDs.

Richard Feng

158

5.0

(1)

Advanced SEO Audit Tool - AI-Powered Insights

ittechinnovators/advanced-seo-audit-tool---ai-powered-insights

Perform end-to-end SEO, performance, UX, social media, and payment gateway analysis. Generate actionable recommendations to enhance website visibility, user experience, and ranking. Optimized for all business types with AI-driven suggestions.