AusBizStack AI Search (GEO), llms.txt Generator & Schema Auditor (ausbizstack/ai-geo-llms-txt-auditor)

Audit any list of US, UK, EU, Canadian, or Australian websites for 2026 AI Search / Generative Engine Optimization (GEO) readiness—and automatically generate ready-to-deploy /llms.txt, /llms-full.txt, and <script type="application/ld+json"> (Organization + WebSite + FAQPage) deliverables for every domain in one run.

Built by AusBizStack as the companion deliverable engine to ausbizstack/b2b-lead-abn-tech-enricher for SEO/GEO agencies, Clay / n8n / Make.com workflows, and MCP AI Agents (mcp.apify.com).

---

What This Actor Does in One Run

1. Audits /robots.txt Across 9 Major AI Crawlers:
   • Inspects explicit and wildcard rules for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-Web, PerplexityBot, Google-Extended, CCBot, and Applebot-Extended.
   • Identifies whether a prospect is accidentally blocking ChatGPT Search, Claude, or Perplexity from citing their business.
2. Verifies Live /llms.txt, /llms-full.txt & /sitemap.xml:
   • Checks HTTP 200 status, validates Markdown structure (filtering out soft-404 HTML pages), and counts sections and curated links.
3. Extracts Homepage & Subpage Metadata + Existing Schema.org JSON-LD:
   • Extracts <title>, meta descriptions, canonical URLs, H1/H2 hierarchy, top internal navigation/documentation links, social profiles, strictly role-based corporate emails (info@, sales@, support@, bookings@ — zero personal emails), phone numbers, and Australian 11-digit ABNs (validated via the ATO modulo-89 checksum).
   • Inspects existing application/ld+json blocks for Organization, LocalBusiness, WebSite, FAQPage, Product, and Service.
4. Scores Every Domain (aiGeoReadinessScore & agencyPitchScore):
   • Computes a 0–100 aiGeoReadinessScore, a 0–100 agencyPitchScore, and a structured identifiedGaps checklist.
5. Auto-Generates Ready-to-Deploy Client Deliverables:
   • generatedLlmsTxt: Complete, spec-compliant /llms.txt Markdown file tailored to the business.
   • generatedLlmsFullTxt: Expanded /llms-full.txt reference corpus for LLM RAG ingestion.
   • generatedJsonLdSchema: Copy-paste <script type="application/ld+json"> containing valid Organization + WebSite + FAQPage Schema.org markup.
   • agencyDeliverableSummary: Client-ready executive audit summary and pitch angle so agencies can close $500–$1,500 AI Search Optimization Packages in seconds.

---

Input Example

{
  "targets": [
    "https://www.canva.com",
    "https://stripe.com",
    "https://www.xero.com",
    "https://dishoom.com"
  ],
  "crawlInternalPages": true,
  "maxInternalPages": 2,
  "generateLlmsTxt": true,
  "generateLlmsFullTxt": true,
  "generateJsonLdSchema": true,
  "concurrency": 4,
  "requestTimeoutSecs": 12
}

---

Output Example (Abbreviated)

{
  "inputTarget": "https://dishoom.com",
  "domain": "dishoom.com",
  "websiteUrl": "https://www.dishoom.com/",
  "companyName": "Dishoom",
  "countryDetected": "UK",
  "status": "ok",
  "pagesInspected": 3,
  "aiGeoReadinessScore": 55,
  "agencyPitchScore": 90,
  "hasLlmsTxt": false,
  "hasLlmsFullTxt": false,
  "hasRobotsTxt": true,
  "hasSitemapXml": true,
  "allowedAiCrawlers": [
    "GPTBot",
    "OAI-SearchBot",
    "ChatGPT-User",
    "ClaudeBot",
    "Claude-Web",
    "PerplexityBot",
    "Google-Extended",
    "CCBot",
    "Applebot-Extended"
  ],
  "blockedAiCrawlers": [],
  "existingJsonLdTypes": ["Organization", "WebSite"],
  "identifiedGaps": [
    "Missing /llms.txt Markdown index — AI agents (ChatGPT, Claude, Perplexity) must scrape noisy DOM HTML.",
    "Missing /llms-full.txt expanded RAG documentation corpus.",
    "Missing FAQPage Schema.org JSON-LD markup for direct Q&A citation in AI Overviews."
  ],
  "roleEmails": ["hello@dishoom.com", "press@dishoom.com", "reservations@dishoom.com"],
  "phones": ["+44 20 7420 9320"],
  "agencyDeliverableSummary": "AI Search (GEO) Readiness Audit for Dishoom (dishoom.com): Current Score 55/100 (Projected 92/100 after deploying attached deliverables)...",
  "generatedLlmsTxt": "# Dishoom\n\n> Dishoom pays homage to the Irani cafés of Bombay...",
  "generatedLlmsFullTxt": "# Dishoom\n\n> ...\n\n## Extended RAG Knowledge Base & Entity Profile (/llms-full.txt)...",
  "generatedJsonLdSchema": "<script type=\"application/ld+json\">\n{\n  \"@context\": \"https://schema.org\",\n  \"@graph\": [...]\n}\n</script>"
}

API authentication

Pass your token as a header, never in the URL:

curl -X POST "https://api.apify.com/v2/acts/ausbizstack~ai-geo-llms-txt-auditor/run-sync-get-dataset-items" \
  -H "Authorization: Bearer YOUR_APIFY_TOKEN" -H "Content-Type: application/json" -d '{...}'

✅ Zero Charge for Failed Domains

You are charged only when real intelligence data is returned. Failed, timed-out or unreachable targets still appear in the dataset with status and an error reason, but cost $0.

Acceptable Use & Spam Act 2003 (Cth) Compliance

This Actor returns publicly available business information only. You are responsible for how you use it. If you send commercial electronic messages to Australian recipients you must comply with the Spam Act 2003 (Cth): have consent (express or inferred), identify yourself accurately, and include a functional unsubscribe. Respect any website notice stating that unsolicited messages are not wanted, and comply with the Privacy Act 1988 (Cth), GDPR and other applicable laws. Do not use this Actor for bulk unsolicited messaging.