# Block AI crawlers robots.txt generator

**Use case:** 

Generate robots.txt rules that block AI crawlers such as GPTBot, CCBot, and Claude while keeping sitemap directives.

## Input

```json
{
  "preset": "block-ai-crawlers",
  "rules": [
    {
      "userAgent": "*",
      "allow": [
        "/public/",
        "/blog/"
      ],
      "disallow": [
        "/admin/",
        "/private/",
        "/api/"
      ],
      "crawlDelay": 2,
      "comment": "Default rules for all bots"
    },
    {
      "userAgent": "Googlebot",
      "allow": [
        "/"
      ],
      "disallow": [],
      "comment": "Give Googlebot full access"
    }
  ],
  "sitemaps": [
    "https://example.com/sitemap.xml",
    "https://example.com/news-sitemap.xml"
  ],
  "host": "example.com",
  "includeTimestamp": true,
  "includeGeneratorComment": true,
  "validateRules": true
}
```

## Output

```json
{
  "preset": {
    "label": "Preset",
    "format": "text"
  },
  "ruleBlockCount": {
    "label": "Rule Blocks",
    "format": "number"
  },
  "sitemapCount": {
    "label": "Sitemaps",
    "format": "number"
  },
  "lineCount": {
    "label": "Lines",
    "format": "number"
  },
  "warnings": {
    "label": "Warnings",
    "format": "text"
  },
  "success": {
    "label": "Success",
    "format": "boolean"
  },
  "error": {
    "label": "Error",
    "format": "text"
  },
  "generatedAt": {
    "label": "Generated At",
    "format": "text"
  }
}
```

## About this Actor

This example demonstrates how to use [Robots.txt Generator](https://apify.com/automation-lab/robots-txt-generator) with a specific input configuration. Visit the [Actor detail page](https://apify.com/automation-lab/robots-txt-generator) to learn more, explore other use cases, and run it yourself.