# Custom ecommerce robots.txt workflow

**Use case:** 

Build custom robots.txt rules for ecommerce sites with crawl delays, private-path blocks, public-path allows, and sitemaps.

## Input

```json
{
  "preset": "none",
  "rules": [
    {
      "userAgent": "*",
      "allow": [
        "/",
        "/products/",
        "/collections/",
        "/blog/"
      ],
      "disallow": [
        "/cart",
        "/checkout",
        "/account",
        "/admin",
        "/search"
      ],
      "crawlDelay": 2,
      "comment": "Default ecommerce crawling rules"
    },
    {
      "userAgent": "Googlebot",
      "allow": [
        "/"
      ],
      "disallow": [
        "/checkout",
        "/account"
      ],
      "comment": "Allow Googlebot broad product discovery"
    }
  ],
  "sitemaps": [
    "https://shop.example.com/sitemap.xml"
  ],
  "host": "shop.example.com",
  "includeTimestamp": true,
  "includeGeneratorComment": true,
  "validateRules": true
}
```

## Output

```json
{
  "preset": {
    "label": "Preset",
    "format": "text"
  },
  "ruleBlockCount": {
    "label": "Rule Blocks",
    "format": "number"
  },
  "sitemapCount": {
    "label": "Sitemaps",
    "format": "number"
  },
  "lineCount": {
    "label": "Lines",
    "format": "number"
  },
  "warnings": {
    "label": "Warnings",
    "format": "text"
  },
  "success": {
    "label": "Success",
    "format": "boolean"
  },
  "error": {
    "label": "Error",
    "format": "text"
  },
  "generatedAt": {
    "label": "Generated At",
    "format": "text"
  }
}
```

## About this Actor

This example demonstrates how to use [Robots.txt Generator](https://apify.com/automation-lab/robots-txt-generator) with a specific input configuration. Visit the [Actor detail page](https://apify.com/automation-lab/robots-txt-generator) to learn more, explore other use cases, and run it yourself.