# Robots.txt and Sitemap Analyzer

**Use case:** 

Analyze robots.txt directives and sitemap URLs for common websites to understand crawl permissions and indexable pages.

## Input

```json
{
  "urls": [
    "apify.com",
    "github.com"
  ],
  "parseSitemaps": true,
  "maxSitemapUrls": 1000
}
```

## Output

```json
{
  "url": {
    "label": "Website"
  },
  "totalSitemapUrls": {
    "label": "Total URLs in Sitemaps"
  },
  "checkTimeMs": {
    "label": "Check (ms)"
  },
  "error": {
    "label": "Error"
  },
  "checkedAt": {
    "label": "Checked At"
  }
}
```

## About this Actor

This example demonstrates how to use [Robots.txt & Sitemap Analyzer](https://apify.com/automation-lab/robots-sitemap-analyzer) with a specific input configuration. Visit the [Actor detail page](https://apify.com/automation-lab/robots-sitemap-analyzer) to learn more, explore other use cases, and run it yourself.