# Build an LLM training dataset from a documentation site

**Use case:** 

Crawl a docs site into clean page text for LLM fine-tuning and RAG. Filter by language, remove duplicates, and export JSONL or CSV.

## Input

```json
{
  "startUrls": [
    {
      "url": "https://docs.python.org/3/tutorial/"
    },
    {
      "url": "https://docs.python.org/3/tutorial/appetite.html"
    },
    {
      "url": "https://docs.python.org/3/tutorial/interpreter.html"
    }
  ],
  "sitemapUrl": "https://docs.example.com/sitemap.xml",
  "maxPages": 30,
  "minTextLength": 100,
  "language": "en",
  "deduplicate": true,
  "maxFingerprints": 50000,
  "crawlStrategy": "recurse",
  "stayWithinDomain": true,
  "maxConcurrency": 5,
  "minRequestDelaySecs": 0,
  "exportRejectedPages": false
}
```

## Output

```json
{
  "url": {
    "label": "URL",
    "format": "string"
  },
  "title": {
    "label": "Title",
    "format": "string"
  },
  "text": {
    "label": "Text",
    "format": "string"
  },
  "wordCount": {
    "label": "Words",
    "format": "integer"
  },
  "language": {
    "label": "Language",
    "format": "string"
  },
  "author": {
    "label": "Author",
    "format": "string"
  },
  "publishedDate": {
    "label": "Published",
    "format": "string"
  }
}
```

## About this Actor

This example demonstrates how to use [AI Training Data Curator](https://apify.com/vamsi-krishna/ai-training-data-curator) with a specific input configuration. Visit the [Actor detail page](https://apify.com/vamsi-krishna/ai-training-data-curator) to learn more, explore other use cases, and run it yourself.