Public URLs to convert into Markdown, document JSONL, and vector-ready chunks. The default is a small live Apify docs sample so Run works immediately.

Type:array

Default:

[
  {
    "url": "https://docs.apify.com/academy/getting-started"
  },
  {
    "url": "https://docs.apify.com/academy/web-scraping-for-beginners"
  },
  {
    "url": "https://docs.apify.com/academy/actor-marketing-playbook/actor-basics/actor-description"
  }
]

Output preset

outputPreset

Optional

Naming and metadata preset for downstream ingestion.

Type:string

Default:openai_vector_store

Options:

openai_vector_storelangchainllamaindexpineconeqdrantgeneric_jsonl

Include URL patterns

includeUrlPatterns

Optional

Only process URLs containing at least one of these strings. Leave empty to allow every URL in the list.

Type:string[]

Default:

[
  "/academy/"
]

Exclude URL patterns

excludeUrlPatterns

Optional

Skip URLs containing any of these strings.

Type:string[]

Default:

[]

Max URLs

maxUrls

Optional

Maximum number of URLs from the input list to process in one run.

Type:integer

Minimum:1

Maximum:5000

Default:3

Include markdown

includeMarkdown

Optional

Include clean Markdown content in page and document records.

Type:boolean

Default:true

Chunk size characters

chunkSizeChars

Optional

Target character length for generated text chunks.

Type:integer

Minimum:500

Maximum:12000

Default:2500

Chunk overlap characters

chunkOverlapChars

Optional

Character overlap between adjacent chunks.

Type:integer

Minimum:0

Maximum:2000

Default:250

Request timeout ms

requestTimeoutMs

Optional

Per-request timeout in milliseconds.

Type:integer

Minimum:1000

Maximum:60000

Default:15000

Dry run

dryRun

Optional

Emit bundled demo URL records without fetching or charging events. Leave off for the live default sample.

Type:boolean

Default:false

Fast Website to Markdown & RAG JSONL Crawler

orbiscribe/website-rag-dataset-builder

Paste a homepage or sitemap and get clean Markdown, metadata, JSONL chunks, and source URLs for RAG at a low per-page price.

Orbiscribe Labs

Docs & Help Center to RAG JSONL

orbiscribe/docs-help-center-rag-snapshot

Paste a docs or help center URL and get clean Markdown, breadcrumbs, page records, and JSONL chunks for RAG.

Orbiscribe Labs

Website & PDF to RAG JSONL Crawler

orbiscribe/linked-pdf-website-rag-crawler

Paste webpage and PDF URLs and get Markdown, JSONL chunks, PDF inventory, source warnings, and RAG-ready records.

Orbiscribe Labs

Sitemap to Changed-Only RAG JSONL

orbiscribe/sitemap-to-rag-delta-dataset

Crawl sitemap.xml files and emit only added, changed, or deleted Markdown/JSONL chunks for cheaper RAG reindexing.

Orbiscribe Labs

AI Training Data Scraper - LLM and RAG-Ready

george.the.developer/ai-training-data-scraper

Extract web content formatted for LLM fine-tuning and RAG pipelines. Output in OpenAI JSONL, Claude JSONL, Markdown, or raw text.

George Kioko

Vector Loader — Document Embedding & Vector DB Ingestion

apricot_blackberry/vector-loader

Vector Loader — Document Embedding & Vector DB Ingestion helps teams get quick, high-signal results with reliable output, clear fields, and fast setup.

Creator Fusion

PDF to Text API | Document Extraction for LLMs & RAG

andok/pdf-text-converter

Convert bulk PDF documents via URL into clean, raw text. The perfect document scraper for LLMs, vector databases, and RAG pipelines.

Andok

Website to Markdown for LLM and RAG

jeweled_jockstrap/my-actor-3

Convert any URL to clean Markdown text for AI applications. Strips HTML extracts content. For LLM training RAG pipelines and vector databases. Free Firecrawl alternative.

Juan Triviño

Docs-to-RAG Optimizer

vamsi-krishna/docs-to-rag-optimizer

Convert public developer documentation into clean Markdown, semantic RAG chunks, token counts, duplicate hashes, JSONL exports, and quality warnings for AI assistants.

Vamsi Krishna

RAG Ingestor: Multi-Source Chunks for Vector DBs

aitoolbreakdown/atb-rag-ingestor

Ingest URLs, sitemaps, and GitHub READMEs into uniform chunks with titles, source URLs, and stable IDs. Ready to push straight into Pinecone, Weaviate, or any RAG pipeline.