Sitemap to URL Crawler: RAG & AI Data Feeder

Pricing

from $0.50 / 1,000 results

Sitemap to URL Crawler: RAG & AI Data Feeder

nstantly extract all public URLs from any website's sitemap.xml recursively. Handles nested sitemap indexes automatically. The fastest & cheapest way to build URL lists for RAG pipelines, LLM training, and SEO audits. Zero-config & blazing fast.

Pricing

from $0.50 / 1,000 results

Rating

0.0

(0)

Developer

Logiover Data

Maintained by Community

Actor stats

Bookmarked

Total users

Monthly active users

a day ago

Last modified

Categories

Automation

SEO tools

You can access the Sitemap to URL Crawler: RAG & AI Data Feeder programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = { "startUrls": [{ "url": "https://apify.com/sitemap.xml" }] }
9
10# Run the Actor and wait for it to finish
11run = client.actor("logiover/sitemap-to-url-crawler").call(run_input=run_input)
12
13# Fetch and print Actor results from the run's dataset (if there are any)
14print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
15for item in client.dataset(run["defaultDatasetId"]).iterate_items():
16    print(item)
17
18# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

Sitemap to URL Scraper: Extract Links for AI & RAG API in Python

The Apify API client for Python is the official library that allows you to use Sitemap to URL Crawler: RAG & AI Data Feeder API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

Sitemap to URL Crawler: RAG & AI Data Feeder API in JavaScript

Sitemap to URL Crawler: RAG & AI Data Feeder API through CLI

Sitemap to URL Crawler: RAG & AI Data Feeder OpenAPI definition

Sitemap to URL Crawler: RAG & AI Data Feeder API

Sitemap Scraper

pvillalva/sitemap-scraper

The Sitemap Scraper extracts and outputs all URLs from a given sitemap.

Percival Villalva

187

5.0

Sitemap URL Extractor

onescales/sitemap-url-extractor

Provide a website link to a sitemap.xml and the app will extract and list all URLs in the sitemap as well as additional data in the sitemap (i.e. https://onescales.com/sitemap.xml).

One Scales

322

5.0

Sitemap API

vivid_astronaut/sitemap

Fabio Suizu

Sitemap Generator

datawinder/sitemap-generator

Automatically crawl a website and generate an SEO-ready sitemap in XML, HTML, or TXT format. Supports crawl depth limits, URL include/exclude patterns, and optional merging with an existing sitemap.xml. Ideal for SEO audits, site migrations, and automation.

Datawinder

Find Sitemap from url

eesti/find-sitemap-from-url

A powerful [Apify Actor] that finds sitemap URLs for any website. This Actor helps you discover XML sitemaps by checking common locations, robots.txt files, and analyzing HTML content for sitemap links.

ando

177

1.0

Sitemap Detector

coder_zoro/sitemap-detector

Find sitemap URLs fast with our free Sitemap Finder tool. Instantly detect sitemaps from any website for SEO audits, indexing checks, and crawl planning. Improve visibility, site structure insights, and search engine performance in just seconds

Zoro

146

5.0

Sitemap Generator - Creates sitemap.xml for any domain

wisteria_banjo/sitemap-generator---creates-sitemap-xml-for-any-domain

Generate a clean, standards-compliant sitemap.xml for a website. This actor crawls a single website, discovers all indexable pages, and produces: ✅ A ready-to-submit sitemap.xml (Google-compliant) ✅ A structured JSON dataset of discovered URLs (for auditing, reporting, and billing)

Chris Xavier

Example Sitemap Cheerio

jancurn/example-sitemap-cheerio

An example actor that first downloads a sitemap in XML format and the crawls each page from the sitemap using the fast CheerioCrawler from Apify SDK.

Jan Čurn

AI RAG Feeder V2

mickeywmoore/ai-rag-feeder-v2

Turn any website into AI-ready Markdown. Scrapes entire domains, removes ads/clutter, and formats text specifically for RAG pipelines and LLM training data.