Sitemap to AI/RAG URL Feed

XML Sitemap Scraper & URL Extractor API - SEO Crawler

pink_comic/sitemap-url-extractor

Parse XML website sitemaps and extract URLs for SEO crawls, audits, migrations, and RAG indexing. Auto-discovers sitemap files from robots.txt, follows nested indexes, and exports URL, lastmod, priority, changefreq, and image metadata.

Sitemap URL Finder

thescrapelab/sitemap-target-url-extractor

Find useful URLs from robots.txt and sitemaps, then output deduplicated URL inventory rows with source sitemap, domain, path, filter, and crawl summary data for SEO and scraping workflows.

Collect URLs from XML sitemaps for SEO and crawling.

Site to Agent Feed (URL to RAG-ready Markdown)

constant_quadruped/site-to-agent-feed

Turn any URL into clean, RAG-ready Markdown + structured JSON for LLMs and AI agents. Self-healing main-content extraction (survives redesigns), headings/links/tables, optional change-detection. No paid APIs.

Sitemap Scraper

pvillalva/sitemap-scraper

The Sitemap Scraper extracts and outputs all URLs from a given sitemap.

Sitemap Extractor - Hardened URL Inventory

8tp/sitemap-url-extractor-coverage-api

Extract URL inventories from public HTTPS sitemap XML and gzip files with robots.txt discovery, bounded index traversal, and machine-readable coverage receipts. No page crawling, login, browser, proxy, or private-network access.

Sitemap URL Extractor - XML Sitemap Scraper

benthepythondev/sitemap-url-extractor

Extract URLs from XML sitemaps and sitemap indexes. Get URL, lastmod, changefreq, priority and source sitemap.

Sitemap URL Exporter — Nested Sitemaps & Robots Discovery

gertner-data/sitemap-url-exporter

Export up to 3,000 deduplicated URLs per website from robots.txt and nested XML sitemaps, with lastmod and source sitemap. Feed crawlers, migrations and ETL from one JSON report. Launch price: $1/1,000 website reports; all exported URLs included.

Sitemap URL Extractor - List All URLs in a Sitemap

dltik/sitemap-url-extractor

Extract every URL from any XML sitemap, with lastmod, changefreq and priority. Resolves sitemap indexes recursively. Pass a sitemap.xml or just a site root to auto-discover its sitemaps. Pure HTTP, no browser — fast and cheap.

Give it any website or sitemap URL and get back every page URL. It reads robots.txt and sitemap.xml, follows nested sitemap indexes, and returns each URL with its last-modified date, change frequency and priority as clean JSON. No key, login or proxy required.