Pricing

Pay per usage

Sitemap & URL Extractor — Get Every URL of a Website

Get every URL of a website: parses sitemap.xml and sitemap-indexes (discovered via robots.txt or the default location), with a same-site crawl fallback when there's no sitemap. Returns each URL + lastmod. No API key.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Daniel Brenner

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

Sitemap & URL Extractor — Get Every URL of a Website API

Below, you can find a list of relevant HTTP API endpoints for calling the Sitemap & URL Extractor — Get Every URL of a Website Actor. For this, you’ll need an Apify account. Replace <YOUR_API_TOKEN> in the URLs with your Apify API token, which you can find under Integrations in Apify Console. For details, see the API reference.

Run Actor

POST
https://api.apify.com/v2/acts/dataquarry~sitemap-url-extractor/runs?token=<YOUR_API_TOKEN>

Note: By adding the method=POST query parameter, this API endpoint can be called using a GET request and thus used in third-party webhooks. Please refer to our Run Actor API documentation.

Run Actor synchronously and get dataset items

POST
https://api.apify.com/v2/acts/dataquarry~sitemap-url-extractor/run-sync-get-dataset-items?token=<YOUR_API_TOKEN>

Note: This endpoint supports both POST and GET request methods. However, only the POST method allows you to pass input data. For more information, please refer to our Run Actor synchronously and get dataset items API documentation.

Get Actor

GET
https://api.apify.com/v2/acts/dataquarry~sitemap-url-extractor?token=<YOUR_API_TOKEN>

For more information, please refer to our Get Actor API documentation.

Actors can be used to scrape web pages, extract data, or automate browser tasks. Use the Sitemap & URL Extractor — Get Every URL of a Website API programmatically via the Apify API.

You can choose from:

Sitemap & URL Extractor — Get Every URL of a Website API in Python

Sitemap & URL Extractor — Get Every URL of a Website API in JavaScript

Sitemap & URL Extractor — Get Every URL of a Website API through CLI

Sitemap & URL Extractor — Get Every URL of a Website OpenAPI definition

You can start Sitemap & URL Extractor — Get Every URL of a Website with the Apify API by sending an HTTP POST request to the Run Actorendpoint. An Actor’s input and its content type can be passed as a payload of the POST request, and additional options can be specified using URL query parameters. The Sitemap & URL Extractor — Get Every URL of a Website is identified within the API by its ID, which is the creator’s username and the name of the Actor.

When the Sitemap & URL Extractor — Get Every URL of a Website run finishes you can list the data from its default dataset(storage) via the API or you can preview the data directly on Apify Console.

Sitemap Extractor

cerebral_aluminum/sitemap-extractor

Extract all URLs from website sitemaps. Pages, images, PDFs. Handles sitemap indexes and WordPress.

Benny

Sitemap Structure Analyzer

onescales/sitemap-structure-analyzer

Analyze any website's sitemap in seconds using sitemap.xml data. Get URL counts by type (product, blog, docs), content freshness, URL patterns, and SEO anomalies — no page fetching required.

One Scales

5.0

(2)

Sitemap URL Extractor

mikolabs/sitemap-url-extractor

Extract every URL and its metadata from any sitemap.xml in seconds. Paste one or more sitemap URLs, run the Actor, and get a clean, structured dataset with url, lastmod, changefreq, priority, and more — ready to export as CSV, JSON, or Excel.

mikolabs

JSON-LD & Schema.org Extractor

andok/jsonld-extractor

Extract structured microdata (JSON-LD) from webpages to audit SEO schema implementations and rich snippets.

Andok

Broken Link Checker — Full Website Crawler

logiover/broken-link-checker

Crawl an entire website and check every internal & external link for broken (404/5xx) responses. Get the source page, anchor text and status code for each link. Schedule it to monitor any site for dead links and SEO errors.

Logiover

Sitemap to URL Crawler — Extract Sitemap.xml URLs for RAG

logiover/sitemap-to-url-crawler

Instantly extract all public URLs from any website sitemap.xml recursively. Handles nested sitemap indexes automatically. The fastest cheap way to build URL lists for RAG pipelines, LLM training datasets, SEO audits and content inventories. Zero-config, no proxy required.

Logiover

Sitemap URL Extractor

automation-lab/sitemap-url-extractor

This actor parses XML sitemaps and extracts all URLs with their metadata. It handles both regular sitemaps and sitemap indexes (recursively follows child sitemaps up to 3 levels deep). For each URL, it captures the last modified date, change frequency, priority, and whether the entry...

Stas Persiianenko

Sitemap URL Extractor & Parser

pink_comic/sitemap-url-extractor

Extract all URLs from XML sitemaps. Auto-discovers sitemaps via robots.txt. Handles nested sitemap indexes. Returns URL, last modified date, change frequency, priority, and image metadata. For SEO audits, content migration, and competitive analysis. No API key needed.

Ava Torres

Sitemap Finder & URL Extractor · Crawl Any XML Sitemap

corent1robert/sitemap-detector

Find and crawl XML sitemaps from any website. Follows sitemap indexes, handles gzip, and exports every page URL with source file and lastmod into a clean dataset. No config needed.

Corentin Robert

Website to Markdown & Text Crawler — AI / RAG Data

logiover/website-text-markdown-crawler

Crawl an entire website and extract clean, boilerplate-free main content as Markdown and plain text — ready for LLM training, RAG pipelines, embeddings and AI agents. No login, no browser, one row per page.

Logiover

Website Metadata Extractor(sitemap, socialLinks, robotsTxt)

codescraper/website-metadata-extractor

A very fast metadata extractor to get all meta tags, robots.txt, sitemaps, social links, H1s, word count, and JSON-LD data. Also provides technology detection for a full analysis. Get your data fast for just $3/month.