Pricing

from $0.05 / 1,000 results

Sitemap URL Finder

Find and export URLs from any website’s robots.txt and sitemaps. Enter a domain or website URL, optionally filter matching URLs by text, and get clean dataset rows with the URL, domain, path, source sitemap, and match details.

Pricing

from $0.05 / 1,000 results

Rating

0.0

(0)

Developer

Inus Grobler

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

Categories

Developer tools

SEO tools

Automation

You can access the Sitemap URL Finder programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {
11    "websites": [
12        {
13            "url": "https://docs.apify.com"
14        }
15    ],
16    "includeUrlText": "",
17    "maxResults": 25
18};
19
20// Run the Actor and wait for it to finish
21const run = await client.actor("thescrapelab/sitemap-target-url-extractor").call(input);
22
23// Fetch and print Actor results from the run's dataset (if any)
24console.log('Results from dataset');
25console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
26const { items } = await client.dataset(run.defaultDatasetId).listItems();
27items.forEach((item) => {
28    console.dir(item);
29});
30
31// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

Sitemap URL Finder | Extract URLs from Website Sitemaps API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use Sitemap URL Finder API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

Sitemap URL Finder API in Python

Sitemap URL Finder API through CLI

Sitemap URL Finder OpenAPI definition

Sitemap URL Finder API

Sitemap Finder & URL Extractor · Crawl Any XML Sitemap

corent1robert/sitemap-detector

Find and crawl XML sitemaps from any website. Follows sitemap indexes, handles gzip, and exports every page URL with source file and lastmod into a clean dataset. No config needed.

Corentin Robert

Sitemap Detector

coder_zoro/sitemap-detector

Find sitemap URLs fast with our free Sitemap Finder tool. Instantly detect sitemaps from any website for SEO audits, indexing checks, and crawl planning. Improve visibility, site structure insights, and search engine performance in just seconds

Zoro

172

5.0

(3)

Property Finder Scraper

crawlerbros/propertyfinder-scraper

Scrape property listings from Property Finder (UAE, Saudi Arabia, Qatar, Bahrain, Egypt). Extract prices, locations, photos, agent info, and 50+ fields per listing. No proxy required.

Crawler Bros

5.0

(21)

Sitemap URL Extractor

onescales/sitemap-url-extractor

Provide a website link to a sitemap.xml and the app will extract and list all URLs in the sitemap as well as additional data in the sitemap (i.e. https://onescales.com/sitemap.xml).

One Scales

523

5.0

(3)

Company Domain & Social Links Finder

crawlerbros/company-domain

Given a company name, return the company's official website domain and its social media links (LinkedIn, X/Twitter, Facebook, Instagram, YouTube, TikTok, GitHub).

Crawler Bros

5.0

(14)

Sitemap Finder & Checker Tool

zerobreak/sitemap-finder-checker-tool

Find, validate, and audit XML sitemaps for any website. Deep-checks accessibility, XML validity, response time, file size, encoding, and health score.

ZeroBreak

XML Sitemap Finder & Extractor API

andok/find-sitemap-from-url

Find and extract all XML sitemaps for any domain. Automatically parses robots.txt, scans HTML tags, and recursively follows indexes. Perfect for SEO & web scraping.

Andok

Robots.txt Auditor & Sitemap Finder

andok/robotstxt-auditor

Scan robots.txt files in bulk to extract sitemap URLs and verify crawler directives for technical SEO compliance.

Andok

Find Sitemap from url

eesti/find-sitemap-from-url

A powerful [Apify Actor] that finds sitemap URLs for any website. This Actor helps you discover XML sitemaps by checking common locations, robots.txt files, and analyzing HTML content for sitemap links.

ando

210

1.0

(1)

LLMs.txt Generator

onescales/the-llms-txt-generator

The most powerful LLMs.txt Generator tool online. Generates LLMs.txt , llms-full.txt and markdown .md files within seconds! Get your website discovered, and recommended by ChatGPT, Claude, Google Gemini, Perplexity, Grok, and every AI. (Great for AEO, AIO, GEO, AI SEO) Made by Hi LLMs

One Scales

106

5.0

(2)