Pricing

from $0.30 / 1,000 url extracteds

Try for free

Go to Apify Store

Sitemap URL Extractor & Checker

Try for free

Get every URL from a site's sitemaps: auto-finds them via robots.txt, walks sitemap indexes, reads .gz/text/RSS sitemaps, filters by pattern or lastmod, and can flag 404s and redirects. Pay per URL.

Pricing

from $0.30 / 1,000 url extracteds

Rating

0.0

(0)

Developer

Tidyfold Tools

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

Why this one

Finds sitemaps for you: reads Sitemap: lines in robots.txt, then tries the common paths (/sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, .gz, .txt)
Walks sitemap indexes of any depth, de-duplicates URLs, and keeps going when one child sitemap is broken (errors are listed in the run summary)
Reads every format: XML, gzipped .xml.gz, plain-text sitemaps, and RSS/Atom feeds
Filters before you pay: include/exclude regex patterns, "modified after" date, and a per-site cap, so you're only charged for the URLs you keep
Optional status check with HEAD and automatic GET fallback for servers that reject HEAD

Output

Field	Example
site	https://docs.apify.com
url	https://docs.apify.com/platform/actors
lastmod, changefreq, priority	2026-10-12, weekly, 0.7
images, hreflang	3, ["en","de"]
sitemap	The sitemap file the URL came from
httpStatus, finalUrl	200 / 301 / 404, and the URL after redirects (status check only)

A summary per site (how the sitemap was found, sitemaps read, URL count, errors) is saved in the run's key-value store.

Use cases

Feeding a clean URL list to a crawler or RAG pipeline, SEO audits (404s and redirects in the sitemap), monitoring competitors' new pages with "modified after", content inventories and site migrations.

Input example

{
  "startUrls": ["docs.apify.com", "https://example.com/sitemap_index.xml"],
  "includePatterns": ["/blog/"],
  "modifiedAfter": "2026-09-01",
  "checkStatus": false
}

Instagram Scraper

apify/instagram-scraper

Extract Instagram posts, reels, profiles, places, hashtags, carousels, and comments. Get data from Instagram using one or more Instagram URLs or search queries: content, context, metrics, metadata. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

449K

4.7

(603)

TikTok Scraper

clockworks/tiktok-scraper

Extract data from TikTok videos, hashtags, and users. Use URLs or search queries to scrape TikTok profiles, hashtags, posts, URLs, shares, followers, hearts, names, video, and music-related data. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Clockworks

360K

4.7

(370)

Instagram Profile Scraper

apify/instagram-profile-scraper

Scrape all Instagram profile info. Just add Instagram usernames, IDs or URLs and extract name, join date, number of followers, location, bio, website, related profiles, video&post count, latest posts. Export scraped data, schedule scraper via API, and integrate with other tools or AI workflows.

Apify

263K

4.7

(175)

Tweet Scraper V2 - X Scraper / Twitter Scraper / X.com Scraper

apidojo/tweet-scraper

Lightning-fast search, URL, list, and profile scraping, with customizable filters. At $0.40 per 1000 tweets, and 30-80 tweets per second, it is ideal for researchers, entrepreneurs, and businesses! Get comprehensive insights from Twitter (X) now!

API Dojo

118K

3.9

(196)

Google Maps Scraper

compass/crawler-google-places

Extract data from thousands of Google Maps locations and businesses, including reviews, reviewer details, images, contact info, including full name, email, and job title, opening hours, prices & more. Export data, run via API, schedule and monitor runs, or integrate with other tools.

Compass

666K

4.7

(1,818)

YouTube Scraper

streamers/youtube-scraper

YouTube crawler and video scraper. Alternative YouTube API with no limits or quotas. Extract and download channel name, likes, number of views, and number of subscribers.

Streamers

184K

4.8

(198)

Instagram Post Scraper

apify/instagram-post-scraper

Scrape Instagram posts. Just add one or more Instagram usernames and get your data in seconds including caption, metrics, images, mentions, coauthors, recent comments, sponsored status, video duration, views. Export scraped data, schedule scraper via API, integrate with other tools or AI workflows.

Apify

148K

4.3

(138)

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract posts, videos, and engagement metrics from Facebook pages. Get text captions, reactions, video transcripts, images, external links, collaborators, and more from Facebook pages and profiles. Export ad data, schedule runs via API, and integrate with other tools or AI workflows.

Apify

131K

4.7

(236)

Web Fetch

apify/web-fetch

Download any web page by URL and convert it to clean Markdown, plain text, HTML, and other formats. Automatically renders JavaScript and works around bot blocking. The Actor provides a real-time API for quick response times.

Apify

39K

5.0

(4)

Amazon Product Scraper

junglee/Amazon-crawler

Use this Amazon scraper to collect data based on URL and country from the Amazon website. Extract product information without using the Amazon API, including reviews, prices, descriptions, and Amazon Standard Identification Numbers (ASINs). Download data in various structured formats.