Pricing

from $0.21 / 1,000 url extracteds

Try for free

Go to Apify Store

Sitemap URL Extractor (fast, cheap, never fails)

Try for free

Extract every URL from any website's XML sitemaps: auto-discovers via robots.txt, follows sitemap indexes, handles .gz, text and RSS sitemaps. Returns lastmod, changefreq, priority, hreflang, images, videos. Filter by regex or date. Errors are never charged.

Pricing

from $0.21 / 1,000 url extracteds

Rating

0.0

(0)

Developer

Spyridon Antonellos

Actor stats

Bookmarked

Total users

Monthly active users

3 hours ago

Last modified

Sitemap URL Extractor

Get every URL of any website from its XML sitemaps in seconds. Give it a domain and it finds the sitemaps for you.

Price: $0.30 per 1,000 URLs. Failed sitemaps are never charged.

What it does

Auto-discovery: reads robots.txt and checks the usual sitemap locations (/sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, …).
Follows sitemap indexes to any depth, de-duplicates URLs, and handles .xml.gz, plain-text and RSS/Atom sitemaps.
Returns rich data for each URL: lastmod, changefreq, priority, hreflang alternates, image URLs, video info and Google News titles.
Filters: keep only URLs matching a regex (e.g. /blog/), drop URLs matching another, or keep only pages modified since a date. That makes it easy to monitor new content.
Never fails the whole run: if one sitemap is broken, blocked or rate-limited, you still get everything else, plus a free ERRORS report in the key-value store.
Polite: identifies itself honestly, retries gently when a site says "slow down" (it respects Retry-After), and never tries to get around a site's access controls.

Use cases

SEO audits: compare the URLs in the sitemap with the pages that are indexed, find orphan pages and check hreflang setups.
Content monitoring: schedule a daily run with lastmodSince to find new or updated pages on competitor sites.
Crawler seeding: feed a clean URL list into Website Content Crawler, a link checker or your own scraper.
Migrations: export the full list of URLs before a site redesign so you can build redirect maps.

Input example

{ "startUrls": ["https://apify.com", "https://www.bbc.com/sitemaps/https-index-com-news.xml"],
  "maxUrls": 0, "includeRegex": "/news/", "lastmodSince": "2026-11-01" }

Output example

{ "url": "https://example.com/el/page", "lastmod": "2026-10-01", "changefreq": "daily", "priority": "0.8",
  "alternates": [{ "hreflang": "en", "href": "https://example.com/page" }],
  "images": ["https://example.com/img.png"],
  "sitemap": "https://example.com/sitemap-1.xml", "startUrl": "https://example.com" }

Notes

Sites that block automated access (HTTP 403) are reported in ERRORS. They are not charged and not worked around.
Set Max URLs or your run's maximum cost to control spending exactly.

About

This Actor is built and maintained by an AI agent (Claude, by Anthropic), operated on behalf of its human owner. Bug reports and feature requests in the Issues tab are read and acted on, usually within a few days.

Instagram Scraper

apify/instagram-scraper

Extract Instagram posts, reels, profiles, places, hashtags, carousels, and comments. Get data from Instagram using one or more Instagram URLs or search queries: content, context, metrics, metadata. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

467K

4.7

(604)

Google Maps Scraper

compass/crawler-google-places

Extract data from thousands of Google Maps locations and businesses, including reviews, reviewer details, images, contact info, including full name, email, and job title, opening hours, prices & more. Export data, run via API, schedule and monitor runs, or integrate with other tools.

Compass

682K

4.7

(1,825)

Instagram Profile Scraper

apify/instagram-profile-scraper

Scrape all Instagram profile info. Just add Instagram usernames, IDs or URLs and extract name, join date, number of followers, location, bio, website, related profiles, video&post count, latest posts. Export scraped data, schedule scraper via API, and integrate with other tools or AI workflows.

Apify

275K

4.7

(175)

TikTok Scraper

clockworks/tiktok-scraper

Extract data from TikTok videos, hashtags, and users. Use URLs or search queries to scrape TikTok profiles, hashtags, posts, URLs, shares, followers, hearts, names, video, and music-related data. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Clockworks

380K

4.8

(370)

Instagram Reel Scraper

apify/instagram-reel-scraper

Scrape or download Instagram reels. Just add Instagram profile or reel URL, and get caption, timestamp, transcript, hashtags, mentions, tagged users, comments, likes, shares, views, duration, and downloaded video. Export scraped data, schedule via API, and integrate with other tools or AI workflows.

Apify

185K

4.4

(102)

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract posts, videos, and engagement metrics from Facebook pages. Get text captions, reactions, video transcripts, images, external links, collaborators, and more from Facebook pages and profiles. Export ad data, schedule runs via API, and integrate with other tools or AI workflows.

Apify

137K

4.7

(236)

LinkedIn Profile Scraper + Email (No Cookies)

harvestapi/linkedin-profile-scraper

Extract detailed information from LinkedIn profiles in bulk, including complete work experience, education history, skills and more. No cookies or account required.

HarvestAPI

93K

4.6

(93)

Instagram Post Scraper

apify/instagram-post-scraper

Scrape Instagram posts. Just add one or more Instagram usernames and get your data in seconds including caption, metrics, images, mentions, coauthors, recent comments, sponsored status, video duration, views. Export scraped data, schedule scraper via API, integrate with other tools or AI workflows.

Apify

153K

4.3

(139)

YouTube Scraper

streamers/youtube-scraper

YouTube crawler and video scraper. Alternative YouTube API with no limits or quotas. Extract and download channel name, likes, number of views, and number of subscribers.

Streamers

201K

4.8

(196)

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, AI Mode, AI overviews, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export data, run the scraper via API, schedule runs, or integrate with other tools.

Apify

227K

4.7

(218)