Pricing

from $0.30 / 1,000 url extracteds

Try for free

Go to Apify Store

Sitemap URL Extractor & Checker

Try for free

Get every URL from a site's sitemaps: auto-finds them via robots.txt, walks sitemap indexes, reads .gz/text/RSS sitemaps, filters by pattern or lastmod, and can flag 404s and redirects. Pay per URL.

Pricing

from $0.30 / 1,000 url extracteds

Rating

0.0

(0)

Developer

Tidyfold Tools

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

Categories

SEO tools

Developer tools

Websites or sitemap URLs

startUrls

Required

A domain (we find its sitemaps via robots.txt and common paths) or a direct sitemap / sitemap index URL. XML, .xml.gz, plain-text and RSS/Atom sitemaps all work.

Type:array

Max URLs per site

maxUrlsPerSite

Optional

Stop after this many URLs per site. 0 = no limit.

Type:integer

Minimum:0

Default:0

Only URLs matching

includePatterns

Optional

Keep only URLs matching at least one of these regular expressions, e.g. /blog/ or /products/.

Type:array

Skip URLs matching

excludePatterns

Optional

Drop URLs matching any of these regular expressions, e.g. /tag/ or ?page=.

Type:array

Modified on or after (YYYY-MM-DD)

modifiedAfter

Optional

Keep only URLs whose sitemap lastmod is on or after this date. URLs without lastmod are dropped when this is set.

Type:string

Check each page's HTTP status

checkStatus

Optional

Send a HEAD request (GET fallback) to every URL and add httpStatus and finalUrl, to find 404s, redirects and errors listed in your sitemap. Priced per checked URL instead of per extracted URL.

Type:boolean

Default:false

Same domain only

sameDomainOnly

Optional

When the sitemap was found automatically, skip URLs on other domains.

Type:boolean

Default:true

Instagram Scraper

apify/instagram-scraper

Extract Instagram posts, reels, profiles, places, hashtags, carousels, and comments. Get data from Instagram using one or more Instagram URLs or search queries: content, context, metrics, metadata. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

449K

4.7

(603)

TikTok Scraper

clockworks/tiktok-scraper

Extract data from TikTok videos, hashtags, and users. Use URLs or search queries to scrape TikTok profiles, hashtags, posts, URLs, shares, followers, hearts, names, video, and music-related data. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Clockworks

360K

4.7

(370)

Instagram Profile Scraper

apify/instagram-profile-scraper

Scrape all Instagram profile info. Just add Instagram usernames, IDs or URLs and extract name, join date, number of followers, location, bio, website, related profiles, video&post count, latest posts. Export scraped data, schedule scraper via API, and integrate with other tools or AI workflows.

Apify

263K

4.7

(175)

Tweet Scraper V2 - X Scraper / Twitter Scraper / X.com Scraper

apidojo/tweet-scraper

Lightning-fast search, URL, list, and profile scraping, with customizable filters. At $0.40 per 1000 tweets, and 30-80 tweets per second, it is ideal for researchers, entrepreneurs, and businesses! Get comprehensive insights from Twitter (X) now!

API Dojo

118K

3.9

(196)

Google Maps Scraper

compass/crawler-google-places

Extract data from thousands of Google Maps locations and businesses, including reviews, reviewer details, images, contact info, including full name, email, and job title, opening hours, prices & more. Export data, run via API, schedule and monitor runs, or integrate with other tools.

Compass

666K

4.7

(1,818)

YouTube Scraper

streamers/youtube-scraper

YouTube crawler and video scraper. Alternative YouTube API with no limits or quotas. Extract and download channel name, likes, number of views, and number of subscribers.

Streamers

184K

4.8

(198)

Instagram Post Scraper

apify/instagram-post-scraper

Scrape Instagram posts. Just add one or more Instagram usernames and get your data in seconds including caption, metrics, images, mentions, coauthors, recent comments, sponsored status, video duration, views. Export scraped data, schedule scraper via API, integrate with other tools or AI workflows.

Apify

148K

4.3

(138)

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract posts, videos, and engagement metrics from Facebook pages. Get text captions, reactions, video transcripts, images, external links, collaborators, and more from Facebook pages and profiles. Export ad data, schedule runs via API, and integrate with other tools or AI workflows.

Apify

131K

4.7

(236)

Web Fetch

apify/web-fetch

Download any web page by URL and convert it to clean Markdown, plain text, HTML, and other formats. Automatically renders JavaScript and works around bot blocking. The Actor provides a real-time API for quick response times.

Apify

39K

5.0

(4)

Amazon Product Scraper

junglee/Amazon-crawler

Use this Amazon scraper to collect data based on URL and country from the Amazon website. Extract product information without using the Amazon API, including reviews, prices, descriptions, and Amazon Standard Identification Numbers (ASINs). Download data in various structured formats.