Vee Map

Every URL a website publishes in its sitemaps, from just a domain name.

Vee Map is a sitemap URL extractor. It is not a maps or Google Maps tool. Give it example.com and it locates the sitemaps, walks through all of them, and returns a de-duplicated list of URLs with lastmod, changefreq, priority and image counts.

Who it is for

• SEO teams running content inventories and audits
• Anyone checking what changed on a site (lastmod) or what a migration missed
• Developers feeding crawlers, search indexes and RAG pipelines with a clean URL list
• Teams monitoring competitor content

What you get: every sitemap URL with lastmod and priority (CSV, JSON, Excel)

One record per URL:

{
  "url": "https://apify.com/store",
  "lastmod": "2026-10-01",
  "changefreq": null,
  "priority": null,
  "imageCount": 0,
  "sitemapUrl": "https://apify.com/sitemap/pages.xml",
  "domain": "apify.com"
}

Fields the site does not publish are null. A SUMMARY record in the key-value store gives, for each site, the number of sitemaps fetched, URLs returned, errors, and sitemaps skipped because of robots.txt (blockedByRobots).

How it works

1. Reads robots.txt and collects its Sitemap: entries. If there are none, it tries /sitemap.xml and /sitemap_index.xml.
2. Follows sitemap indexes recursively. Handles plain XML, gzip (.xml.gz) and plain-text sitemaps.
3. Applies your regex filter, removes duplicates and returns the URLs.

It uses plain HTTP, with no browser and no proxies, and never fetches page content.

How it is different

• robots.txt is honoured. Disallow rules for a sitemap URL are applied, and skipped files are counted in SUMMARY. Crawl-delay is respected, up to 30 seconds. The Actor identifies itself as VeeMap. If robots.txt cannot be fetched because of a server or network error, the site is skipped and reported, because permission cannot be verified. A missing robots.txt (404) counts as allowed.
• No need to know where the sitemap is. A domain is enough. You can also pass a direct sitemap URL, and if it returns nothing the Actor falls back to robots.txt discovery.
• Stays on the target site. Sitemaps are followed only when they are on the same site as your start URL (same host, www. variant or subdomain). Entries pointing to other domains are ignored and logged.
• Safe regex filter. urlFilter accepts ordinary patterns such as /blog/ or (en|fr)/. Patterns that hang, like (a+)+$, are rejected with a clear message.
• Failures do not stop the run. A missing or broken sitemap is logged and skipped, and the other sites continue.
• Size-safe. Each file is capped at 50 MB on the wire and 50 MB after decompression, which is the sitemaps.org limit.
• Cost control. maxUrlsPerSite, maxUrls and maxSitemaps cap the run, and charging stops if you set a maximum cost.

If you get 0 URLs

The SUMMARY record and the run's status message explain why, for example: the site has no sitemap (robots.txt lists none and /sitemap.xml does not exist; pass a direct sitemap URL), the site blocks access (HTTP 403), robots.txt disallows the sitemap, or your urlFilter matched nothing. If robots.txt cannot be fetched (server error, timeout, DNS), the site is skipped and the reason says which. A page that appears in several sitemaps or several inputs is returned and billed once.

Input examples

One domain:

{ "startUrls": ["example.com"] }

Only blog URLs from two sites, at most 5,000 each:

{
  "startUrls": ["https://example.com", "https://example.org"],
  "urlFilter": "/blog/",
  "maxUrlsPerSite": 5000
}

A direct sitemap, with a contact in the User-Agent:

{
  "startUrls": ["https://www.example.com/sitemap.xml"],
  "contact": "https://example.com/contact"
}

Field	What it does
startUrls	Domains or direct sitemap URLs (max 100)
maxUrlsPerSite	Cap per site (default 10,000)
maxUrls	Cap for the whole run (default 100,000)
urlFilter	Optional regex a URL must match (max 200 characters; back-references such as \1 are not allowed)
maxSitemaps	Max sitemap files per site (default 500)
maxRuntimeSecs	Stop after this many seconds (default 3,600)
contact	Optional URL or email added to the User-Agent

What it costs

Pay per event. You pay only for results, and the price is the same on every Apify plan.

Event	Price
URL extracted	$0.40 per 1,000 URLs ($0.0004 each)
Run start	$0.005 per run

Platform usage (compute) is included in these prices, so there is no separate bill for it. The pricing table at the top of the Actor page is Apify's standard layout and lists every plan (Free, Starter, Scale, Business). Our price is the same in every column, so you never pay more or less because of your plan.

Examples, each for one run:

You get	Results	Run start	Total
1,000 URLs	$0.40	$0.405
10,000 URLs	$4.00	$4.005
100,000 URLs	$40.00	$40.005

Filters and caps (limits such as the per-run result caps and your input filters) only reduce the number of results you pay for. Set a maximum cost on the run in Apify if you want a hard spending limit.

FAQ

Is this for Google Maps? No. It extracts URLs from website sitemaps.

What if the site has no sitemap? The site is reported in SUMMARY with zero URLs. Vee Map does not crawl pages to invent one.

Does it fetch the pages themselves? No. It reads only the sitemap and robots files that sites publish for crawlers.

Can it read compressed sitemaps? Yes, .xml.gz and gzip-encoded responses.

Does it follow sitemap indexes? Yes, recursively, up to maxSitemaps files per site.

Why is lastmod empty? The site does not publish it in that sitemap.

Why was a site skipped? Check SUMMARY: robots.txt could not be fetched, the start URL is invalid or not allowed, or robots.txt disallowed the sitemap.

Are URLs de-duplicated? Yes, within each site.

Limits

• Up to 100 start URLs. maxUrls and maxUrlsPerSite up to 1,000,000. maxSitemaps up to 1,000.
• Only public http(s) hosts on ports 80 and 443. Private, loopback, link-local and cloud-metadata addresses are blocked, and redirects must stay on the same site.
• Fetches time out after 30 seconds, with a 250 ms delay between fetches.
• URLs longer than 2,048 characters are dropped, and only http(s) URLs are returned.
• Treat output URLs as untrusted data. lastmod is limited to 64 characters (a leading =, @ or +/- that is not followed by a digit gets an apostrophe so spreadsheets cannot run it as a formula), and changefreq is one of the standard values (always, hourly, daily, weekly, monthly, yearly, never) or null.
• Notes in SUMMARY are cut to 200 characters and contain no control characters.

Responsible use, legal and data protection

This Actor retrieves publicly available data only. It is not affiliated with, endorsed by or sponsored by the websites you scan. All trademarks belong to their owners and are used only to describe compatibility.

Vee provides tools only; you are fully responsible for how you use them and the results. The tool is provided as is, without warranty, and Vee gives no legal advice. You are fully responsible for your use of the data: comply with the terms of service, robots.txt, copyright and database rights of the sites you target, and with all applicable law, including data protection laws (GDPR, UK GDPR and similar). Output URLs contain personal data whenever a site puts it in a URL, for example profile slugs, user names or email addresses in URLs from forums or social sites. Have a lawful basis before you store or reuse any personal data, minimise it and delete it when no longer needed, and do not use it for unsolicited contact. Do not republish copyrighted text or images without permission. Vee does not collect or keep your inputs or results; they stay in your Apify account. Vee reserves the right to change or withdraw the Actor if a site owner objects. To raise a concern, open an issue on this Actor's Issues tab on Apify.

Support

Questions or a feature request? Open an issue on this Actor's Issues tab on Apify and we will reply there.