Website Sitemap URL Extractor — XML Sitemap & Sitemap Index URLs (No Website API Key, No Website Login)

Extract a clean, deduplicated list of public website URLs from an XML sitemap or a bounded sitemap index. Website Sitemap URL Extractor turns published sitemap entries into structured Apify dataset rows, preserving available last-modified timestamps, change-frequency hints, priority values, the originating sitemap, and index depth. Use it for sitemap URL extraction, SEO inventories, migration planning, recurring URL comparisons, and preparing a URL list for a separate content-processing workflow.

Start with the small, real-world example below to inspect your first results. Eligible Apify account credits may cover a trial; credit availability and runtime depend on your account and the target website. There is no guaranteed free allowance or completion time.

No website API key, website account, login cookies, or browser session is required for a publicly accessible sitemap. An Apify account is required to run the Actor, and an Apify API token is required for authenticated API and MCP access. These are platform credentials, not credentials for the target website.

Table of contents

• What can this Actor do?
• Who is this for?
• What data can I extract?
• How to extract sitemap URLs
• Input example and field reference
• Real output example
• Coverage, accuracy, and honest limits
• Run outcomes and troubleshooting
• Pricing and spending controls
• Tips and practical workflows
• Want other website or platform data?
• MCP for AI agents
• Integrations and API
• Support
• Frequently asked questions
• Related searches
• Disclaimer
• Resources
• 中文说明

What can this Actor do?

• ✅ Read an explicitly supplied, public HTTP or HTTPS XML sitemap URL.
• ✅ Extract page URLs from a sitemap URL set, rather than downloading each page.
• ✅ Follow child sitemap references from a sitemap index within your configured sitemap-file budget.
• ✅ Remove duplicate page URLs within the extraction result so that repeated entries do not become separate result rows.
• ✅ Preserve lastModified, changeFrequency, and priority when that information is available in the sitemap.
• ✅ Keep sourceSitemap and depth alongside each extracted URL for provenance and interpretation.
• ✅ Stop within explicit URL-count, sitemap-count, and extraction-time limits.
• ✅ Write flat, structured records to an Apify dataset for JSON, CSV, Excel, and other supported dataset exports.
• ✅ Fit into scheduled jobs, webhook-driven processing, authenticated API calls, and Apify MCP workflows.

It is a sitemap extractor, not a full website crawler. It does not fetch page bodies, extract article text, render JavaScript pages, find every link on a website, or verify whether each returned URL is currently a live page. If you need those operations, treat the extracted dataset as input to a separate tool, with its own permissions, cost, and output contract.

Who is this for?

SEO specialists and technical auditors: assemble a publisher-declared URL inventory with optional update metadata. Use it as one input to an audit, not as proof that all URLs are indexed by a search engine or that all website pages appear in the sitemap.

Website owners and migration teams: export the sitemap inventory before a migration, compare it with a later run, and build your own URL-mapping spreadsheet. A sitemap is a useful starting point for redirects, but you still need a separate validation step for final migration acceptance.

Analysts and researchers: collect published page addresses from public websites in a consistent tabular format. Keep the sitemap source so that later analysis can distinguish changes to published inventories from changes to page content.

Developers and data engineers: build a bounded URL-discovery stage for a larger pipeline. The result limit and flat schema are suitable for passing records into your own queue, database, or downstream processor without claiming this Actor performs those later steps.

AI-agent builders: let an agent request an authorized, public sitemap inventory through Apify, then reason over the returned URLs. The agent should not assume it has read the contents of those URLs.

Content operations and commerce teams: inventory URLs published in a product, category, help-center, documentation, or editorial sitemap. The Actor returns the URLs and sitemap metadata, not product prices, article bodies, stock levels, or page titles.

What data can I extract?

Each dataset item represents one unique public page URL stored in that run. The dataset view is overview, titled Sitemap URLs. Its six columns match the fields below, so important values are available as ordinary table columns rather than hidden in a nested object.

Core fields in returned URL records

Field	Type	Description
url	string	Extracted public HTTP/HTTPS page URL from a sitemap entry. This is the page address, not the sitemap address. It is not proof that the page was fetched, exists now, is canonical, or is indexed.
sourceSitemap	string	Sitemap XML file that contained the page entry. For an index-based extraction, this can be a child sitemap rather than the input index.
depth	integer	Sitemap traversal depth associated with the record. A direct URL-set input produces depth 0, as in the real example. Interpret this as sitemap traversal information, not website navigation depth.

Metadata populated when available

Field	Type	Description
lastModified	string | null	Last-modified value supplied in the sitemap, when available. Preserve the source value and timezone; do not assume the Actor independently checked when the page changed.
changeFrequency	string | null	Publisher-provided change-frequency hint, when available. It is neither a measured update frequency nor a promise that the page changes on that schedule.
priority	number | null	Numeric priority supplied by the sitemap, when available. It is a publisher hint, not a search ranking, importance score assigned by this Actor, or traffic estimate.

Missing optional metadata is legitimate. In the real smoke example, changeFrequency is null on every returned record, and the home-page record also has lastModified: null. Do not replace these values with guessed timestamps, empty text labeled as a real measurement, or the date of extraction.

Dataset links and diagnostic records

The Actor's output schema exposes results, which links to the overview dataset view, and fullJson, which links to the complete dataset items. These are links to records, not extra per-page data fields.

Successful nonempty runs also write a STATS record in the default key-value store. Its top-level shape includes itemCount, stats, and errorCount; the details inside stats can depend on the extraction outcome. Do not design a customer integration around undocumented nested statistic fields.

For an input-error outcome, the OUTPUT key-value-store record provides a reason and a correction hint. This diagnostic record is distinct from the Actor's generated output links and from the dataset itself. It is not a billable URL record. A successful extraction need not create an OUTPUT error record; the supplied positive smoke capture recorded that diagnostic record as missing.

How to extract sitemap URLs

1. Open Website Sitemap URL Extractor in Apify and use the input form.
2. Paste the full public sitemap URL into sitemapUrl. Supply the XML sitemap or XML sitemap-index address, not merely a homepage or an HTML sitemap page.
3. Set a small maxUrls for the first run. For a direct sitemap test, the real example uses 3 URLs and 1 sitemap file.
4. Choose maxSitemaps according to whether you are reading a direct URL set or an index containing multiple files.
5. Set timeoutSeconds within the supported range. This is the extraction budget, not a speed guarantee.
6. Run the Actor and wait for a terminal run status. An API response indicating that a run started is not evidence that extraction has finished.
7. Open the dataset, inspect the Sitemap URLs view, and export the format your workflow needs.
8. If there are no rows, inspect the run status, status message, and OUTPUT diagnostic record when present before deciding whether to retry.

The website controls its sitemap contents and accessibility. The same input can produce different URLs later because the sitemap was reordered, updated, removed, or blocked. Save the input and run date alongside an export if reproducibility matters.

Input example and field reference

Exact input used for the real smoke run

The following input is the recorded input for run F0qbJJrNZBEO2I57e, not a fabricated demonstration. It targets one direct GOV.UK sitemap and asks for a small sample:

{
  "sitemapUrl": "https://www.gov.uk/sitemaps/sitemap_1.xml",
  "maxUrls": 3,
  "maxSitemaps": 1,
  "timeoutSeconds": 90
}

Input fields

Field	Type	Required	Default	Description
sitemapUrl	string	Yes	None; form prefill is https://www.gov.uk/sitemap.xml	Absolute public HTTP/HTTPS address of an XML sitemap or sitemap index. A prefill is a convenience example, not a default target for an omitted required field.
maxUrls	integer	No	100	Maximum unique page URL rows to return; supported range 1–10,000. Also bounds the number of billable result rows for this run. It does not guarantee that many URLs exist or will be returned.
maxSitemaps	integer	No	10	Maximum sitemap files to fetch, including the starting sitemap and child files; supported range 1–30. It counts files, not page URLs.
timeoutSeconds	integer	No	120	Global budget for fetching and parsing sitemaps; supported range 15–240 seconds. It is not a per-file timeout and does not promise complete coverage before expiry.

Use JSON numbers for the integer settings: 3, not "3". Supply one sitemapUrl string per run; this schema does not advertise an array of sitemap URLs, search keywords, a domain-discovery mode, or a page-body extraction switch.

Choosing a direct sitemap versus an index

A URL-set sitemap contains page entries directly. It is a good way to inspect a specific section with maxSitemaps: 1. The real example uses this arrangement, so its records have depth: 0.

A sitemap index points to other sitemap files. Set enough maxSitemaps to allow the root index and the relevant children, while keeping maxUrls aligned with your intended result volume. The maximum file budget is still 30; an index can contain more references than this Actor will visit in one run.

The form's GOV.UK index prefill and the direct sitemap in the smoke example are different inputs. Do not interpret the example as proof that every GOV.UK sitemap, every nested-index layout, or an entire domain was exhaustively processed.

URL safety and input boundaries

Use a publicly reachable sitemap on a standard HTTP/HTTPS endpoint. The documented input excludes localhost, private-network addresses, URLs containing embedded credentials, and nonstandard-port targets. Public redirects are checked before following them. These safety boundaries are intentional; the Actor is not a tool for reaching intranets, local services, authenticated admin endpoints, or private network resources.

Do not put website passwords, cookies, authorization headers, private access tokens, or an Apify API token into sitemapUrl. Authenticate API requests to Apify with the platform token in the request header, as shown later, not by adding it to the target website URL.

Real output example

The supplied smoke run F0qbJJrNZBEO2I57e completed on 2026-10-09 with status SUCCEEDED and status message Extracted 3 URLs. It used build 0.1.2 and returned the following three dataset rows. The source values and nulls are preserved exactly:

[
  {
    "url": "https://www.gov.uk/",
    "depth": 0,
    "priority": 0.5,
    "lastModified": null,
    "sourceSitemap": "https://www.gov.uk/sitemaps/sitemap_1.xml",
    "changeFrequency": null
  },
  {
    "url": "https://www.gov.uk/government/statistics/economic-estimates-employment-in-dcms-sectors-july-2023-to-june-2024/economic-estimates-employment-in-the-dcms-sectors-july-2023-to-june-2024-technical-and-quality-assurance-report",
    "depth": 0,
    "priority": 0.375,
    "lastModified": "2026-09-30T14:17:21+01:00",
    "sourceSitemap": "https://www.gov.uk/sitemaps/sitemap_1.xml",
    "changeFrequency": null
  },
  {
    "url": "https://www.gov.uk/government/statistics/economic-estimates-employment-in-dcms-sectors-july-2023-to-june-2024/pre-release-access-list-for-the-economic-estimates-employment-in-dcms-sectors-july-2023-to-june-2024",
    "depth": 0,
    "priority": 0.375,
    "lastModified": "2026-09-30T14:17:21+01:00",
    "sourceSitemap": "https://www.gov.uk/sitemaps/sitemap_1.xml",
    "changeFrequency": null
  }
]

What this verifies: a real direct-sitemap extraction produced three flat URL records with source provenance, numeric priorities, depth values, and nullable metadata. What it does not verify: full-site coverage, page-body extraction, page availability, nested-index behavior at every supported limit, sustained throughput, or settled result-event billing. The captured run billing counter was 0; that capture is not used to claim that positive-result runs are free or that billing reconciliation has been verified.

The example uses a public government website as a test target. It does not imply endorsement, partnership, ownership of that site's content, or a promise that this specific file remains available indefinitely.

Snapshot and freshness caveat: these are the captured run's rows, not a claim that they match a later download of the sitemap. An independent check of current GOV.UK XML from a different network location did not reproduce the same URL membership. CDN/cache differences or sitemap updates are possible explanations, but the root cause is not confirmed. This example and related GOV.UK checks must not be treated as independently verified current-XML membership or a guaranteed fresh, single-version sitemap snapshot. Record the run date and preserve source observations separately when freshness matters.

Coverage, accuracy, and honest limits

A sitemap inventory is not a full website inventory

The Actor can only extract eligible entries from the supplied sitemap and the sitemap files reached within your bounds. Pages omitted by the website publisher are not discovered automatically. Orphan pages, search pages, links visible only after login, and URLs found only in HTML are outside the sitemap inventory unless the publisher explicitly includes them.

Limits are ceilings, not throughput promises

The input allows at most 10,000 unique URLs, 30 sitemap files, and 240 seconds of extraction time. Defaults are 100 URLs, 10 sitemap files, and 120 seconds. These bounds protect scope and let you start small; they do not establish a tested processing rate, unlimited recursion, exhaustive domain coverage, or guaranteed completion for any particular website.

A run can return fewer records than maxUrls because there are fewer usable entries, duplicates are removed, the sitemap-file budget restricts discovery, or the time budget prevents visiting more files. A nonempty result is not, by itself, proof that all referenced sitemaps were processed. Review diagnostics when completeness matters.

Metadata is publisher-supplied

lastModified may be missing, old, timezone-specific, or inaccurate. changeFrequency and priority are optional hints. The Actor does not infer these values from page content, compare page revisions, or turn priority into a Google ranking signal. Your downstream analysis should keep nulls and distinguish supplied metadata from verified observations.

Deduplication has a defined purpose, not unlimited normalization

Repeated page URLs do not become repeated dataset rows within the extraction. Do not assume the Actor merges every semantically equivalent URL: HTTP versus HTTPS, tracking parameters, aliases, redirects, translated pages, and alternate canonical forms may represent different addresses. Cross-run deduplication belongs in your own comparison logic.

Formats and features not promised here

The supported input contract is a public XML sitemap or sitemap index. This README does not promise HTML sitemap scraping, plain-text URL-list parsing, arbitrary file uploads, gzip compatibility for every target, image/video/news extension extraction, alternate-language mapping, robots.txt discovery, browser rendering, sitemap generation, or authentication to a private sitemap. There are no output fields for page title, meta description, status code, article body, product price, or canonical tags.

Access problems remain visible

A public URL may still be blocked, rate-limited, redirected unexpectedly, or temporarily unavailable. This Actor does not advertise a paid proxy, selectable proxy region, website login, CAPTCHA solving, or bypass of a website access restriction. An upstream access, transport, or service problem is a failure to investigate, not a reason to invent an empty successful inventory.

Run outcomes and troubleshooting

Input errors finish SUCCEEDED with 0 dataset rows and 0 charge; see the OUTPUT record for the reason. The zero-charge statement here refers to Actor result-event charges. Apify platform usage can be accounted separately under your account's billing model.

Outcome	Run status	Dataset and result-event behavior	What to do
Valid input with extracted URLs	SUCCEEDED	Unique extracted URLs are stored; each stored row is the defined result event.	Read/export the dataset and inspect the provenance fields.
Recognized invalid or unsupported customer input	SUCCEEDED	0 URL rows and 0 result-event charges; OUTPUT explains the problem.	Correct the URL, input types, or configured bounds before retrying.
Explicitly recognized no-results or verified-not-found outcome	SUCCEEDED	0 URL rows and 0 result-event charges, with an explanatory diagnostic.	Check the target and its current sitemap contents.
Recognized free-plan limit outcome, if applicable	SUCCEEDED	0 URL rows and 0 result-event charges, with a limit explanation.	Follow the displayed reason; no specific numeric daily allowance is promised here.
Upstream block, access error, network failure, timeout, or unexpected internal error	FAILED	Failed zero-row runs do not charge result events.	Review the run message, confirm target access, and report persistent failures.
Unexplained empty extraction without a recognized benign reason	FAILED	No URL rows; no result-event charge for that empty failure.	Treat it as an unresolved issue, not proof that the website has no pages.

A SUCCEEDED status means the Actor finished normally, including handling a customer correction without a service failure. It does not necessarily mean URLs were returned. Check the dataset count and the explanatory message. Conversely, HTTP 403, 429, transport failures, and timeouts must not be interpreted as harmless input errors merely because the dataset is empty.

If you are using the API, invalid request JSON or a request rejected by Apify before a run starts is a platform/API validation response, not a completed Actor run. The SUCCEEDED input-error contract describes requests handled inside an actual Actor run.

For a practical triage sequence: verify the exact input, confirm that the address is XML rather than HTML, check the configured limits, inspect the terminal status, then inspect OUTPUT when present. For a persistent upstream failure, avoid immediately scheduling aggressive retries against the same website; first determine whether access is available and permitted.

Pricing and spending controls

Pricing is PAY_PER_EVENT, presented to users as pay per stored sitemap URL. The primary event is Sitemap URL extracted (apify-default-dataset-item). One unique public page URL written to the dataset is one defined result event. The current actor pricing snapshot contains these exact USD prices:

Apify pricing tier	USD per stored URL	USD per 1,000 stored URLs
FREE	$0.00010	$0.10
BRONZE	$0.00009	$0.09
SILVER	$0.00008	$0.08
GOLD	$0.00008	$0.08
PLATINUM	$0.00008	$0.08
DIAMOND	$0.00008	$0.08

The paid tiers shown above provide a lower per-result rate than FREE; SILVER through DIAMOND have the same price in this snapshot. FREE is the tier name, not a promise that the Actor price is zero. Apify subscription fees, platform usage, storage, transfers, or other account-level billing are not represented by this result-price table. Confirm your account's billing terms and the current Store pricing before running a production workload.

The result-event formula is stored unique URL rows × your applicable per-row price. For example, at the GOLD price, 100 stored rows correspond to $0.008 in result-event charges; 10,000 stored rows correspond to $0.80. At FREE, 100 stored rows correspond to $0.01. These are arithmetic examples from the configured prices, not claims about the smoke run's settled bill or an all-inclusive account cost.

Failed and zero-row runs do not charge this result event. Recognized customer input errors end SUCCEEDED with no URL rows and no result-event charge. Duplicate URLs removed before storage are not additional URL rows. Optional metadata does not create separate billable records.

To keep scope predictable, set maxUrls to the number of results you actually need and use Apify's platform spending controls where available. Start with a small sample before requesting a larger inventory. A result limit is not a guarantee that all files will be visited, and an application should not assume that repeated scheduled runs share a deduplication ledger or are free after the first extraction.

Tips and practical workflows

1. Start with the specific sitemap you need

If a site publishes separate sitemaps for documentation, products, categories, or articles, use the appropriate direct file when you only need that section. This makes your intent clearer than starting with a large index and assuming the desired section will fit within a small file budget. You must obtain the sitemap address yourself; the Actor does not advertise domain-to-sitemap discovery.

2. Preserve provenance in your spreadsheet

Keep sourceSitemap next to url, even if you hide it in a presentation. When an entry disappears between exports, provenance helps you ask whether a section changed, a child sitemap moved, or the extraction bounds changed. Keep the extraction date in your own run history; do not overwrite publisher-provided lastModified with that date.

3. Compare sets, not only counts

For scheduled inventories, compare URL sets between runs and separate added, removed, and unchanged URLs in your own workflow. The same count can hide substantial changes. Conversely, a count difference may come from changed maxUrls, maxSitemaps, target XML, or a file-access problem rather than an editorial change.

4. Treat a bounded sample as a bounded sample

The real example deliberately requests only three URLs. It is useful for field inspection and integration testing, not for a whole-site SEO report. Record the chosen bounds with your export so that another analyst does not mistake a sample for a complete domain inventory.

5. Validate pages separately when that matters

For redirect testing, link-health checking, content indexing, or migration verification, feed the exported URLs into a separately authorized processor. That processor must handle its own HTTP checks, website policies, request pacing, and cost. This Actor's url column is an address, not an HTTP-success certificate.

6. Avoid repeated empty retries

A recognized input error needs a correction, not a larger result limit. A verified no-results outcome needs target review. An upstream failure needs access or service investigation. Distinguishing these cases prevents an automated schedule from repeating a malformed target or turning an access problem into misleading empty exports.

7. Be explicit about export types

Use JSON when null values and number types matter. In a spreadsheet, make sure long URLs are kept intact and that timestamps are not silently converted to a different timezone. The flat schema supports simple tabular exports, but spreadsheet applications may apply their own cell formatting.

Want other website or platform data?

For a sitemap inventory, use Website Sitemap URL Extractor. For page content or a different platform, review the available tools on the myagizm Actor profile and choose an Actor whose own input, output, pricing, and access requirements match the job. Do not infer capabilities from a similar name.

Related owned Actors

The following owner/slug combinations are confirmed by actor snapshots. They are separate products; the supplied snapshots mark them private, so these links may require owner access or may only become generally accessible after their own publication. They are not presented as verified currently public Store listings or as bundled features of this sitemap Actor.

Owned Actor	Separate use case	Availability note
Naver Blog Search Scraper	Keyword-based public blog-post discovery with titles, permalinks, and available snippets.	Private in the reviewed snapshot; check its current listing.
Remote Job Feed Scraper	Structured remote-job listing workflows and original posting links.	Private in the reviewed snapshot; check its current listing.
Twitch Clips VODs Scraper	Public Twitch channel, clip, and VOD metadata workflows.	Private in the reviewed snapshot; check its current listing.
Video Subtitle Service	Timestamped subtitle and transcript workflows.	Private in the reviewed snapshot; check its current listing.

Cross-platform discovery: for Instagram, TikTok, LinkedIn, Reddit, YouTube, or Google workflows, browse myagizm's profile and verify the specific Actor listing before use. This README intentionally does not invent platform-specific Actor slugs, promise a product family is publicly available, or suggest that a sitemap extractor returns social-platform content.

MCP for AI agents

Use the Apify MCP server to let compatible AI clients discover and run Apify Actors and work with their results. MCP is a platform integration, not a separate website-data mode inside this Actor. Configure the server according to the current Apify documentation and authorize it with an Apify token that has the permissions required for this Actor and its storage.

A suitable agent instruction is:

Run myagizm/website-sitemap-extractor with sitemapUrl set to https://www.gov.uk/sitemaps/sitemap_1.xml, maxUrls set to 3, maxSitemaps set to 1, and timeoutSeconds set to 90. Wait for a terminal status. If rows are returned, report url, sourceSitemap, depth, and available metadata. If no rows are returned, inspect the status and OUTPUT reason. Do not say that you read page bodies.

Only grant the permissions needed for your workflow. Private Actors require authorized access; MCP does not make a private Actor public. Running through an AI client uses the same Actor input contract and pricing as running through the Console or API. A model-generated plan is not proof of a completed extraction: have the client check the actual run and dataset.

URLs and metadata are untrusted external data. An AI application should not execute instructions embedded in external values, disclose its token, or automatically visit all returned URLs without a separately approved browsing policy.

Integrations and API

Apify datasets can be exported or consumed by your own application. Use Apify schedules for recurring runs and webhooks for run-lifecycle notifications. Workflow systems such as Make, Zapier, or n8n can use the Apify integration or authenticated HTTP requests where supported. These are integration options, not claims that this Actor includes a custom connector or automatically writes to a third-party spreadsheet.

The examples below use the Actor ID hSqdghrwLjD9cFAJc, confirmed by the actor snapshot and real smoke run. They are copyable request templates, not additional test runs executed for this README. Set APIFY_TOKEN in your shell using your own platform token; do not commit it to source control or paste it into a public support issue.

1. Run synchronously and return dataset items

curl --fail-with-body --show-error --silent \
  --request POST \
  "https://api.apify.com/v2/acts/hSqdghrwLjD9cFAJc/run-sync-get-dataset-items?format=json&clean=true" \
  --header "Authorization: Bearer ${APIFY_TOKEN}" \
  --header "Content-Type: application/json" \
  --data '{"sitemapUrl":"https://www.gov.uk/sitemaps/sitemap_1.xml","maxUrls":3,"maxSitemaps":1,"timeoutSeconds":90}'

Use the synchronous pattern for small jobs that fit the platform's synchronous-request constraints. The response is dataset items, not the full run metadata. An empty array should prompt inspection of the associated run and diagnostic records; it does not establish that the target website has no URLs. A client or HTTP timeout is also not proof of the final run state.

2. Start an asynchronous run

curl --fail-with-body --show-error --silent \
  --request POST \
  "https://api.apify.com/v2/acts/hSqdghrwLjD9cFAJc/runs" \
  --header "Authorization: Bearer ${APIFY_TOKEN}" \
  --header "Content-Type: application/json" \
  --data '{"sitemapUrl":"https://www.gov.uk/sitemaps/sitemap_1.xml","maxUrls":3,"maxSitemaps":1,"timeoutSeconds":90}'

Read data.id from the start response, then poll the run through the documented GET /v2/actor-runs/{runId} endpoint until it reaches a terminal state. Do not consume a partial dataset as a final answer while the run is still READY or RUNNING. On success, use that run's defaultDatasetId. On a zero-row customer outcome, inspect OUTPUT in that run's default key-value store when available.

3. Fetch the completed run's dataset

Set DATASET_ID to the defaultDatasetId returned for your own run, not to a hard-coded demonstration dataset:

curl --fail-with-body --show-error --silent \
  --request GET \
  "https://api.apify.com/v2/datasets/${DATASET_ID}/items?format=json&clean=true&view=overview" \
  --header "Authorization: Bearer ${APIFY_TOKEN}"

Choose the format and pagination parameters documented by Apify for your export. For large responses, page through the dataset rather than assuming one client response contains everything. If your application needs all raw fields, read the dataset without a view transformation. Never publish token-bearing or signed private dataset links in documentation; sharing an export should follow your account's access settings and data-handling policy.

Support

Use the Actor's Issues tab on its Apify listing for reproducible problems and feature requests. Include the run ID, public sitemap URL if it can safely be shared, the non-secret input settings, terminal status, expected behavior, and actual row count. For a missing optional field, explain whether the source sitemap includes it; a null value alone is not evidence of a bug.

Do not share Apify tokens, website passwords, cookies, signed private dataset URLs, private customer exports, or confidential URL inventories. If the target itself is sensitive, describe the issue without exposing it publicly and follow the available support channel's privacy options. Account billing, access permissions, and platform incidents may require Apify support rather than an Actor code change.

Suggestions for new output fields or additional sitemap formats are welcome, but they are requests, not promised existing capabilities. This README documents the current contract and supplied real example, not a future feature roadmap.

Frequently asked questions

1. Do I need a website API key, login, or cookies?

No, not for a publicly accessible sitemap. The Actor does not ask for target-website authentication. You still need an Apify account to run it, and an Apify token for authenticated API or MCP access. A private sitemap cannot be made public by omitting credentials.

2. Can I paste just a domain or homepage?

Supply the absolute XML sitemap or sitemap-index URL. A homepage, a search query, or a bare domain is not a sitemap-discovery request. If the homepage returns HTML, it is not the documented XML input format.

3. What is the difference between a sitemap and a sitemap index?

A URL-set sitemap lists pages directly. A sitemap index lists sitemap-file references. Index extraction may need additional file budget to reach page entries; it is still bounded by maxSitemaps, maxUrls, and the global time budget.

4. Does it crawl the pages or scrape their text?

No. It extracts published sitemap URLs and available sitemap metadata. Article text, page titles, prices, canonical tags, and page HTTP status codes require a separate tool and are not output fields here.

5. Will it find every page on a website?

No. It sees entries published in the visited sitemap files, and configured bounds may limit which files or entries are reached. A site can publish an incomplete or outdated sitemap. This is not exhaustive HTML-link crawling or a search-engine index report.

6. How many URLs can I request?

maxUrls supports 1–10,000, with default 100. This is an upper bound on returned unique URL rows, not a guaranteed yield or throughput claim. The actual result can be smaller.

7. How many sitemap files can it read?

maxSitemaps supports 1–30, with default 10, and includes the starting sitemap. A direct URL-set sample can use 1; an index needs room for child files if it is to yield page entries.

8. How does the time setting work?

timeoutSeconds supports 15–240, with default 120. It is the global fetching/parsing budget, not a per-page budget. Platform run timeouts and synchronous HTTP client limits are separate settings and should not be confused with it.

9. Are duplicate URLs charged more than once?

Repeated URLs removed within the extraction do not create additional stored result rows. Do not assume semantic merging of all URL variants or deduplication across separate runs. Repeating a run can produce a new billable dataset inventory.

10. Why are some metadata fields null?

The sitemap may not provide them. The Actor does not invent missing lastModified, changeFrequency, or priority values. The real sample includes legitimate nulls, which should remain distinguishable from verified measurements.

11. Does priority tell me the page's Google ranking?

No. It is a sitemap-provided numeric hint, not a search ranking or an Actor-calculated SEO score. Similarly, lastModified does not prove a page was independently checked for content changes.

12. What does depth mean?

It describes sitemap traversal depth for the extracted record. In the real direct-sitemap example it is 0. It is not the number of clicks from the website homepage and does not measure HTML information architecture.

13. Can I use localhost, private IPs, or authenticated URLs?

No. The documented target is public HTTP/HTTPS. Local/private destinations, credential-bearing URLs, and nonstandard-port targets are excluded. The Actor is not intended for intranet extraction or passing website secrets in a URL.

14. Can I configure a proxy or bypass a blocked website?

No selectable proxy or login input is documented. The Actor does not advertise paid proxy use or access-control bypass. If a target blocks access or returns a rate limit, investigate the upstream failure rather than treating it as a successful empty inventory.

15. Why can SUCCEEDED still have zero results?

Recognized customer input errors and explicit benign no-result outcomes are handled normally. They finish SUCCEEDED with zero URL rows and zero result-event charge; inspect OUTPUT for the reason. The terminal status alone is not a result-count guarantee.

16. Which problems produce FAILED?

Upstream blocks, access/transport failures, timeouts, unexpected service errors, and unexplained empty extractions remain FAILED. This distinction avoids hiding an operational issue as a harmless customer correction. Failed zero-row runs do not charge the URL result event.

17. Is the FREE tier free to run?

FREE is the account-tier label. Its configured URL result-event price is $0.00010, while BRONZE is $0.00009 and SILVER through DIAMOND are $0.00008. Eligible credits and account/platform billing terms determine your final account cost.

18. Can I export to JSON, CSV, or Excel and schedule runs?

Yes, use the Apify dataset export and schedule features. The Actor's records are flat for tabular use. Scheduling creates separate runs; it does not automatically calculate a cross-run URL diff, share deduplication, or guarantee unchanged sitemap content.

19. Can an AI agent run it through MCP?

Yes, through Apify's MCP integration with an appropriately authorized token. MCP does not bypass Actor visibility, permissions, input validation, or pricing. Tell the agent to verify terminal status and rows, and not to claim it has read the page content.

20. Is extracting a public sitemap automatically lawful?

Public availability does not remove website terms, privacy obligations, intellectual-property rights, or local law. Use only data you are authorized to access and process, and assess GDPR/CCPA or other applicable requirements for your own use case. This tool does not provide legal advice or permission on the website owner's behalf.

Related searches

XML sitemap URL extractor · website sitemap scraper · sitemap index URL extraction · public sitemap to JSON · sitemap to CSV export · sitemap to Excel · deduplicated sitemap URLs · sitemap lastmod extraction · sitemap priority metadata · sitemap changefreq extraction · sitemap source provenance · sitemap traversal depth · bounded sitemap index reader · SEO sitemap inventory · technical SEO URL list · website migration URL inventory · product sitemap URL extraction · documentation sitemap inventory · public government sitemap URLs · scheduled sitemap export · sitemap URL comparison workflow · Apify sitemap Actor · sitemap extraction API · AI agent sitemap inventory · Apify MCP sitemap tool · no website login sitemap extractor · public XML sitemap parser · sitemap dataset export

Disclaimer

This is an independent sitemap extraction tool. It is not affiliated with or endorsed by GOV.UK, any target website, search engine, or trademark owner. Website names and trademarks belong to their respective owners; examples identify public test targets rather than commercial partners.

Use only public data you are authorized to access and process. Respect applicable law, website terms, access restrictions, intellectual-property rights, and privacy obligations, including GDPR and CCPA where relevant. A URL can contain personal or sensitive information even if it is publicly listed; minimize collection, control export access, and choose appropriate retention for your workflow.

Results reflect the published sitemap data accessible during a bounded run. Completeness, metadata accuracy, page availability, search-engine indexing, runtime, and downstream business outcomes are not guaranteed. This README is technical product documentation, not legal advice.

Resources

• Website Sitemap URL Extractor
• myagizm Actor profile
• Apify API documentation
• Apify Actors documentation
• Apify datasets and export
• Apify schedules
• Apify integrations
• Apify webhooks
• Apify MCP server

---

中文说明

网站 Sitemap URL 提取器 — XML 网站地图与索引 URL 导出（无需网站 API Key，无需网站登录）

从公开 XML sitemap 或有范围限制的 sitemap 索引中提取网页 URL，并将去重后的结果保存为结构化 Apify 数据集。网站 Sitemap URL 提取器会保留网站地图中可用的最后修改时间、更新频率提示、优先级、来源 sitemap 与索引层级，适合技术 SEO URL 清单、网站迁移准备、定期网址比较，以及为后续独立内容处理流程准备 URL 输入。

建议先使用下方真实小样本检查结果。符合条件的 Apify 账户额度可能覆盖试用费用；是否有额度、运行多久，取决于账户和目标网站。本说明不承诺固定免费配额，也不承诺固定完成时间。

读取公开 sitemap 不需要目标网站的 API Key、账号、登录 Cookie 或浏览器会话。运行 Actor 仍需要 Apify 账户；使用受保护的 API 或 MCP 功能仍需要 Apify API token。 平台令牌和目标网站登录凭据是两回事，请勿混淆。

中文目录

• 本 Actor 能做什么
• 适合哪些用户
• 可以提取哪些数据
• 如何提取网站地图 URL
• 输入示例与字段说明
• 真实输出示例
• 覆盖范围准确性与限制
• 运行状态与故障排查
• 价格与费用控制
• 使用建议与实际工作流
• 需要其他网站或平台数据
• 通过 MCP 接入 AI 智能体
• 集成与 API 调用
• 支持与问题反馈
• 常见问题
• 相关搜索
• 免责声明
• 参考资源

本 Actor 能做什么

• ✅ 读取您明确提供的、公开可访问的 HTTP/HTTPS XML sitemap 地址。
• ✅ 从 URL 列表型 sitemap 中提取页面网址，而不是逐个下载网页。
• ✅ 在设置的文件数量范围内跟随 sitemap 索引中的子文件引用。
• ✅ 对提取结果中的重复页面 URL 去重，避免重复条目变成额外数据行。
• ✅ 保留网站地图提供的 lastModified、changeFrequency、priority。
• ✅ 为每条结果记录 sourceSitemap 与 depth，方便追溯和解释。
• ✅ 使用 URL 数量、sitemap 文件数量和统一时间预算约束提取范围。
• ✅ 将扁平记录保存到 Apify 数据集，供 JSON、CSV、Excel 等格式导出。
• ✅ 接入定时任务、webhook、受保护的 API 以及 Apify MCP 工作流。

这是 sitemap 提取器，不是完整网站爬虫。 它不抓取页面正文、不提取文章内容、不渲染 JavaScript 页面、不遍历网站所有 HTML 链接，也不逐个确认返回的 URL 当前是否可访问。如果需要这些操作，请将 URL 数据集交给另一个独立工具，并单独评估访问权限、费用与输出字段。

适合哪些用户

SEO 专员与技术审计人员： 将网站发布者声明的 URL 整理为清单，保留可用的更新时间信息。清单可以作为审计输入，但不能证明这些页面已被搜索引擎收录，也不能证明网站所有页面都在 sitemap 中。

网站负责人和迁移团队： 在迁移前导出网址清单，与迁移后的结果比较，再自行建立 URL 映射或重定向表。网站地图适合作为起点，但迁移验收仍需要独立检查页面状态和跳转行为。

分析师和研究人员： 将公开网站发布的页面地址保存为统一表格。保留来源 sitemap，可帮助区分“网站发布的网址清单发生变化”和“网页正文发生变化”，两者并不相同。

开发者与数据工程师： 在更大的数据流程中建立有范围限制的 URL 发现步骤。扁平结构方便把结果交给自有队列、数据库或后续处理器，但这些后续处理不属于本 Actor 的功能。

AI 智能体开发者： 让智能体通过 Apify 获取已授权访问的公开 sitemap 清单，再对地址和元数据进行分析。获得 URL 不等于已经读取了 URL 对应的页面内容。

内容运营和电商团队： 整理产品、分类、帮助中心、文档或文章 sitemap 中发布的网址。本 Actor 不返回商品价格、库存、文章正文或页面标题；这些内容需要其他工具。

可以提取哪些数据

每条数据集记录对应本次运行中保存的一个唯一公开页面 URL。数据集视图名称为 overview，显示标题为 Sitemap URLs。下列六个字段都以普通表格列呈现，不需要展开嵌套对象才能读取。

URL 记录中的核心字段

字段	类型	说明
url	string	从 sitemap 条目提取的公开 HTTP/HTTPS 页面地址。它不是 sitemap 文件地址，也不能证明页面已被访问、当前存在、属于规范地址或已被收录。
sourceSitemap	string	实际包含该页面条目的 XML sitemap 文件。读取索引时，这可能是子文件，而不是输入的根索引。
depth	integer	与该记录关联的 sitemap 遍历层级。直接读取 URL 列表型文件时为 0，真实示例即如此。它不是网页距首页的点击层级。

仅在来源提供时填充的元数据

字段	类型	说明
lastModified	string | null	sitemap 声明的最后修改时间。请保留原始值及其时区；Actor 没有独立验证页面实际何时发生修改。
changeFrequency	string | null	发布者提供的更新频率提示。它不是实测更新频率，也不是页面会按该周期更新的保证。
priority	number | null	sitemap 提供的数值优先级。它不是搜索排名、Actor 计算的重要性评分或访问量估算。

可选字段为空是正常情况。真实示例中，三条记录的 changeFrequency 都是 null，首页记录的 lastModified 也是 null。不要把这些空值替换成猜测日期，也不要把本次提取日期冒充网页修改日期。

数据集链接与诊断记录

Actor 的输出结构提供 results 链接，指向 overview 数据集视图，以及 fullJson 链接，指向完整数据记录。这些是结果访问入口，不是额外页面字段。

非空成功运行还会在默认键值存储中写入 STATS，其顶层结构包含 itemCount、stats 和 errorCount。内部统计细节可能因运行结果而异，不建议围绕未公开约定的嵌套统计字段建立生产集成。

发生输入错误时，默认键值存储的 OUTPUT 记录会解释原因并给出修改建议。它与 Actor 自动生成的数据集链接、数据集 URL 记录不是同一个对象，也不是计费结果行。正常成功提取不一定生成错误诊断 OUTPUT；提供的正向 smoke 记录中，该诊断记录不存在。

如何提取网站地图 URL

1. 在 Apify 打开网站 Sitemap URL 提取器，进入输入表单。
2. 在 sitemapUrl 中填写完整的公开 XML sitemap 地址或 XML sitemap 索引地址。不要只填首页，也不要将 HTML 网站地图当作 XML 输入。
3. 第一次运行使用较小的 maxUrls。下方真实示例请求 3 个 URL 和 1 个 sitemap 文件。
4. 根据读取直接文件还是索引，设置合适的 maxSitemaps。
5. 在支持范围内设置 timeoutSeconds。这是提取时间预算，不是速度承诺。
6. 启动运行并等待终态。API 表示“运行已创建”并不代表提取已完成。
7. 打开数据集，查看 Sitemap URLs 表格，再导出所需格式。
8. 没有结果时，先检查运行状态、状态说明，以及存在时的 OUTPUT 诊断，再决定是否重试。

网站自己决定 sitemap 的内容和可访问性。同样的输入在未来可能返回不同网址，因为文件被更新、重新排序、删除或限制访问。需要可复现分析时，请在自己的记录中保存输入参数、运行日期和导出文件。

输入示例与字段说明

真实 smoke 运行使用的输入

以下 JSON 来自运行 F0qbJJrNZBEO2I57e 的实际输入记录，并非虚构演示。它直接读取一个 GOV.UK sitemap，只请求少量结果：

{
  "sitemapUrl": "https://www.gov.uk/sitemaps/sitemap_1.xml",
  "maxUrls": 3,
  "maxSitemaps": 1,
  "timeoutSeconds": 90
}

输入字段

字段	类型	必填	默认值	说明
sitemapUrl	string	是	无；表单预填为 https://www.gov.uk/sitemap.xml	公开 XML sitemap 或 sitemap 索引的绝对 HTTP/HTTPS 地址。预填示例不是省略必填字段后的默认目标。
maxUrls	integer	否	100	最多返回的唯一页面 URL 行数，支持 1–10,000，也限制本次最多产生的 URL 结果事件数量。不保证来源有这么多有效条目。
maxSitemaps	integer	否	10	最多读取的 sitemap 文件数，包括起始文件和子文件，支持 1–30。它计数的是文件，不是页面 URL。
timeoutSeconds	integer	否	120	读取和解析 sitemap 的统一时间预算，支持 15–240 秒。不是每个文件各自获得的预算，也不保证到期前完整处理所有引用。

整数设置请使用 JSON 数字，如 3，不要传字符串 "3"。每次运行提供一个 sitemapUrl 字符串；当前输入结构不提供多 sitemap 数组、搜索关键词、域名自动发现或正文抓取开关。

直接文件与索引的选择

URL 列表型 sitemap直接包含页面条目。只需要检查某个具体部分时，可以使用直接文件和 maxSitemaps: 1。真实示例采用该方式，因此返回记录的 depth 为 0。

sitemap 索引包含其他 sitemap 文件的引用。若希望读取子文件，maxSitemaps 必须给根索引和相关子文件留出空间，同时按业务需要设置 maxUrls。文件数量上限仍为 30，索引中列出的引用可能远多于单次运行能处理的数量。

表单预填的 GOV.UK 根索引与 smoke 示例中的直接文件是不同输入。该示例不能证明已遍历所有 GOV.UK sitemap，也不能证明每种多层索引形式或整个域名都已完整测试。

URL 安全边界

请使用通过标准 HTTP/HTTPS 端点公开访问的 sitemap。输入说明明确排除 localhost、私有网络地址、包含用户名或密码的 URL，以及非标准端口目标。跟随公开重定向前会检查目标。这些边界用于限制访问范围，本 Actor 不用于访问内网、管理后台、本地服务或私有网络资源。

不要在 sitemapUrl 中放入网站密码、Cookie、授权头、私有访问令牌或 Apify token。访问 Apify API 的平台令牌应放在请求头中，不能拼到目标网站 URL 上。

真实输出示例

提供的 smoke 运行 F0qbJJrNZBEO2I57e 于 2026-10-09 完成，状态为 SUCCEEDED，状态说明为 Extracted 3 URLs.，使用构建 0.1.2。以下为实际返回的三条记录，来源值和空值均原样保留：

[
  {
    "url": "https://www.gov.uk/",
    "depth": 0,
    "priority": 0.5,
    "lastModified": null,
    "sourceSitemap": "https://www.gov.uk/sitemaps/sitemap_1.xml",
    "changeFrequency": null
  },
  {
    "url": "https://www.gov.uk/government/statistics/economic-estimates-employment-in-dcms-sectors-july-2023-to-june-2024/economic-estimates-employment-in-the-dcms-sectors-july-2023-to-june-2024-technical-and-quality-assurance-report",
    "depth": 0,
    "priority": 0.375,
    "lastModified": "2026-09-30T14:17:21+01:00",
    "sourceSitemap": "https://www.gov.uk/sitemaps/sitemap_1.xml",
    "changeFrequency": null
  },
  {
    "url": "https://www.gov.uk/government/statistics/economic-estimates-employment-in-dcms-sectors-july-2023-to-june-2024/pre-release-access-list-for-the-economic-estimates-employment-in-dcms-sectors-july-2023-to-june-2024",
    "depth": 0,
    "priority": 0.375,
    "lastModified": "2026-09-30T14:17:21+01:00",
    "sourceSitemap": "https://www.gov.uk/sitemaps/sitemap_1.xml",
    "changeFrequency": null
  }
]

该示例验证了什么： 一次真实直接 sitemap 提取返回了三条扁平 URL 记录，其中有来源、数值优先级、层级以及允许为空的元数据。没有验证什么： 全站覆盖、正文提取、页面当前可访问性、所有边界下的索引行为、持续吞吐量或已结算的结果事件费用。捕获的运行计费计数为 0；不能据此宣称有结果的运行免费，也不能据此宣称完成了计费对账。

使用公开政府网站作为测试目标不代表背书、合作、内容所有权，也不保证该具体文件会永久保留。

快照与新鲜度说明： 以上是该次运行保存的记录，不代表它们与稍后下载的 sitemap 完全一致。从另一网络位置独立读取当前 GOV.UK XML 时，未复现相同的 URL 集合。CDN 或缓存差异、网站地图更新都是可能原因，但根因尚未确认。不能把本示例或相关 GOV.UK 检查描述为已独立验证与当前 XML 一致，也不能保证结果是实时、单一版本的网站地图快照。对新鲜度有要求时，请保存运行日期，并将独立来源观察另行留存。

覆盖范围准确性与限制

网站地图清单不等于完整网站清单

Actor 只能从输入 sitemap 及限定范围内访问到的子文件中提取符合条件的条目。发布者没有列入 sitemap 的页面不会被自动发现。孤立页面、仅登录后可见的链接、搜索结果页，以及只存在于 HTML 链接中的页面，都不能因此被认为已纳入清单。

数量上限不是速度保证

输入支持的上限为 10,000 个唯一 URL、30 个 sitemap 文件、240 秒提取预算；默认分别为 100 个 URL、10 个文件、120 秒。这些参数约束任务范围，但不构成实测处理速率、无限递归能力、全域名完整覆盖或任何网站的完成时间保证。

实际结果可能少于 maxUrls：可用条目本来就少，重复项被去除，文件预算限制发现范围，或者时间预算不足以继续读取。返回了一些数据，也不能单凭这一点断言所有引用文件已处理完。对完整性有要求时，应同时查看诊断信息并进行独立验证。

元数据来自发布者

lastModified 可能缺失、过期、带特定时区或不准确。changeFrequency 和 priority 都是可选提示。Actor 不通过正文推断缺失值、不比较页面版本，也不将优先级转换成 Google 排名信号。分析时请区分“来源声明”和“独立验证”。

去重不等于所有 URL 形式的语义合并

提取结果中的重复 URL 不会变成重复数据行，但不能假定 HTTP 与 HTTPS、带参数与不带参数、别名、重定向地址、不同语言版本或其他规范地址形式一定被合并。跨运行去重需要您自己的逻辑；本次唯一并不意味着下一次运行不会再次返回。

本文没有承诺的格式与功能

当前输入约定是公开 XML sitemap 或 sitemap 索引。本文不承诺 HTML 网站地图解析、纯文本 URL 文件、任意上传文件、所有目标的 gzip 兼容性、图片/视频/新闻扩展提取、多语言 alternate 映射、robots.txt 自动发现、浏览器渲染、网站地图生成或私有 sitemap 登录。

当前输出没有页面标题、meta description、HTTP 状态码、文章正文、商品价格或 canonical 标签字段。需要这些数据时，应使用独立工具，而不是从 sitemap 元数据推断。

上游访问问题会保留为失败

公开 URL 仍可能被屏蔽、限流、意外跳转或暂时不可用。本 Actor 不宣传付费代理、可选代理地区、网站登录、验证码破解或访问限制绕过。访问或传输问题需要排查，不能为了得到“成功状态”而伪造空清单。

运行状态与故障排查

输入错误会以 SUCCEEDED 状态结束，产生 0 条数据记录且 0 费用；原因请查看 OUTPUT。 此处的零费用指 Actor 的结果事件费用。Apify 平台使用费用可能按照账户计费规则单独核算。

情况	运行状态	数据与结果事件	建议操作
有效输入并提取到 URL	SUCCEEDED	保存唯一 URL；每条已保存 URL 是约定的结果事件。	读取或导出数据集，检查来源字段。
已识别的客户输入错误或不支持的输入	SUCCEEDED	0 行、0 结果事件费用，OUTPUT 说明原因。	修正 URL、类型或参数范围后重试。
明确识别的无结果或经验证的不存在情况	SUCCEEDED	0 行、0 结果事件费用，并给出诊断。	检查目标及其当前 sitemap 内容。
已识别的免费计划限制，若适用	SUCCEEDED	0 行、0 结果事件费用，并说明限制原因。	按提示处理；本文不承诺具体每日数量。
上游屏蔽、访问错误、网络故障、超时或未预期错误	FAILED	失败的零行运行不收取 URL 结果事件费用。	查看运行说明，确认访问情况，反馈持续故障。
没有明确正常原因的空提取	FAILED	没有 URL 行，不产生该空失败的结果事件费用。	视为待排查问题，不能当作网站没有页面的证据。

SUCCEEDED 表示 Actor 正常结束，也包括以正常方式处理客户需要修改的输入，并不一定表示返回了 URL。请同时检查数据行数和解释信息。HTTP 403、429、传输故障和超时也不能因为数据集为空就被当作无害输入错误。

API 请求 JSON 格式错误，或请求在运行创建前被 Apify 拒绝，属于平台/API 校验响应，不是一次已完成的 Actor 运行。这里的输入错误成功结束约定，针对已进入 Actor 并由其处理的请求。

建议排查顺序：确认完整输入；确认地址返回 XML 而非 HTML；检查参数上限；检查运行终态；再读取存在时的 OUTPUT。持续上游失败时，不要直接对同一网站安排密集重试，先确认访问是否可用且获准。

价格与费用控制

计费模型为 PAY_PER_EVENT，对用户表现为按已保存的 sitemap URL 计费。主要事件为 Sitemap URL extracted（apify-default-dataset-item）。一个写入数据集的唯一公开页面 URL 对应一个约定结果事件。当前 Actor 价格快照中的美元价格如下：

Apify 价格层级	每条已保存 URL，美元	每 1,000 条已保存 URL，美元
FREE	$0.00010	$0.10
BRONZE	$0.00009	$0.09
SILVER	$0.00008	$0.08
GOLD	$0.00008	$0.08
PLATINUM	$0.00008	$0.08
DIAMOND	$0.00008	$0.08

以上付费层级的单位结果价格低于 FREE；在当前快照中，SILVER 到 DIAMOND 的价格相同。FREE 是账户层级名称，不代表该 Actor 单价为零。 订阅费用、平台使用、存储、传输及其他账户费用不包含在该结果价格表中。生产运行前，请查看当前商店价格和您的账户计费规则。

结果事件费用公式为：已保存的唯一 URL 行数 × 适用单价。按 GOLD 价格，100 行对应 $0.008 结果事件费用，10,000 行对应 $0.80；按 FREE 价格，100 行对应 $0.01。这些是依据配置单价计算的示例，不是 smoke 运行已结算账单，也不是含全部平台费用的总价。

失败和零行运行不产生该结果事件费用。已识别的输入错误正常结束为 SUCCEEDED，不返回 URL，也不收取结果事件费用。保存前去除的重复地址不会增加结果行数；可选元数据不会单独产生额外结果记录。

请将 maxUrls 设为实际需要的数量，并在适用时使用 Apify 平台费用上限控制。先做小样本，再扩大任务。结果上限不能保证所有文件均被访问；定时重复运行也不意味着跨运行共享去重账本或首次之后自动免费。

使用建议与实际工作流

1. 优先使用需要的具体 sitemap

网站若按文档、商品、分类或文章分开发布 sitemap，只需要其中一部分时可以直接读取相关文件。比起从巨大根索引开始、却只给很小文件预算，这更容易表达目标范围。具体 sitemap 地址需要您自行获取；当前功能不宣传域名到 sitemap 的自动发现。

2. 保留来源信息

即使展示表格时隐藏 sourceSitemap，也建议在原始导出中保留它。某个 URL 消失时，来源可以帮助判断是否是栏目变化、子文件迁移或提取边界改变。请在自己的运行记录中保存提取时间，不要覆盖发布者提供的 lastModified。

3. 比较 URL 集合，不只比较数量

定时清单可以在您自己的流程中比较新增、移除和未变化的地址。相同总数可能掩盖大量内容替换；数量变化也可能只是参数、目标文件或访问情况改变，不一定表示网站内容减少。

4. 明确标记小样本

真实示例只请求三条 URL，适合检查字段和测试集成，不适合作为全站 SEO 报告。请把输入上限与导出一起保存，避免后续分析人员把样本误认为完整域名清单。

5. 页面检查交给独立流程

如需重定向测试、链接健康检查、正文索引或迁移验收，请将导出 URL 交给另一个经过授权的处理器。它要自行遵守网站政策、请求节奏和费用约束。本 Actor 的 url 列只提供地址，不提供 HTTP 成功证明。

6. 区分错误后再重试

输入错误需要修正输入，不是把上限调大；明确无结果需要检查目标；上游失败需要调查访问或服务问题。区分这些情况可以避免定时任务反复提交错误地址，或把访问故障保存为误导性的空导出。

7. 注意导出格式的类型处理

需要保留空值和数字类型时优先使用 JSON。用电子表格时，确认长 URL 没有被截断，时间字段没有被软件悄悄转换为另一时区。扁平字段方便导出，但表格软件仍可能自动应用自己的格式。

需要其他网站或平台数据

需要 sitemap 清单时使用网站 Sitemap URL 提取器。需要正文或其他平台数据时，请浏览 myagizm 的 Actor 主页，并按各工具自身的输入、输出、价格与访问要求选择，不要仅凭相似名称推断能力。

已确认归属的相关 Actor

下列 owner/slug 组合已由 Actor 快照确认。它们都是独立产品；提供的快照中均为私有状态，链接可能需要所有者权限，或需等待其独立发布后才可公开访问。本表不将它们描述为已经确认公开的商店产品，也不表示这些功能包含在 sitemap Actor 中。

所属 Actor	独立用途	可用性说明
Naver Blog Search Scraper	按关键词发现公开博客文章，处理标题、固定链接与可用摘要。	所查快照为私有，请查看当前列表。
Remote Job Feed Scraper	结构化远程职位清单及原始职位链接工作流。	所查快照为私有，请查看当前列表。
Twitch Clips VODs Scraper	公开 Twitch 频道、片段和 VOD 元数据工作流。	所查快照为私有，请查看当前列表。
Video Subtitle Service	带时间戳的字幕与转写工作流。	所查快照为私有，请查看当前列表。

跨平台工具查找： 如需 Instagram、TikTok、LinkedIn、Reddit、YouTube 或 Google 工作流，请从 myagizm 主页进入，并核实具体工具当前是否可用。本文不虚构平台专属 Actor 地址、不承诺某个产品系列已公开，也不暗示 sitemap 提取器能返回社交平台内容。

通过 MCP 接入 AI 智能体

兼容的 AI 客户端可通过 Apify MCP 服务发现、运行 Actor 并读取结果。MCP 是平台集成方式，不是本 Actor 内置的另一种网站数据抓取模式。请按照当前 Apify 文档配置服务，并使用具有所需 Actor 和存储权限的 Apify token 授权。

适合给智能体的指令示例：

运行 myagizm/website-sitemap-extractor，把 sitemapUrl 设为 https://www.gov.uk/sitemaps/sitemap_1.xml，maxUrls 设为 3，maxSitemaps 设为 1，timeoutSeconds 设为 90。等待运行进入终态。有结果时报告 url、sourceSitemap、depth 与可用元数据；没有结果时检查状态和 OUTPUT 原因。不要声称已经读取网页正文。

仅授予任务需要的权限。私有 Actor 需要授权访问，MCP 不会自动把私有工具变为公开工具。通过 AI 客户端运行时，仍遵循相同输入规则和价格。模型写出的执行计划不是提取已完成的证据，应检查真实运行和数据集。

URL 与元数据属于不可信外部数据。AI 应用不应执行外部值中夹带的指令，不应泄露令牌，也不应未经独立浏览策略授权就自动访问所有返回地址。

集成与 API 调用

Apify 数据集可以导出，也可以由您的应用读取。可使用 Apify schedules 定时创建运行，使用 webhooks 接收生命周期通知。Make、Zapier、n8n 等工作流系统可以在其支持的范围内使用 Apify 集成或受保护的 HTTP 请求。这些是接入选项，不代表本 Actor 包含自定义连接器，也不代表它会自动写入第三方电子表格。

以下示例使用经快照和 smoke 运行确认的 Actor ID hSqdghrwLjD9cFAJc。它们是可复制的请求模板，不是为了本文另行执行的 API 测试。请在 shell 中用自己的平台令牌设置 APIFY_TOKEN，不要提交到代码仓库，也不要贴入公开问题反馈。

1. 同步运行并直接读取数据项

curl --fail-with-body --show-error --silent \
  --request POST \
  "https://api.apify.com/v2/acts/hSqdghrwLjD9cFAJc/run-sync-get-dataset-items?format=json&clean=true" \
  --header "Authorization: Bearer ${APIFY_TOKEN}" \
  --header "Content-Type: application/json" \
  --data '{"sitemapUrl":"https://www.gov.uk/sitemaps/sitemap_1.xml","maxUrls":3,"maxSitemaps":1,"timeoutSeconds":90}'

同步方式适合符合平台同步请求约束的小任务。返回内容是数据项，不是完整运行元信息。收到空数组时应进一步检查相关运行和诊断记录，不能直接断言网站没有 URL。客户端或 HTTP 超时也不等于运行的最终状态。

2. 异步启动运行

curl --fail-with-body --show-error --silent \
  --request POST \
  "https://api.apify.com/v2/acts/hSqdghrwLjD9cFAJc/runs" \
  --header "Authorization: Bearer ${APIFY_TOKEN}" \
  --header "Content-Type: application/json" \
  --data '{"sitemapUrl":"https://www.gov.uk/sitemaps/sitemap_1.xml","maxUrls":3,"maxSitemaps":1,"timeoutSeconds":90}'

从启动响应读取 data.id，再按照文档通过 GET /v2/actor-runs/{runId} 查询，直到进入终态。运行仍为 READY 或 RUNNING 时，不要把部分数据集当作最终结果。成功后使用该运行的 defaultDatasetId；零行客户输入结果应检查默认键值存储中存在时的 OUTPUT。

3. 获取已完成运行的数据集

将 DATASET_ID 设置为您自己的运行返回的 defaultDatasetId，不要硬编码演示数据集：

curl --fail-with-body --show-error --silent \
  --request GET \
  "https://api.apify.com/v2/datasets/${DATASET_ID}/items?format=json&clean=true&view=overview" \
  --header "Authorization: Bearer ${APIFY_TOKEN}"

需要其他导出格式或分页时，请使用 Apify 文档规定的参数。较大数据集应分页读取，不能假定单次客户端响应包含全部数据。如需完整原始字段，可不使用视图转换。不要公开带令牌或签名的私有数据集链接；分享导出应遵循账户访问设置和数据处理政策。

支持与问题反馈

可在 Actor 的 Apify 页面 Issues 标签页反馈可复现问题或提出功能建议。请提供运行 ID、可以安全公开的 sitemap URL、非敏感输入参数、终态、预期行为与实际记录数量。若问题是可选字段缺失，请说明来源 XML 是否确实包含该字段；只有 null 不足以证明故障。

请勿分享 Apify token、网站密码、Cookie、带签名的私有数据集链接、客户私有导出或保密 URL 清单。目标敏感时，不要将地址直接公开；应按支持渠道提供的隐私选项处理。账户计费、权限和平台故障可能需要 Apify 支持，而不是修改 Actor。

欢迎建议新增字段和 sitemap 格式，但建议不代表当前已经支持，也不构成未来交付承诺。本说明记录的是当前约定和提供的真实示例。

常见问题

1. 需要网站 API Key、登录或 Cookie 吗？

读取公开 sitemap 不需要目标网站认证。但运行需要 Apify 账户，受保护的 API 和 MCP 需要 Apify token。不提交网站凭据不能让私有 sitemap 自动变成公开文件。

2. 可以只输入域名或首页吗？

请填写完整 XML sitemap 或索引地址。首页、关键词和裸域名不是 sitemap 自动发现请求。如果目标返回 HTML，它不属于本文约定的 XML 输入。

3. sitemap 与 sitemap 索引有什么不同？

普通 URL 列表型文件直接包含页面；索引包含其他 sitemap 引用。索引通常需要更多文件预算才能读取页面条目，仍受到文件数、URL 数和全局时间预算限制。

4. 会抓取网页正文吗？

不会。Actor 提取已发布的 sitemap URL 与可用元数据。正文、标题、价格、canonical 标签和页面 HTTP 状态码均需独立工具，当前不是输出字段。

5. 能找到网站每一个页面吗？

不能。它只读取访问到的 sitemap 条目，而网站地图本身可能不完整或过期，运行边界也可能截断范围。它不是完整 HTML 链接爬取，也不是搜索引擎收录报告。

6. 最多可以请求多少 URL？

maxUrls 支持 1–10,000，默认 100。这是返回唯一 URL 行数的上限，不是保证结果数量或处理速率，实际返回可能更少。

7. 最多可以读取多少 sitemap 文件？

maxSitemaps 支持 1–30，默认 10，包含起始文件。直接读取普通文件可以设为 1；索引若要读取子文件，需为子文件留出预算。

8. 时间参数如何工作？

timeoutSeconds 支持 15–240，默认 120，是统一读取和解析预算，不是每页预算。平台运行超时与同步 HTTP 客户端限制是另外的设置，不要混为一谈。

9. 重复 URL 会重复计费吗？

本次提取中保存前被去除的重复 URL 不会新增结果行。但不能假定所有语义相近的 URL 形式均被合并，也不能假定跨运行共享去重；再次运行可能重新产生计费清单。

10. 为什么元数据有空值？

来源 sitemap 可能没有提供相应信息。Actor 不会猜测缺失的修改时间、更新频率或优先级。真实样本也包含正常空值，应保留它与已验证数据之间的区别。

11. priority 表示 Google 排名吗？

不是。它是 sitemap 发布者的数值提示，不是搜索排名或 Actor 计算的 SEO 评分。lastModified 同样不能证明已独立检查正文变化。

12. depth 是什么意思？

它描述提取记录对应的 sitemap 遍历层级。真实直接文件示例为 0。它不表示从网站首页点击多少次，也不是 HTML 信息架构深度。

13. 可以访问 localhost、私有 IP 或带账号密码的 URL 吗？

不可以。输入约定为公开 HTTP/HTTPS。排除本地和私有目标、带凭据的地址以及非标准端口。本 Actor 不用于内网提取，也不应通过 URL 传递网站秘密。

14. 可以设置代理或绕过屏蔽吗？

当前没有公开约定的代理选择或登录输入，也不宣传付费代理和访问控制绕过。目标屏蔽或限流时应排查上游访问问题，不能把它当作正常空清单。

15. 为什么 SUCCEEDED 也可能没有结果？

已识别的客户输入错误或明确正常无结果情况会正常结束。它们返回零 URL、零结果事件费用，并在 OUTPUT 中说明原因。终态并不保证记录数量。

16. 哪些情况会 FAILED？

上游屏蔽、访问和传输失败、超时、未预期服务错误及没有明确正常原因的空提取保留为 FAILED。这样可以避免把运行问题隐藏成客户输入修正。失败的零行运行不收取 URL 结果事件费用。

17. FREE 计划是否完全免费？

FREE 是账户层级名称，配置的单条 URL 结果事件价格为 $0.00010；BRONZE 为 $0.00009，SILVER 至 DIAMOND 为 $0.00008。账户额度与平台计费规则决定最终费用。

18. 可以导出 JSON、CSV、Excel，并定时运行吗？

可以使用 Apify 数据集导出与定时功能。字段是扁平结构，适合表格。定时任务会创建独立运行，不会自动计算跨运行差异、共享去重，也不能保证网站地图内容保持不变。

19. AI 智能体可以通过 MCP 使用吗？

可以，通过 Apify MCP 并使用适当授权的令牌。MCP 不绕过可见性、权限、校验或价格。请要求智能体核实终态和真实记录，不要声称已读取页面正文。

20. 公开 sitemap 是否意味着任何使用都合法？

公开可访问不免除网站条款、隐私义务、知识产权和当地法律要求。仅访问和处理您获准使用的数据，按具体用途评估 GDPR、CCPA 等要求。本工具不提供法律意见，也不能代网站所有者授予权限。

相关搜索

XML 网站地图 URL 提取 · sitemap 索引解析 · 网站 sitemap 抓取 · 公开 sitemap 转 JSON · sitemap 转 CSV · 网站地图 Excel 导出 · sitemap URL 去重 · sitemap lastmod 提取 · 网站地图最后修改时间 · sitemap 优先级元数据 · changefreq 更新频率提示 · sitemap 来源追溯 · 网站地图遍历层级 · 有范围限制的索引读取 · 技术 SEO URL 清单 · 网站迁移网址清单 · 产品 sitemap URL 导出 · 文档网站地图清单 · 公开政府网站 sitemap · 定时网站地图导出 · sitemap URL 差异比较 · Apify 网站地图 Actor · sitemap 提取 API · AI 智能体 URL 清单 · Apify MCP 网站地图工具 · 无需网站登录的 sitemap 提取 · 公开 XML sitemap 解析 · 网站地图数据集导出

免责声明

本工具为独立 sitemap 提取工具，与 GOV.UK、目标网站、搜索引擎及相关商标权利人没有隶属或背书关系。网站名称和商标归其各自所有者；示例仅用于标识公开测试目标，不表示商业合作。

仅使用您获准访问和处理的公开数据。请遵守适用法律、网站条款、访问限制、知识产权及隐私义务，包括适用时的 GDPR 和 CCPA。公开 URL 也可能包含个人或敏感信息，请最小化收集、控制导出权限，并按用途设置保留期限。

结果反映有范围限制的运行期间可读取的 sitemap 声明。完整性、元数据准确性、页面可访问性、搜索引擎收录、运行时间及后续业务效果均不保证。本说明属于技术产品文档，不是法律意见。

参考资源

• 网站 Sitemap URL 提取器
• myagizm Actor 主页
• Apify API 文档
• Apify Actors 文档
• Apify 数据集与导出
• Apify 定时运行
• Apify 集成
• Apify webhooks
• Apify MCP 服务