Go to example tasks
Discover all site URLs via sitemap.xml
Created by
新世紀書僮
Find every URL in a website XML sitemap via robots.txt and common paths, follow sitemap indexes, and tag PDF/DOCX/XLSX links for SEO audits or RAG ingest. Exports DOC_TO_MARKDOWN_INPUT for the PDF/DOCX Actor.
Sitemap URL Extractor — PDF/DOCX Tags + robots.txtingenious_quip_bxq/sitemap-url-discovery
URL
Type
Last modified
Change freq.
+3 fieldsTextNumberBooleanListObject
Input
Websites or sitemap URLs(required)
url:https://www.apify.com
Max URLs:100
Read robots.txt:true
Try common sitemap paths:true
Output fields
URL
Type
Last modified
Change freq.
Priority
Source sitemap
Input
Sign up on Apify01
Create your Apify account to access the Sitemap URL Extractor — PDF/DOCX Tags + robots.txt.
Start the run02
The Actor will start running based on the input automatically.
Receive the output03
Monitor the progress in real-time. You will be notified as soon as your dataset is complete and ready for review.
Integrate into your workflow04
The final output is delivered in JSON, CSV, or Excel format, ready to be plugged into your workflow.
