Website Metadata Extractor (meta tags, sitemap, robots) 🔎
Pricing
$12.00/month + usage
Website Metadata Extractor (meta tags, sitemap, robots) 🔎
🔍 Website Metadata Extractor 🌐 Extract essential website data: meta tags, robots.txt, and sitemap.xml in one scan. 📊 Analyze SEO elements, crawler directives, and site structure. ✅ Perfect for SEO audits, 🔎 competitor research, and 🚀 understanding how search engines view your website.
0.0 (0)
Pricing
$12.00/month + usage
1
Total users
4
Monthly users
3
Runs succeeded
>99%
Issues response
1.1 days
Last modified
14 days ago
File input and bulk domain crawling cause repeated errors and timeouts
Closed
We are trying to use the actor "Website Metadata Extractor (meta tags, sitemap, robots)" to process a bulk list of domains. However, we are consistently facing two key issues:
Timeout errors – Especially when processing a large list, the actor often fails to complete or takes too long.
File input doesn't work – When we upload a file with domain URLs, it fails every time. No output is generated, and the run either crashes or gives no result.
Also, the actor requires each domain to start with https:// — otherwise, it fails. This adds extra manual work when working with bulk lists.
We would really appreciate:
Recommendations for the ideal number of domains per run
A fix or guidance on how to use the file input successfully
Automatic handling of domains without the https:// prefix
Powerful Bachelor (powerful_bachelor)
Thank you for your feedback. We are currently working on implementing the requested modifications to the actor. Please allow us some time to apply and test the changes. We will provide an update soon.
Powerful Bachelor (powerful_bachelor)
Thank you for your patience while we worked on improving the Website Metadata Extractor actor. We're pleased to inform you that we've fixed the issues you reported. We recommend processing 25-50 domains per run. The new version is live now. Would you like to try the new version with your list of domains?
