Pricing

$12.00/month + usage

Go to Apify Store

Website Metadata Extractor (meta tags, sitemap, robots) 🔎

Try Actor

Developed by

Powerful Bachelor

🔍 Website Metadata Extractor 🌐 Extract essential website data: meta tags, robots.txt, and sitemap.xml in one scan. 📊 Analyze SEO elements, crawler directives, and site structure. ✅ Perfect for SEO audits, 🔎 competitor research, and 🚀 understanding how search engines view your website.

0.0 (0)

Pricing

$12.00/month + usage

Issues response

1.1 days

Last modified

2 months ago

SEO tools

Developer tools

Automation

You can access the Website Metadata Extractor (meta tags, sitemap, robots) 🔎 programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = { "startUrls": ["https://www.apify.com"] }
9
10# Run the Actor and wait for it to finish
11run = client.actor("powerful_bachelor/website-metadata-extractor").call(run_input=run_input)
12
13# Fetch and print Actor results from the run's dataset (if there are any)
14print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
15for item in client.dataset(run["defaultDatasetId"]).iterate_items():
16    print(item)
17
18# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

Website Metadata Extractor (meta tags, sitemap, robots) 🔎 API in Python

The Apify API client for Python is the official library that allows you to use Website Metadata Extractor (meta tags, sitemap, robots) 🔎 API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

Website Metadata Extractor (meta tags, sitemap, robots) 🔎 API in JavaScript

Website Metadata Extractor (meta tags, sitemap, robots) 🔎 API through CLI

Website Metadata Extractor (meta tags, sitemap, robots) 🔎 OpenAPI definition

Website Metadata Extractor (meta tags, sitemap, robots) 🔎 API

Sitemap URL Extractor

onescales/sitemap-url-extractor

Provide a link to a sitemap.xml and the app will extract and list all URLs in the sitemap as well as additional data in the sitemap (i.e. https://onescales.com/sitemap.xml).

One Scales

102

5.0

Sitemap Scraper

pvillalva/sitemap-scraper

The Sitemap Scraper extracts and outputs all URLs from a given sitemap.

Percival Villalva

5.0

Find Sitemap from url

eesti/find-sitemap-from-url

A powerful [Apify Actor] that finds sitemap URLs for any website. This Actor helps you discover XML sitemaps by checking common locations, robots.txt files, and analyzing HTML content for sitemap links.

ando

Meta Data Extractor

dainty_screw/metadata-extractor-reliable-web-page-metadata-extraction

Metadata Extractor is your go-to tool for extracting meta-data from web pages. Using Cheerio, it parses HTML to extract titles, descriptions, authors, and more.Perfect for content managers and SEO experts.

codemaster devops

Sitemap Detector

coder_zoro/sitemap-detector

Find sitemap URLs fast with our free Sitemap Finder tool. Instantly detect sitemaps from any website for SEO audits, indexing checks, and crawl planning. Improve visibility, site structure insights, and search engine performance in just seconds

Zoro

5.0

URL Metadata Crawler

easyapi/url-metadata-crawler

Extracting comprehensive metadata from web pages. Gather vital information like meta tags, favicons, Open Graph tags, and more, all while enjoying flexible options for customization. Perfect for SEO specialists, developers, and content creators looking to enhance their web presence! 🌐

EasyApi

Simple SEO Data Extractor

onescales/simple-seo-data-extractor

Grab SEO data from any webpage / URL and export the URL, Title Tag, Meta Description, Meta Keywords, Status Code, Canonical Tag and Meta Robots easily. Run the scraper for 1-100,000 pages. Run one time or on schedule or via API.

One Scales

5.0

Simple SEO Auditor Plus

pajoe/simple-seo-auditor-plus

Run a fast, free SEO audit to analyze your website’s meta tags, broken links, page speed, mobile usability, and structured data. Get detailed, actionable insights to boost your search rankings.

va-gasd

Pro SEO Audit Tool - Get Your Website Data for Search Engines

dainty_screw/seo-audit-tool-pro

Elevate your website's search engine ranking with our SEO Audit Tool Pro. Conduct comprehensive SEO audits to identify broken links, missing images, and potential page enhancements. Unlock insights for optimal website performance and visibility.

codemaster devops

SEO Checker

louisdeconinck/seo-checker

SEO Checker is an advanced Actor that performs comprehensive on-site SEO analysis for any website. It crawls web pages and extracts crucial SEO elements, providing detailed insights to help improve your website's search engine optimization.