Website Metadata Extractor(sitemap, socialLinks, robotsTxt)

Pricing

$3.00/month + usage

Try for free

Go to Apify Store

Website Metadata Extractor(sitemap, socialLinks, robotsTxt)

Try for free

A very fast metadata extractor to get all meta tags, robots.txt, sitemaps, social links, H1s, word count, and JSON-LD data. Also provides technology detection for a full analysis. Get your data fast for just $3/month.

Pricing

$3.00/month + usage

Rating

5.0

(1)

Developer

CodeScraper

Maintained by Community

Actor stats

Bookmarked

Total users

Monthly active users

4 days ago

Last modified

Categories

SEO tools

Developer tools

Automation

You can access the Website Metadata Extractor(sitemap, socialLinks, robotsTxt) programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {
11    "startUrls": [
12        "https://apify.com",
13        "https://www.google.com",
14        "https://www.youtube.com",
15        "https://www.shopify.com"
16    ]
17};
18
19// Run the Actor and wait for it to finish
20const run = await client.actor("codescraper/website-metadata-extractor").call(input);
21
22// Fetch and print Actor results from the run's dataset (if any)
23console.log('Results from dataset');
24console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
25const { items } = await client.dataset(run.defaultDatasetId).listItems();
26items.forEach((item) => {
27    console.dir(item);
28});
29
30// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

Website Metadata Extractor & SEO Scraper API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use Website Metadata Extractor(sitemap, socialLinks, robotsTxt) API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

Website Metadata Extractor(sitemap, socialLinks, robotsTxt) API in Python

Website Metadata Extractor(sitemap, socialLinks, robotsTxt) API through CLI

Website Metadata Extractor(sitemap, socialLinks, robotsTxt) OpenAPI definition

Website Metadata Extractor(sitemap, socialLinks, robotsTxt) API

Website Metadata Extractor (meta tags, sitemap, robots) 🔎

powerful_bachelor/website-metadata-extractor

🔍 Website Metadata Extractor 🌐 Extract essential website data: meta tags, robots.txt, and sitemap.xml in one scan. 📊 Analyze SEO elements, crawler directives, and site structure. ✅ Perfect for SEO audits, 🔎 competitor research, and 🚀 understanding how search engines view your website.

Powerful Bachelor

Sitemap Scraper

pvillalva/sitemap-scraper

The Sitemap Scraper extracts and outputs all URLs from a given sitemap.

Percival Villalva

171

5.0

Find Sitemap from url

eesti/find-sitemap-from-url

A powerful [Apify Actor] that finds sitemap URLs for any website. This Actor helps you discover XML sitemaps by checking common locations, robots.txt files, and analyzing HTML content for sitemap links.

ando

165

1.0

Sitemap URL Extractor

onescales/sitemap-url-extractor

Provide a website link to a sitemap.xml and the app will extract and list all URLs in the sitemap as well as additional data in the sitemap (i.e. https://onescales.com/sitemap.xml).

One Scales

289

5.0

Meta Data Extractor

dainty_screw/metadata-extractor-reliable-web-page-metadata-extraction

Metadata Extractor is your go-to tool for extracting meta-data from web pages. Using Cheerio, it parses HTML to extract titles, descriptions, authors, and more.Perfect for content managers and SEO experts.

codemaster devops

5.0

Website extract

mrahil/my-actor

It is website extractor

Mohammed Rahil

137

Sitemap Generator

igview-owner/sitemap-generator

Automatically crawl any website and generate XML, HTML, and text sitemaps for SEO optimization. Perfect for submitting to Google Search Console, Bing Webmaster Tools, and improving search engine indexing. no manual work required. Free sitemap generator tool for WordPress, Blogger, and all website.

Sachin Kumar Yadav

Metadata Scraper

louisdeconinck/metadata-scraper

Automatically scrape metadata such as title, description, heading and article from websites. It will crawl the start URLs and then scrape the metadata from the detail pages automatically navigating through the pagination.

Louis Deconinck

123

5.0

URL to Metadata - mail, social and more

njoylab/url-summary-scraper

A powerful Apify actor that extracts essential website information, including title, description, images, mail, and social media links. Perfect for quick data gathering and insights from any URL.

njoylab

104

5.0

URL Metadata Crawler

easyapi/url-metadata-crawler

Extracting comprehensive metadata from web pages. Gather vital information like meta tags, favicons, Open Graph tags, and more, all while enjoying flexible options for customization. Perfect for SEO specialists, developers, and content creators looking to enhance their web presence! 🌐