Pricing

from $0.70 / 1,000 domain analyzed & extracteds

Try for free

Go to Apify Store

Website Metadata Extractor(sitemap, socialLinks, robotsTxt)

Try for free

A fast metadata extractor for all meta tags, robots.txt, sitemaps, 15+ social links, H1s, word count, JSON-LD data, contacts & technology detection. Now with AI business & industry classification. Pay-per-event from $1/1k domains with zero server rental.

Pricing

from $0.70 / 1,000 domain analyzed & extracteds

Rating

5.0

(2)

Developer

CodeScraper

Actor stats

Bookmarked

117

Total users

Monthly active users

19 days ago

Last modified

Categories

SEO tools

Developer tools

Automation

Target Website URLs

startUrls

Required

List of website URLs or domains (e.g. 'apify.com', 'https://shopify.com', 'www.google.com'). Any format is supported.

Type:string[]

Fast Mode (Skip robots.txt & Sitemaps)

disableDomainAnalysis

Optional

If enabled, the actor will skip external robots.txt and sitemap.xml resolution, analyzing only the on-page HTML for maximum extraction speed.

Type:boolean

Default:false

Max Parallel Concurrency

maxConcurrency

Optional

Maximum number of domains processed simultaneously. Optimized for high-throughput multi-domain crawling.

Type:integer

Minimum:1

Maximum:50

Default:20

Extract Meta Tags & SEO Elements

extractMetaTags

Optional

Extracts title, description, canonical, charset, viewport, OpenGraph, Twitter Cards, theme color, icons, and verification tags.

Type:boolean

Default:true

Extract robots.txt Rules

extractRobotsTxt

Optional

Fetches and parses robots.txt into structured per-user-agent allow/disallow rules.

Type:boolean

Default:true

Discover & Validate Sitemaps

extractSitemaps

Optional

Extracts sitemap URLs from robots.txt and verifies the standard /sitemap.xml location.

Type:boolean

Default:true

Extract Verified Social Profiles

extractSocialLinks

Optional

Performs multi-engine deep scanning across DOM, headers, footers, JSON-LD, and scripts for brand social profiles.

Type:boolean

Default:true

Detect Technology Stack

extractTechnologies

Optional

Detects CMS (WordPress, Shopify, Wix), Frameworks (React, Next.js, Vue, Nuxt), Analytics, and Web Tools.

Type:boolean

Default:true

Extract Content Headings (H1, H2, H3)

extractHeadings

Optional

Extracts primary H1, all H1s, all H2s, and all H3 tags for on-page SEO content hierarchy analysis.

Type:boolean

Default:true

Extract JSON-LD Structured Data

extractJsonLd

Optional

Parses all Schema.org structured data blocks (Organization, Product, WebSite, FAQ, Article, etc.).

Type:boolean

Default:true

Extract Hreflang Alternates

extractHreflang

Optional

Extracts international alternate language and regional URL definitions.

Type:boolean

Default:true

Extract Content Word & Char Counts

extractContentMetrics

Optional

Calculates total visible body word count, character count, and language declaration.

Type:boolean

Default:true

Target Social Media Platforms

socialPlatforms

Optional

Select the social media networks to target for extraction.

Type:string[]

Extract Email & Phone Numbers

includeContactInfo

Optional

If enabled, discovers mailto: email addresses and tel: phone links.

Type:boolean

Default:false

Enable AI Nature of Business & Company Intelligence

enableAiClassification

Optional

When enabled, uses high-speed Groq AI (openai/gpt-oss-120b) to extract full company intelligence (industry, summary, B2B/B2C, business model, ICP, core offerings, stage, compliance signals) into the natureOfBusiness field.

Type:boolean

Default:false

Groq API Key (Optional)

groqApiKey

Optional

Optional: Your Groq API key from console.groq.com. If omitted, the Actor will use the GROQ_API_KEY environment variable configured in your Apify Console.

Type:string

Custom Allowed Industries (Optional)

customIndustries

Optional

Optional: Custom list of industries for AI classification. If left empty, the standard list of 100+ global industries is used.

Type:string[]

German Imprint Scraper (Contact+Social Links)

codescraper/german-impressum-scraper-fast

An ultra-fast, AI-enhanced scraper that automatically finds and extracts Impressum data from German websites. Get perfectly structured company names, addresses, emails, phones, tax IDs, and social links for B2B enrichment.

CodeScraper

5.0

(2)

Website Social Links Extractor

codescraper/website-social-links-extractor

⚡ Extract 30+ social profiles (💼 LinkedIn, 🪶 X, 📘 Facebook, 📸 Instagram, ▶️ YouTube, 🎵 TikTok, 🐙 GitHub) from any website using Playwright. Scans JS SPAs, filters out tracking pixels, and delivers clean company channels ready for Excel, CSV, and CRM exports.

CodeScraper

190

5.0

(5)

Sitemap URL Extractor

onescales/sitemap-url-extractor

Provide a website link to a sitemap.xml and the app will extract and list all URLs in the sitemap as well as additional data in the sitemap (i.e. https://onescales.com/sitemap.xml).

One Scales

662

5.0

(3)

Google Search SERP Crawler

radiant_martian/google-search-serp-crawler

This is a Google Search crawler with comprehensive information categorized for your AI agent or RPA use.

DataGarage

192

Sitemap Extractor

apify/sitemap-extractor

This Apify Actor extracts all URLs from a website's sitemaps and checks their status codes via lightweight HTTP requests. It provides a clean list of valid links, acting as an ideal pre-processor to ensure your larger crawling projects target only active URLs.

Apify

439

3.2

(8)

Complete SEO Audit Tool - Website Crawler with 0-100 Scores

smart-digital/complete-seo-audit-tool

SEO checker, analyzer and website audit tool: crawl a site and get a 0-100 SEO score for every page, plus a site audit report. Technical and on-page SEO: meta tags, headings, Core Web Vitals, broken links, images and schema, each issue with its priority and a fix hint, as JSON.

My Smart Digital

801

5.0

(4)

Email Domain Scraper - Extract Emails From Websites by Domain

s-r/free-email-domain-scraper

Enter up to 10 company domains or website URLs and get the email addresses published for each one: one row per address with its domain, ready for CSV, Excel or JSON. For lead generation, outreach lists and finding press, sales or support contacts. You only pay per email found.

1.9K

5.0

(2)

Meta Data Extractor

dainty_screw/metadata-extractor-reliable-web-page-metadata-extraction

Metadata Extractor is your go-to tool for extracting meta-data from web pages. Using Cheerio, it parses HTML to extract titles, descriptions, authors, and more.Perfect for content managers and SEO experts.

codemaster devops

5.0

(1)

YouTube Channel & Video Scraper (Emails & Leads)

codescraper/youtube-channel-scraper

Blazing-fast YouTube scraper for creator emails, social links & video metrics. Target by keyword, channel, or video URL. Get 100% verified emails (AI-cleaned), social profiles, subscriber stats & video analytics. Instant zero-cost pre-filtering: free compute, pay only for qualified leads!

CodeScraper

5.0

(2)

G2 Reviews Scraper

azzouzana/g2-scraper

From $1/1K 💰 Scrape G2 product reviews by URL or slug. Export JSON or CSV with rating, pros, cons, NPS and 45+ fields. Sort, filter, up to 30K/run. From $1/1K. Competitive intel & AI workflows.