Website Content Crawler

Under maintenance

Pricing

from $0.01 / 1,000 results

Try for free

Go to Apify Store

Website Content Crawler

Under maintenance

Try for free

Crawl websites for SEO audits. Extracts HTML, title, meta tags, headings, links, & text content from pages. Automatic sitemap detection & parsing Extracts metadata (title, description, OG tags) Heading structure (H1, H2, H3) Internal & external link analysis Image extraction w/alt text Word count

Pricing

from $0.01 / 1,000 results

Rating

0.0

(0)

Developer

John Rippy

Maintained by Community

Actor stats

Bookmarked

Total users

Monthly active users

3 days ago

Last modified

Categories

SEO tools

Developer tools

Automation

You can access the Website Content Crawler programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {};
11
12// Run the Actor and wait for it to finish
13const run = await client.actor("alizarin_refrigerator-owner/website-crawler").call(input);
14
15// Fetch and print Actor results from the run's dataset (if any)
16console.log('Results from dataset');
17console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
18const { items } = await client.dataset(run.defaultDatasetId).listItems();
19items.forEach((item) => {
20    console.dir(item);
21});
22
23// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

Website Content Crawler API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use Website Content Crawler API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

Website Content Crawler API in Python

Website Content Crawler API through CLI

Website Content Crawler OpenAPI definition

Website Content Crawler API

Website Analyzer Crawler

quarterly_lettuce/website-analyzer-crawler

A powerful web crawler that analyzes websites and extracts comprehensive SEO data including meta tags, headings structure, word count, internal/external links, and images.

Abhishek Kumar Giri

DataForSeo On-page SEO

alizarin_refrigerator-owner/dataforseo-onpage

This actor performs in-depth on-page SEO audits using the DataForSEO On-Page API. Page titles, meta descriptions, headings, content, technical SEO factors w/actionable recommendations. Technical SEO, Content Analysis, Meta Tags, Heading Structure, Image Optimization, Link Analysis & Core Web Vitals

John Rippy

Website Title & Heading Quality Checker

gr_59017/website-title-heading-quality-checker

Analyzes website title tags and heading structure (H1–H6) to evaluate SEO quality, content hierarchy, and best practices. Detects issues like missing or multiple H1s, improper heading order, and suboptimal title length, and returns a quality score with suggestions.

Gautam Rana

Website Metadata Extractor (meta tags, sitemap, robots) 🔎

powerful_bachelor/website-metadata-extractor

🔍 Website Metadata Extractor 🌐 Extract essential website data: meta tags, robots.txt, and sitemap.xml in one scan. 📊 Analyze SEO elements, crawler directives, and site structure. ✅ Perfect for SEO audits, 🔎 competitor research, and 🚀 understanding how search engines view your website.

Powerful Bachelor

Website text scraper

spark_actors/website-text-scraper

Extracts key content from any website URL you provide. It fetches the page’s title, meta description, all headings (H1 to H6), paragraphs, links, and tables — delivering structured data for easy use. Ideal for quick insights, SEO analysis, or data extraction without complex setup.

muhammad ubaid

No-BS Content Crawler 🖕

successful_nonagon/no-bs-content-crawler

Fast web crawler that extracts clean text from websites. Returns readable content, headings, and links. Perfect for content aggregation, SEO research, and data collection.

hafsah nuzhat

5.0

Website Metadata Extractor(sitemap, socialLinks, robotsTxt)

codescraper/website-metadata-extractor

A very fast metadata extractor to get all meta tags, robots.txt, sitemaps, social links, H1s, word count, and JSON-LD data. Also provides technology detection for a full analysis. Get your data fast for just $3/month.

CodeScraper

5.0

SEO Data Extractor

nocodeventure/seo-data-extractor

Extract comprehensive SEO metadata, headings, links, images, Open Graph tags, Twitter Cards, and technical data from websites. Perfect for SEO audits, competitor analysis, and content optimization. Runs on Apify platform with structured JSON output.

No-Code Venture

Meta Tags Scraper

rl1987/meta-tags-scraper

Web page metadata scraper.

R.L.

Competitor-Based Keyword Recommendations for On-Page SEO

antonio_espresso/keyword-competitor-recommendation

This actor takes a keyword, language, and Google engine, then returns structured SEO insights: ideal word count, title/content terms with usage ranges, relevant questions (H1–H3, PAA), and competitor data including URLs, rankings, titles, and content scores.