Robots.txt Validator

Pricing

Pay per usage

Try for free

Go to Apify Store

Robots.txt Validator

Try for free

List of website base URLs whose robots.txt files will be validated

Pricing

Pay per usage

Rating

5.0

(1)

Developer

riya rawat

Maintained by Community

Actor stats

Bookmarked

Total users

Monthly active users

11 days ago

Last modified

JavaScript Website Scraper (Crawlee + CheerioCrawler)

A lightweight and efficient Apify Actor for scraping static and semi-dynamic websites using Crawlee with CheerioCrawler. This Actor extracts page titles and URLs from provided start pages and stores them in an Apify dataset.

Designed to be fast, easy to customize, and fully compliant with Apify Actor Store rules and the Apify $1 Million Challenge.

Key Features

Fast HTML parsing using Cheerio (no browser required)
Built on Crawlee’s CheerioCrawler
Supports Apify Proxy rotation to reduce blocking
Clearly defined input and output
Stores structured data in Apify Datasets
Easy to extend for custom scraping use cases

Quick Start

Run the Actor locally:

$apify run

Redirect Chain Analyzer

predictable_function/my-actor-2

List of URLs whose HTTP redirect chains will be analyzed

riya rawat

5.0

Sitemap Health Validator

predictable_function/my-actor

Validates sitemap.xml files and checks health of listed URLs

riya rawat

5.0

Website Title & Heading Quality Checker

gr_59017/website-title-heading-quality-checker

Analyzes website title tags and heading structure (H1–H6) to evaluate SEO quality, content hierarchy, and best practices. Detects issues like missing or multiple H1s, improper heading order, and suboptimal title length, and returns a quality score with suggestions.

Gautam Rana

5.0

Website Uptime & Response Monitor

predictable_function/website-uptime-response-monitor

Monitors website uptime and response time for given URLs.

riya rawat

5.0

Cookie Consent Checker

pillowy_travel/cookie-consent-checker

Checks whether a website displays a cookie consent banner and basic GDPR compliance signals.

SAHIL KUMAR

PDF Proofreader

gr_59017/pdf-proofreader

Analyzes PDF documents to detect basic spelling and grammar issues by extracting text content. Provides a proofreading quality score and highlights common writing mistakes to help improve document clarity and correctness.

Gautam Rana

Website HTTPS & SSL Health Checker

pillowy_travel/website-https-ssl-health-checker

Checks HTTPS availability and SSL certificate health for given URLs

SAHIL KUMAR

5.0

Broken Link Finder

pillowy_travel/broken-link-finder

Finds and analyzes broken links on given web pages

SAHIL KUMAR

SEO Duplicate Content Detector

gr_59017/seo-duplicate-content-detector

Detects duplicate or identical content across multiple webpages by analyzing visible page text. Helps identify SEO duplicate content issues, content reuse, and potential ranking risks using simple content comparison and scoring.

Gautam Rana