Pricing

$1.00 / 1,000 statuscodes

Go to Store

HTTP Status Codes and URL Checker

Try for free

Developed by

Antonio Blago

A HTTP Status Codes Crawler is a tool that scans a website and retrieves HTTP status codes for each page. This helps in diagnosing errors and optimizing technical SEO.

0.0 (0)

Pricing

$1.00 / 1,000 statuscodes

Total users

Monthly users

Runs succeeded

88%

Last modified

12 days ago

SEO tools

Automation

Start URLs

start_urlsarrayRequired

List of URLs to start crawling from.

Follow Links

follow_linksbooleanOptional

Whether to follow links from the start URLs.

Default value of this property is false

Choose Sitemap modus

sitemap_modestringOptional

mode: 'sitemap' (only use sitemap), 'crawl' (only crawl), or 'auto' (default: try sitemap first)

Default value of this property is "auto"

Maximum Depth

max_depthintegerOptional

The maximum depth for crawling from the start URL.

Default value of this property is 1

Max URLs to Crawl

CLOSESPIDER_PAGECOUNTintegerOptional

The maximum number of pages to crawl before stopping.

Default value of this property is 5

Concurrent Requests per Domain

CONCURRENT_REQUESTS_PER_DOMAINintegerOptional

Number of simultaneous requests per domain.

Default value of this property is 1

User-Agent

USER_AGENTstringOptional

The user agent string for requests.

Default value of this property is "custom-user-agent apify"

Respect Robots.txt

ROBOTSTXT_OBEYbooleanOptional

Whether the crawler should obey the robots.txt rules.

Default value of this property is true

Download Delay

DOWNLOAD_DELAYintegerOptional

The time delay (in seconds) between requests.

Default value of this property is 1

HTML Scraper pro

scrapingxpert/html-scraper-pro

The HTML Scraper Pro is a powerful tool designed to extract the HTML source code and metadata from websites. It uses advanced web scraping techniques to retrieve the full HTML content of web pages,page title and HTTP status code.This tool is ideal for data extraction, website analysis, and archiving

scrapingxpert

URL Metadata Crawler

easyapi/url-metadata-crawler

Extracting comprehensive metadata from web pages. Gather vital information like meta tags, favicons, Open Graph tags, and more, all while enjoying flexible options for customization. Perfect for SEO specialists, developers, and content creators looking to enhance their web presence! 🌐

EasyApi

Website Spell Checker with AI

oscardz/website-spell-checker

The Website Spell Checker with AI scans your site for spelling and grammar errors, providing detailed reports with corrections. Export results in various formats or integrate into your workflow for automated checks, ensuring your content is always polished.

Oscar Rodriguez

148

Metadata Scraper

autofacts/metadata-scraper

A powerful web scraper that extracts various types of structured metadata from web pages, including JSON-LD, Microdata, Open Graph, Twitter Cards, and more. Perfect for SEO analysis, content aggregation, and research purposes.

Autofactor

5.0

Page Scraping Analyzer

apify/page-analyzer

Performs analysis of a webpage to figure out the best way how to scrape its data. Provide a URL and data points to find and get back a detailed dashboard showing how the data can be scraped. Works with initial and rendered HTML, JavaScript variables and dynamically loaded data.

Apify

1.3K

4.4

General Purpose Web Scraping and Metadata Extraction

moving_beacon-owner1/my-actor-10

This project uses the Apify platform to scrape data from web pages, collect metadata, and store results in an Apify dataset. It features functions for managing date ranges, encoding identifiers, and handling large datasets, aiming to efficiently extract and store structured data for analysis.

Jamshaid Arif

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

8.2K

4.7

URL to Metadata

njoylab/url-summary-scraper

A powerful Apify actor that extracts essential website information, including title, description, images, and social media links. Perfect for quick data gathering and insights from any URL.

njoylab

5.0

Vanilla JS Scraper

mstephen190/vanilla-js-scraper

Scrape the web using familiar JavaScript methods! Crawls websites using raw HTTP requests, parses the HTML with the JSDOM package, and extracts data from the pages using Node.js code. Supports both recursive crawling and lists of URLs. This actor is a non jQuery alternative to CheerioScraper.

Matthias Stephens

466

Metadata Extractor

jancurn/extract-metadata

A small efficient actor that loads a web page, parses its HTML using Cheerio library and extracts the following meta-data from the <HEAD> tag, such as page title, description, author etc.