HTTP Status Codes and URL Checker avatar
HTTP Status Codes and URL Checker

Pricing

$1.00 / 1,000 StatusCodes

Go to Store
HTTP Status Codes and URL Checker

HTTP Status Codes and URL Checker

antonio_espresso/website-status-code-crawler

Developed by

Antonio Blago

Maintained by Community

A HTTP Status Codes Crawler is a tool that scans a website and retrieves HTTP status codes for each page. This helps in diagnosing errors and optimizing technical SEO.

0.0 (0)

Pricing

$1.00 / 1,000 StatusCodes

1

Monthly users

5

Runs succeeded

>99%

Last modified

23 days ago

Start URLs

start_urlsarrayRequired

List of URLs to start crawling from.

follow_linksbooleanOptional

Whether to follow links from the start URLs.

Default value of this property is false

Choose Sitemap modus

sitemap_modestringOptional

mode: 'sitemap' (only use sitemap), 'crawl' (only crawl), or 'auto' (default: try sitemap first)

Default value of this property is "auto"

Maximum Depth

max_depthintegerOptional

The maximum depth for crawling from the start URL.

Default value of this property is 1

Max URLs to Crawl

CLOSESPIDER_PAGECOUNTintegerOptional

The maximum number of pages to crawl before stopping.

Default value of this property is 5

Concurrent Requests per Domain

CONCURRENT_REQUESTS_PER_DOMAINintegerOptional

Number of simultaneous requests per domain.

Default value of this property is 1

User-Agent

USER_AGENTstringOptional

The user agent string for requests.

Default value of this property is "custom-user-agent apify"

Respect Robots.txt

ROBOTSTXT_OBEYbooleanOptional

Whether the crawler should obey the robots.txt rules.

Default value of this property is true

Download Delay

DOWNLOAD_DELAYintegerOptional

The time delay (in seconds) between requests.

Default value of this property is 1

Pricing

Pricing model

Pay per result 

This Actor is paid per result. You are not charged for the Apify platform usage, but only a fixed price for each dataset of 1,000 items in the Actor outputs.

Price per 1,000 items

$1.00