# Bulk HTTP Status Checker: Redirects, SSL Expiry and Run Changes (`getascraper/http-status-intelligence-monitor`) Actor Bulk HTTP status checker and uptime monitor for URL lists, sitemaps or page links. Returns final status, every redirect hop, DNS, TLS and first-byte timing, SSL expiry and soft 404 flags per URL, plus a change feed of newly broken, recovered and slower pages between runs. Pay per result. - **URL**: https://apify.com/getascraper/http-status-intelligence-monitor.md - **Developed by:** [GetAScraper](https://apify.com/getascraper) (community) - **Categories:** SEO tools, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.88 / 1,000 url checks This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## ๐Ÿšฆ Bulk HTTP Status Checker: Redirects, SSL Expiry and Run Changes
Know which links broke before your visitors do
Check a list of web addresses, a whole sitemap, or every link on a page. Get status, redirects, speed, certificate expiry and what changed since the last run.
๐Ÿ”— Full redirect trail
Every step with its code, so a messy chain shows up before it costs you traffic. 		๐Ÿšจ Only what needs fixing
Problems are labelled and ranked. Return just the broken, slow or expiring addresses. 		๐Ÿ”„ Changes between runs
See what broke, recovered, moved or slowed down since last time, with uptime percent. 		โœ… Rules that fit your site
Expected status, required text, headers and certificate warnings in one pass.
Paste your addresses, point to a sitemap, or give it a page. It checks every address, follows each redirect, measures speed, reads the certificate, and tells you what is broken and what changed since the last run. No login, no API key. ### ๐Ÿ” What does this HTTP status checker do? It is a broken link checker, redirect checker and HTTP status code checker in one Actor. It also watches your addresses over time, so a scheduled run turns into a simple uptime and change report. For every address you get: - The final status, the first status, and the full redirect trail with a plain label for each step (http to https, add www, trailing slash, different domain, sent to the home page). - How long it took: total time, time to first byte, and the time spent on domain lookup, connection and the secure handshake. - The SSL certificate: who issued it, when it expires, how many days are left, and whether it is trusted. - Size, type, server, caching and indexing headers, plus the IP address that answered. - A ranked list of problems in plain language, with a short code for each so you can filter. - What changed since the last run, how long an address has been broken, and its uptime percent. You can find addresses three ways: check a list you provide, read them out of a sitemap, or open a page and check every link on it. Deep pages that quietly redirect to the home page are flagged, and with the soft 404 option on, so are pages that load but read like an error page. ### ๐Ÿ’ก Who uses this? - **I am an SEO manager checking a site migration.** I paste the old-to-new redirect map and see at once which pages end in the wrong place or take three hops. - **I am an agency owner running monthly audits for clients.** I schedule one run per client and send the list of broken links and expiring certificates. - **I am a site owner who wants a cheap early warning.** I check my key pages on a schedule and see when one starts failing or slowing down. - **I am a developer after a release.** I confirm that every important address still answers with the right status, the right text and the right headers. ### ๐Ÿš€ How to use
STEP 1
Choose your addresses
Paste a list, point to a sitemap, or give a page and check every link on it. 		STEP 2
Set your rules
Say what a good answer looks like: status, text, headers, speed and certificate life. 		STEP 3
Read problems and changes
Get a ranked list of what is wrong and what changed since the last run.
### ๐Ÿ“ฅ Input | Field | Type | Required | Description | | --- | --- | --- | --- | | `startUrls` | array of URLs | No | The addresses to check. Paste them, upload a text or CSV file, or link to a list online. Bare domains work. To set a rule for one address only, add expectedStatus, expectedFirstStatus or expectedFinalUrl under user data. | | `sourceMode` | enum | No | Check the addresses you listed, read them out of a sitemap (or find the sitemap from a home page), or check every link found on the pages you list. | | `redirectExpectations` | array of key-value pairs | No | A redirect map for site moves: the old address on the left, the address it must end up at on the right. Mismatches are reported as problems. | | `crawlDepth` | integer | No | When checking links on a page, how many levels of pages on the same website to read. 1 reads only the pages you listed. | | `maxPagesToCrawl` | integer | No | The most pages opened to look for links. Defaults to 10. | | `linkScope` | enum | No | Check links that stay on the same website, links to other websites, or both. | | `includeAssets` | boolean | No | Also check images, scripts, stylesheets and embedded frames that a page loads. Off by default. | | `includeUrlPatterns` | array of strings | No | Keep only addresses that match at least one entry. Use \* as a wildcard. Leave empty to keep everything. | | `excludeUrlPatterns` | array of strings | No | Drop addresses that match any entry, for example \*.pdf or /cart/. | | `respectRobotsTxt` | boolean | No | Skip opening pages that a site's robots.txt asks crawlers to avoid when looking for links. On by default. | | `expectedStatus` | array of strings | No | Report a problem when the final answer is not one of these. Accepts codes (200, 301), classes (2xx) and ranges (200-299). | | `mustContainText` | array of strings | No | Report a problem when any of these texts is missing from the page. Not case sensitive. | | `mustNotContainText` | array of strings | No | Report a problem when any of these texts is found on the page, such as an error message or leftover placeholder. | | `requiredHeaders` | array of strings | No | Report a problem when a response header is missing or lacks the text you give, for example x-frame-options: DENY. | | `warnAboveRedirects` | integer | No | Mark an address that needs more redirects than this. Defaults to 2. Use 0 to switch off. | | `slowThresholdMs` | integer | No | Mark an address as slow when the whole check takes longer than this many milliseconds. Defaults to 3000. | | `maxResponseTimeMs` | integer | No | Treat an address as failing when it takes longer than this many milliseconds. 0 means no limit. | | `sslWarnDays` | integer | No | Warn when a secure address has a certificate that expires within this many days. Defaults to 30. | | `detectSoft404` | boolean | No | Flag pages that answer 200 but read like a not-found page. Opens the page itself. Off by default. | | `extractPageBasics` | boolean | No | Add title, description, main heading, canonical address, robots tag and an indexable flag. Opens the page itself. Off by default. | | `includeSecurityHeaders` | boolean | No | Add the security headers each address sends and list the ones that are missing. Off by default. | | `includeResponseHeaders` | boolean | No | Add every response header of the final answer to each row. Cookies are left out. Off by default. | | `probeHttp2` | boolean | No | Find out whether each website offers HTTP/2. Off by default. | | `maxItems` | integer | No | The most addresses checked in one run. Defaults to 10. Up to 10,000. | | `outputFilter` | enum | No | Return every address, only those with problems, only those that changed since the last run, or both problems and changes. | | `trackHistory` | boolean | No | Remember each address between runs so rows show what changed, how long it has been broken and its uptime. On by default. | | `monitorScope` | string | No | A name for this monitor. Runs with the same name are compared with each other. Use one name per scheduled list. | | `responseRegressionPercent` | integer | No | Flag an address that answers this much slower than its usual time. 100 means twice as slow. 0 switches it off. | | `confirmChanges` | boolean | No | Check an address once more when it looks different from last time, so a short blip is not reported as a change. On by default. | | `requestMethod` | enum | No | Use the lightest request first and a full request when the server rejects it, always use a full request, or never fall back. | | `followRedirects` | boolean | No | Follow redirects and record every step. Turn off to report only the first answer. On by default. | | `maxRedirects` | integer | No | Stop following after this many redirects and report a redirect problem. Defaults to 10. | | `timeoutSecs` | integer | No | How long to wait for a server before giving up. Defaults to 20 seconds. | | `retries` | integer | No | Extra attempts after a timeout, a dropped connection or a busy server. Defaults to 1. | | `userAgentType` | enum | No | Send an honest monitor identity (default) or a common desktop browser identity for sites that turn monitors away. | | `userAgent` | string | No | Your own identity text. When filled in, it replaces the choice above. | | `customHeaders` | array of key-value pairs | No | Extra headers sent with every request, for example Accept-Language. Logins, cookies and tokens are never sent. | | `maxConcurrency` | integer | No | How many addresses are checked at the same moment across all websites. Defaults to 10. | | `maxConcurrencyPerHost` | integer | No | The most requests in flight to any one website. Defaults to 2. Keep it low for small sites. | | `delayPerHostMs` | integer | No | A short pause in milliseconds before each new request to the same website. Defaults to 250. | | `proxyConfiguration` | object | No | Optional Apify Proxy settings to see how a website answers from another country. Off by default. | ### ๐Ÿ“ค Data table Each checked address becomes one row. Fields that do not apply are left out instead of being filled with placeholders. | Field | Type | Description | | --- | --- | --- | | `url` | string | The address that was checked (fragments removed). | | `source` | string | Where the address came from: input, sitemap, page-link, page-asset or redirect-map. | | `foundOn` | string | For sitemap and page links: the sitemap or page the address was found on. | | `foundOnCount` | integer | How many times the address was found in the pages that were read. | | `anchorText` | string | For page links: the clickable text of the first link found. | | `linkType` | string | For page links: the kind of element it came from (a, img, script, link, iframe, media, area). | | `isInternal` | boolean | For page links: true when it stays on the same website as the page. | | `outcome` | string | Plain result of the final answer: ok, redirect, blocked, client-error, server-error, network-error, redirect-error or blocked-address. | | `severity` | string | Worst problem on the row: ok, info, warning or error. | | `statusCode` | integer | Status code of the first response, before any redirect. | | `finalStatusCode` | integer | Status code of the final response after redirects. Missing when no response was received. | | `finalStatusText` | string | Text that goes with the final status code, for example Not Found. | | `statusClass` | string | Group of the final status: 2xx, 3xx, 4xx, 5xx or error. | | `isOk` | boolean | True when the final response is a success (2xx). | | `isBroken` | boolean | True for a missing page, a server error, no response, or a redirect problem. Login walls and rate limits are not counted as broken. | | `finalUrl` | string | Where the address ended up after following redirects. | | `redirectLocation` | string | Where the first response sends visitors, when it is a redirect. | | `redirectCount` | integer | How many redirects were followed. | | `redirectKind` | string | permanent, temporary or mixed. Missing when there was no redirect. | | `redirectTypes` | array | What the redirects do, for example http-to-https, add-www, trailing-slash-added, different-domain, to-homepage. | | `redirectChain` | array | Every step: address, status, destination, what the step does, method and timing. Empty when there was no redirect. | | `redirectedToHomepage` | boolean | True when a deep page redirects to the home page of the same site. | | `responseTimeMs` | integer | Total time for all steps of the check, in milliseconds. | | `ttfbMs` | integer | Time until the first byte of the final response, in milliseconds. | | `dnsMs` | integer | Time to look up the domain for the final step, in milliseconds. | | `connectMs` | integer | Time to open the connection for the final step, in milliseconds. | | `tlsMs` | integer | Time for the secure handshake in the final step, in milliseconds. | | `contentType` | string | Content type of the final response. | | `contentLength` | integer | Size of the final response in bytes, from the server header or measured. | | `contentEncoding` | string | Compression used by the final response, when any. | | `server` | string | Server header of the final response. | | `cacheControl` | string | Cache-Control header of the final response. | | `xRobotsTag` | string | X-Robots-Tag header of the final response. | | `lastModified` | string | Last-Modified header of the final response. | | `resolvedIp` | string | IP address the final response came from. | | `ipVersion` | integer | 4 or 6, the version of that IP address. | | `supportsHttp2` | boolean | Whether the server offers HTTP/2. Only set when HTTP/2 detection is on. | | `httpVersion` | string | HTTP/2 or HTTP/1.1, as negotiated. Only set when HTTP/2 detection is on. | | `method` | string | Request method behind the final response: HEAD or GET. | | `usedGetFallback` | boolean | True when HEAD was refused or reported an error and a full GET request was made to confirm. | | `fallbackReason` | string | Why the GET request was made, for example HEAD_405. | | `requestsMade` | integer | Total requests sent for this address, including redirect steps and retries. | | `sslValid` | boolean | True when the certificate of the first secure step is trusted and matches the host name. | | `sslError` | string | Why the certificate is not trusted, when it is not. | | `sslProtocol` | string | Secure protocol version, for example TLSv1.3. | | `sslIssuer` | string | Organisation that issued the certificate. | | `sslSubject` | string | Host name the certificate was issued to. | | `sslValidFrom` | string | Start of the certificate validity, ISO 8601. | | `sslValidTo` | string | End of the certificate validity, ISO 8601. | | `sslDaysToExpiry` | integer | Days until the certificate expires. Negative when it already has. | | `title` | string | Page title. Only set when page details are on. | | `metaDescription` | string | Meta description. Only set when page details are on. | | `h1` | string | First main heading. Only set when page details are on. | | `canonicalUrl` | string | Canonical address declared by the page. Only set when page details are on. | | `canonicalMatches` | boolean | Whether the canonical address is the address that was reached. | | `metaRobots` | string | Robots meta tag content, lower case. | | `isIndexable` | boolean | True when the page is successful, not noindex and its canonical address matches. Only set when page details are on. | | `soft404Suspected` | boolean | True when the page answers 200 but looks like a not-found page. Only set when soft 404 detection is on. | | `soft404Reason` | string | What made the page look like a not-found page. | | `securityHeaders` | object | Security headers that were present. Only set when security header checks are on. | | `missingSecurityHeaders` | array | Security headers that were not set. Only set when security header checks are on. | | `responseHeaders` | object | All headers of the final response, without cookies. Only set when requested. | | `assertionsPassed` | boolean | True when every rule you set passed. Missing when no rule applied. | | `failedAssertions` | array | Plain-language list of the rules that failed. | | `problemCodes` | array | Short codes for every problem found, for example broken-link, slow-response, ssl-expiring-soon. | | `problems` | array | Every problem with its code, severity and a plain-language message. | | `error` | string | What went wrong when there was no usable response. | | `errorType` | string | Kind of failure: dns-error, timeout, connection-refused, connection-reset, tls-error, private-address-blocked, redirect-loop, too-many-redirects and others. | | `checkedAt` | string | When the check ran, ISO 8601. | | `isFirstCheck` | boolean | True when this address was never checked in this monitor before. | | `changes` | array | What changed since the last run: first-seen, status-changed, newly-broken, recovered, redirect-target-changed or slower. | | `hasChanges` | boolean | True when something other than first-seen changed. | | `statusChanged` | boolean | True when the status of the first or final response differs from last time. | | `destinationChanged` | boolean | True when the address it ends up at differs from last time. | | `newlyBroken` | boolean | True when it worked last time and is broken now. | | `recovered` | boolean | True when it was broken last time and works now. | | `slowerThanBefore` | boolean | True when it answered much slower than its usual time. | | `previousStatusCode` | integer | First-response status code at the last run. | | `previousFinalStatusCode` | integer | Final status code at the last run. | | `previousFinalUrl` | string | Address it ended up at in the last run. | | `previousOutcome` | string | Outcome at the last run. | | `previousResponseTimeMs` | integer | Total response time at the last run. | | `previousBrokenSince` | string | On a recovered address: when it was first seen broken before it came back. | | `previousCheckedAt` | string | When the last run checked it. | | `typicalResponseMs` | integer | Typical response time over recent runs (median). | | `uptimePercent` | number | Share of all runs where the address was not broken, including this one. | | `checksTotal` | integer | How many runs have checked this address. | | `consecutiveFailures` | integer | How many runs in a row it has been broken. | | `brokenSince` | string | When it was first seen broken in the current streak. | | `lastOkAt` | string | When it was last seen working. | | `firstSeenAt` | string | When this monitor first checked the address. | | `rechecked` | boolean | True when a difference from last time was double-checked before being reported. | #### Sample row A real row for an http address that redirects to https, with page details and HTTP/2 detection turned on, on its second run: ```json { "url": "http://github.com/", "outcome": "ok", "severity": "ok", "statusCode": 301, "finalStatusCode": 200, "statusClass": "2xx", "isOk": true, "isBroken": false, "finalUrl": "https://github.com/", "redirectCount": 1, "responseTimeMs": 333, "problemCodes": [], "changes": [], "source": "input", "checkedAt": "2026-10-13T06:02:23.572Z", "finalStatusText": "OK", "redirectLocation": "https://github.com/", "redirectKind": "permanent", "redirectTypes": [ "http-to-https" ], "redirectChain": [ { "url": "http://github.com/", "statusCode": 301, "statusText": "Moved Permanently", "location": "https://github.com/", "redirectTypes": [ "http-to-https" ], "method": "GET", "responseTimeMs": 82, "ttfbMs": 82 }, { "url": "https://github.com/", "statusCode": 200, "statusText": "OK", "method": "GET", "responseTimeMs": 251, "ttfbMs": 132 } ], "ttfbMs": 132, "dnsMs": 14, "connectMs": 39, "tlsMs": 37, "contentType": "text/html; charset=utf-8", "contentLength": 572813, "contentEncoding": "gzip", "server": "github.com", "cacheControl": "max-age=0, private, must-revalidate", "resolvedIp": "20.207.73.82", "ipVersion": 4, "method": "GET", "requestsMade": 2, "problems": [], "sslValid": true, "sslProtocol": "TLSv1.3", "sslIssuer": "Sectigo Limited", "sslSubject": "github.com", "sslValidFrom": "2026-09-01T00:00:00.000Z", "sslValidTo": "2026-11-29T23:59:59.000Z", "sslDaysToExpiry": 47, "supportsHttp2": true, "httpVersion": "HTTP/2", "title": "GitHub ยท Change is constant. GitHub keeps you ahead. ยท GitHub ยท GitHub", "metaDescription": "Join the world's most widely adopted, AI-powered developer platform where millions of developers, businesses, and the largest open source community build software that advances humanity.", "h1": "The future of building happens together", "canonicalUrl": "https://github.com/", "canonicalMatches": true, "isIndexable": true, "soft404Suspected": false, "isFirstCheck": false, "hasChanges": false, "statusChanged": false, "destinationChanged": false, "newlyBroken": false, "recovered": false, "previousStatusCode": 301, "previousFinalStatusCode": 200, "previousFinalUrl": "https://github.com/", "previousOutcome": "ok", "previousResponseTimeMs": 354, "previousCheckedAt": "2026-10-13T06:02:22.760Z", "uptimePercent": 100, "checksTotal": 2, "consecutiveFailures": 0, "lastOkAt": "2026-10-13T06:02:23.659Z", "firstSeenAt": "2026-10-13T06:02:22.760Z" } ``` #### Change feed Every run also saves a change feed. Each entry names the address, what changed and the previous value: ```json { "url": "https://www.iana.org/", "type": "recovered", "checkedAt": "2026-10-13T06:00:33.942Z", "detail": "Was broken since 2026-10-11T08:00:00.000Z, now working again.", "statusCode": 200, "previousStatusCode": 503, "finalUrl": "https://www.iana.org/", "previousFinalUrl": "https://www.iana.org/", "responseTimeMs": 44, "previousResponseTimeMs": 36 } ``` The run summary counts addresses by status, lists the slowest addresses and the worst offenders, shows certificates close to expiry, and records any addresses that were skipped and why. ### ๐Ÿ’ฐ Pricing Pricing is pay per result: you pay for each checked address that is returned to your dataset, and nothing else. Every row costs the same, whether or not you turn on page details, security headers, HTTP/2, soft 404 detection or link finding. Higher Apify plans get a lower price per result. - Runs that return nothing cost nothing, and there are no subscriptions. - Addresses that are refused (local, private or not http and https) and rows hidden by "only problems" or "only changes" are not returned, so they are not billed. - Pages that are opened only to find links are not billed. You pay for the addresses checked, not the pages read. - The maximum charge you set for a run is always respected. The run stops cleanly when it is reached and you keep every row delivered so far. ### โญ Enjoying this HTTP status checker?
โญ โญ โญ โญ โญ
Fewer broken pages, found before your visitors find them.
A 5-star rating takes 10 seconds and helps other SEO teams and site owners find it. Your feedback also tells us what to build next.
โ˜…  Rate this Actor on Apify
### โ“ FAQ ##### Does it hit my servers too hard? No. By default it makes two requests at a time to any one website and waits a quarter of a second between them. You can lower both numbers. If a site answers "too many requests", the monitor pauses that site and slows down on its own. When it reads pages to find links, it follows the site's robots.txt rules. ##### Can it check internal or private addresses? No, on purpose. It only checks public internet addresses. Local names, private network addresses, cloud metadata addresses and anything that is not http or https are refused, and so are redirects that lead there. Addresses you list that are refused are not billed, and the run summary lists each one with the reason. ##### How does it show what changed since the last run? Give your monitor a name and run it on a schedule. The Actor remembers each address between runs and marks it as newly broken, recovered, changed status, redirecting somewhere new, or slower than usual. A difference is checked a second time before it is reported. The first run only saves the starting point. ##### Why does an address show as access restricted? Some sites answer 401, 403 or 429 to automated requests, even when the page is fine for visitors. These are marked as access restricted, not broken, because the check cannot tell. Try the browser identity option, lower the number of parallel checks, or exclude those addresses. ### ๐Ÿ”— Other actors - [HTTP Request Runner: Bulk API Calls with Retries and Redirects](https://apify.com/getascraper/http-request-runner) โ†— - Send bulk requests and follow redirects when you need the response itself, not only its status. - [Technical SEO Audit & Indexability Checker](https://apify.com/getascraper/seo-release-indexability-guard) โ†— - Audit pages for indexability and technical SEO problems before and after a release. - [Website Technology Change Monitor: Tech Signals](https://apify.com/getascraper/website-technology-change-monitor) โ†— - Get told when a website changes the technology it runs on. - [SaaS Pricing Monitor: Competitor Plan Changes](https://apify.com/getascraper/saas-pricing-change-monitor) โ†— - Watch competitor pricing pages and see plan changes between runs. - [Moz Domain Authority Scraper: Bulk DA and Spam Score, No Login](https://apify.com/getascraper/moz-scraper) โ†— - Add domain authority and spam score to the sites you check. # Changelog This Actor's version history is a separate document: https://apify.com/getascraper/http-status-intelligence-monitor/changelog.md # Actor input Schema ## `startUrls` (type: `array`): The web addresses to check. Paste them one by one, upload a text or CSV file, or link to a list online. Bare domains such as example.com work too. Only public http and https addresses are checked. Advanced: add expectedStatus, expectedFirstStatus or expectedFinalUrl under 'User data' to check a rule for one address only. ## `sourceMode` (type: `string`): Check exactly the addresses you listed, read them out of a sitemap (give the sitemap address, or just the site's home page and the sitemap is found for you), or open the listed pages and check every link found on them. ## `redirectExpectations` (type: `array`): Optional. Tests a list of redirects, for example after a site move. On the left, the old address. On the right, the address it must end up at. Use a path such as /new-page to stay on the same site. Mismatches are reported as problems. ## `crawlDepth` (type: `integer`): How far to follow links inside the same website when checking the links found on a page. 1 reads only the pages you listed. 2 also reads the pages they link to, and so on. ## `maxPagesToCrawl` (type: `integer`): The most pages opened to look for links. Raising it finds more links but takes longer. ## `linkScope` (type: `string`): Check links that stay on the same website, links to other websites, or both. ## `includeAssets` (type: `boolean`): When on, files that a page loads (images, scripts, stylesheets, embedded frames) are checked as well as its normal links. ## `includeUrlPatterns` (type: `array`): Keep only addresses that match at least one entry. Use \* as a wildcard, for example https://example.com/blog/\*. Without a \* the entry matches when the address contains it. Leave empty to keep everything. ## `excludeUrlPatterns` (type: `array`): Drop addresses that match any entry, for example \*.pdf or /cart/. Same wildcard rules as above. ## `respectRobotsTxt` (type: `boolean`): When on, pages that a site's robots.txt asks crawlers to avoid are not opened to look for links. Checking the status of a link does not open the page and is not affected. ## `expectedStatus` (type: `array`): Report a problem when the final answer is not one of these. Accepts exact codes (200, 301), classes (2xx) and ranges (200-299). Leave empty to skip this rule. ## `mustContainText` (type: `array`): Report a problem when any of these texts is missing from the page. Not case sensitive. Turns on reading the page itself, so these checks are slower than status-only checks. ## `mustNotContainText` (type: `array`): Report a problem when any of these texts is found on the page, for example an error message or a leftover placeholder. Not case sensitive. ## `requiredHeaders` (type: `array`): Report a problem when a header is missing or lacks the text you give. Write the header name, or the name, a colon and the text it must contain. ## `warnAboveRedirects` (type: `integer`): Mark an address when it needs more redirects than this to reach its destination. Long chains slow pages down. Use 0 to switch the warning off. ## `slowThresholdMs` (type: `integer`): Mark an address as slow when the whole check, redirects included, takes longer than this. Use 0 to switch off. ## `maxResponseTimeMs` (type: `integer`): Treat an address as failing when it takes longer than this. Use 0 for no limit. ## `sslWarnDays` (type: `integer`): Mark secure addresses whose certificate expires within this many days. An expired or untrusted certificate is always reported. ## `detectSoft404` (type: `boolean`): Reads the page and flags those whose title or heading says the page was not found. Slower, because the page itself is opened. ## `extractPageBasics` (type: `boolean`): Adds the page title, description, main heading, canonical address, robots meta tag and an indexable flag. Slower, because the page itself is opened. ## `includeSecurityHeaders` (type: `boolean`): Adds the security headers each address sends (HSTS, CSP, frame, content type and referrer policy) and lists the ones that are missing. ## `includeResponseHeaders` (type: `boolean`): Adds every response header of the final answer to each row. Cookies are left out. ## `probeHttp2` (type: `boolean`): Opens one extra secure connection per website to see whether the server offers HTTP/2. ## `maxItems` (type: `integer`): The most addresses checked in one run. Defaults to 10, up to 10,000. ## `outputFilter` (type: `string`): Return every address, or only the ones that need attention. Choosing changes returns nothing on the very first run, because that run only saves the starting point. ## `trackHistory` (type: `boolean`): Remembers each address between runs so every row shows what changed, how long it has been broken and its uptime percentage. Turn off for a one-off check. ## `monitorScope` (type: `string`): Runs that share the same name are compared with each other. Use a different name for each list you monitor on a schedule. Letters, numbers and hyphens only. ## `responseRegressionPercent` (type: `integer`): Mark an address when it answers this much slower than its usual time over recent runs. 100 means twice as slow. Use 0 to switch off. ## `confirmChanges` (type: `boolean`): When an address looks different from last time, it is checked once more a moment later so a short blip is not reported as a change. ## `requestMethod` (type: `string`): HEAD is lightest, and falls back to a full request when the server rejects it or reports an error, so a wrong answer to HEAD is never reported. GET always makes a full request. HEAD never falls back. ## `followRedirects` (type: `boolean`): When on, redirects are followed and every step is recorded. When off, only the first answer is reported. ## `maxRedirects` (type: `integer`): Stops following after this many redirects and reports a redirect problem. ## `timeoutSecs` (type: `integer`): How long to wait for a server to answer before giving up. ## `retries` (type: `integer`): Extra attempts after a timeout, a dropped connection or a busy server (429, 502, 503, 504). ## `userAgentType` (type: `string`): The identity sent with each request. The default names this monitor honestly. Some sites turn such requests away, so you can send a common browser identity instead. ## `userAgent` (type: `string`): Optional. Your own identity text. When filled in, it replaces the choice above. ## `customHeaders` (type: `array`): Optional extra headers sent with every request, for example Accept-Language. Logins, cookies and tokens (Authorization, Cookie) are never sent. ## `maxConcurrency` (type: `integer`): How many addresses are checked at the same moment across all websites. ## `maxConcurrencyPerHost` (type: `integer`): The most requests in flight to any one website. Keep this low to be gentle on small sites. ## `delayPerHostMs` (type: `integer`): A short wait before each new request to the same website. ## `proxyConfiguration` (type: `object`): Optional. Send requests through Apify Proxy to see how a website answers from another country. Leave off to check from the default location. ## Actor input object example ```json { "startUrls": [ { "url": "http://github.com" }, { "url": "https://example.com" }, { "url": "https://example.com/this-page-does-not-exist" }, { "url": "https://www.iana.org/" }, { "url": "https://www.wikipedia.org/" } ], "sourceMode": "urls", "redirectExpectations": [], "crawlDepth": 1, "maxPagesToCrawl": 10, "linkScope": "all", "includeAssets": false, "includeUrlPatterns": [], "excludeUrlPatterns": [], "respectRobotsTxt": true, "expectedStatus": [], "mustContainText": [], "mustNotContainText": [], "requiredHeaders": [], "warnAboveRedirects": 2, "slowThresholdMs": 3000, "maxResponseTimeMs": 0, "sslWarnDays": 30, "detectSoft404": false, "extractPageBasics": false, "includeSecurityHeaders": false, "includeResponseHeaders": false, "probeHttp2": false, "maxItems": 10, "outputFilter": "all", "trackHistory": true, "monitorScope": "default", "responseRegressionPercent": 100, "confirmChanges": true, "requestMethod": "HEAD_FALLBACK", "followRedirects": true, "maxRedirects": 10, "timeoutSecs": 20, "retries": 1, "userAgentType": "monitor", "userAgent": "", "customHeaders": [], "maxConcurrency": 10, "maxConcurrencyPerHost": 2, "delayPerHostMs": 250, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `results` (type: `string`): No description ## `summary` (type: `string`): No description ## `changes` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "http://github.com" }, { "url": "https://example.com" }, { "url": "https://example.com/this-page-does-not-exist" }, { "url": "https://www.iana.org/" }, { "url": "https://www.wikipedia.org/" } ], "sourceMode": "urls", "redirectExpectations": [], "crawlDepth": 1, "maxPagesToCrawl": 10, "linkScope": "all", "includeAssets": false, "includeUrlPatterns": [], "excludeUrlPatterns": [], "respectRobotsTxt": true, "expectedStatus": [], "mustContainText": [], "mustNotContainText": [], "requiredHeaders": [], "warnAboveRedirects": 2, "slowThresholdMs": 3000, "maxResponseTimeMs": 0, "sslWarnDays": 30, "detectSoft404": false, "extractPageBasics": false, "includeSecurityHeaders": false, "includeResponseHeaders": false, "probeHttp2": false, "maxItems": 10, "outputFilter": "all", "trackHistory": true, "monitorScope": "default", "responseRegressionPercent": 100, "confirmChanges": true, "requestMethod": "HEAD_FALLBACK", "followRedirects": true, "maxRedirects": 10, "timeoutSecs": 20, "retries": 1, "userAgentType": "monitor", "userAgent": "", "customHeaders": [], "maxConcurrency": 10, "maxConcurrencyPerHost": 2, "delayPerHostMs": 250, "proxyConfiguration": { "useApifyProxy": false } }; // Run the Actor and wait for it to finish const run = await client.actor("getascraper/http-status-intelligence-monitor").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [ { "url": "http://github.com" }, { "url": "https://example.com" }, { "url": "https://example.com/this-page-does-not-exist" }, { "url": "https://www.iana.org/" }, { "url": "https://www.wikipedia.org/" }, ], "sourceMode": "urls", "redirectExpectations": [], "crawlDepth": 1, "maxPagesToCrawl": 10, "linkScope": "all", "includeAssets": False, "includeUrlPatterns": [], "excludeUrlPatterns": [], "respectRobotsTxt": True, "expectedStatus": [], "mustContainText": [], "mustNotContainText": [], "requiredHeaders": [], "warnAboveRedirects": 2, "slowThresholdMs": 3000, "maxResponseTimeMs": 0, "sslWarnDays": 30, "detectSoft404": False, "extractPageBasics": False, "includeSecurityHeaders": False, "includeResponseHeaders": False, "probeHttp2": False, "maxItems": 10, "outputFilter": "all", "trackHistory": True, "monitorScope": "default", "responseRegressionPercent": 100, "confirmChanges": True, "requestMethod": "HEAD_FALLBACK", "followRedirects": True, "maxRedirects": 10, "timeoutSecs": 20, "retries": 1, "userAgentType": "monitor", "userAgent": "", "customHeaders": [], "maxConcurrency": 10, "maxConcurrencyPerHost": 2, "delayPerHostMs": 250, "proxyConfiguration": { "useApifyProxy": False }, } # Run the Actor and wait for it to finish run = client.actor("getascraper/http-status-intelligence-monitor").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "http://github.com" }, { "url": "https://example.com" }, { "url": "https://example.com/this-page-does-not-exist" }, { "url": "https://www.iana.org/" }, { "url": "https://www.wikipedia.org/" } ], "sourceMode": "urls", "redirectExpectations": [], "crawlDepth": 1, "maxPagesToCrawl": 10, "linkScope": "all", "includeAssets": false, "includeUrlPatterns": [], "excludeUrlPatterns": [], "respectRobotsTxt": true, "expectedStatus": [], "mustContainText": [], "mustNotContainText": [], "requiredHeaders": [], "warnAboveRedirects": 2, "slowThresholdMs": 3000, "maxResponseTimeMs": 0, "sslWarnDays": 30, "detectSoft404": false, "extractPageBasics": false, "includeSecurityHeaders": false, "includeResponseHeaders": false, "probeHttp2": false, "maxItems": 10, "outputFilter": "all", "trackHistory": true, "monitorScope": "default", "responseRegressionPercent": 100, "confirmChanges": true, "requestMethod": "HEAD_FALLBACK", "followRedirects": true, "maxRedirects": 10, "timeoutSecs": 20, "retries": 1, "userAgentType": "monitor", "userAgent": "", "customHeaders": [], "maxConcurrency": 10, "maxConcurrencyPerHost": 2, "delayPerHostMs": 250, "proxyConfiguration": { "useApifyProxy": false } }' | apify call getascraper/http-status-intelligence-monitor --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,getascraper/http-status-intelligence-monitor" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/SbrF5pFKLUjPaq6Bb/builds/NWX9LDrZkA9gbFDxf/openapi.json