# Bulk URL Status Checker: Redirects, SSL Expiry, Metadata (`charlesmarkette/bulk-url-status-checker-redirects-ssl`) Actor Check thousands of URLs for broken links, status codes, full redirect chains, loops and response time, plus SSL certificate expiry and security headers. Optionally grab title, meta description, canonical and Open Graph tags. Enrich any dataset of links. - **URL**: https://apify.com/charlesmarkette/bulk-url-status-checker-redirects-ssl.md - **Developed by:** [Charles Markette](https://apify.com/charlesmarkette) (community) - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.38 / 1,000 checked urls This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README **Bulk URL Status Checker** checks thousands of URLs and returns, for each one: the **final HTTP status code**, the full **redirect chain** (every hop with its status and `Location`), **redirect loops**, response time, content type and server, plus the **SSL certificate** of the host: issuer, expiry date, **days until expiry**, and whether it's trusted and valid for that host name. Optionally it also reports **security headers** (HSTS, CSP and four more, with a score out of 6) and **page metadata**: title, meta description, canonical, robots meta, lang, first H1, Open Graph and Twitter card tags, favicon and JSON-LD types. Paste URLs, or point it at any Apify dataset of links and the columns are added next to each item. ### Why this Actor - **Every redirect hop is recorded.** Redirects are followed one by one, so you see `http → https → www → final page`, with the status code of each hop, not just where it ended. Loops and over-long chains are flagged instead of hanging. - **Dead links are reported, not dropped.** DNS failures, timeouts, refused connections and TLS errors each get a plain-English `error` and a row of their own. - **One certificate check per host.** 5,000 URLs on one site need one certificate check for the SSL columns; the result is reused. - **Polite.** At most 2 requests at a time to any one host by default, a few retries with backoff on 429/502/503/504 (honouring `Retry-After`), and a User-Agent that names the tool. - **Guards against risky link lists.** Every host is resolved first and private, loopback, link-local, multicast, reserved and cloud-metadata addresses are refused, on the first URL and on every redirect hop. ### Use cases - **SEO audits:** find 404s, redirect chains longer than one hop, redirect loops, and pages whose canonical or robots meta isn't what you expect. - **Site migrations:** feed in the old URLs and check that each one 301s to the right new page (`finalUrl`, `redirectChain`). - **Cleaning scraped link lists:** check every website a scraper collected and keep only the ones that answer 2xx. - **SSL expiry monitoring:** schedule a weekly run over your domains and alert on `sslDaysUntilExpiry` below 21 or `sslValid: false`. - **Security header review:** see which of your sites are missing HSTS, CSP or X-Frame-Options. - **Link previews:** collect title, description and `og:image` for a list of URLs. ### How to use it 1. Add **URLs** (full URLs or bare domains such as `example.com`, which become `https://example.com/`), and/or pick a **Dataset to enrich**. For datasets, set **URL fields** if yours aren't named url, website or link. 2. Choose what to check. SSL is on by default; turn on **security headers** or **page metadata** if you need them. 3. Run it. Each URL becomes one output row. Export as CSV, Excel or JSON, or chain it with an integration. **Input example** ```json { "urls": ["https://apify.com", "http://wikipedia.org", "example.com/old-page"], "method": "auto", "followRedirects": true, "maxRedirects": 10, "timeoutSecs": 15, "checkSsl": true, "checkSecurityHeaders": true, "extractMetadata": true } ``` **Output example** (shortened; values are illustrative) ```json { "inputUrl": "http://wikipedia.org", "url": "http://wikipedia.org/", "finalUrl": "https://www.wikipedia.org/", "statusCode": 200, "ok": true, "statusText": "OK", "redirectCount": 2, "redirectChain": [ {"url": "http://wikipedia.org/", "statusCode": 301, "location": "https://wikipedia.org/"}, {"url": "https://wikipedia.org/", "statusCode": 301, "location": "https://www.wikipedia.org/"} ], "isRedirectLoop": false, "method": "GET", "responseTimeMs": 182, "contentType": "text/html", "contentLength": 98213, "server": "ATS/9.2.11", "lastModified": "2026-10-14T09:12:40Z", "ipAddress": "185.15.59.224", "sslIssuer": "DigiCert Inc", "sslIssuerCommonName": "DigiCert TLS Hybrid ECC SHA384 2020 CA1", "sslSubject": "*.wikipedia.org", "sslExpiresAt": "2027-03-12T23:59:59Z", "sslDaysUntilExpiry": 148, "sslValidForHost": true, "sslValid": true, "sslError": null, "securityHeadersScore": "3/6", "securityHeadersMissing": ["content-security-policy", "referrer-policy", "permissions-policy"], "title": "Wikipedia", "metaDescription": "Wikipedia is a free online encyclopedia…", "canonical": null, "lang": "en", "ogImage": "https://www.wikipedia.org/portal/wikipedia.org/assets/img/Wikipedia-logo-v2.png", "jsonLdTypes": [], "error": null, "checkedAt": "2026-10-15T10:31:07Z" } ``` **All columns** - **Request:** `inputUrl`, `url` (normalised), `finalUrl`, `statusCode` (of the final response), `ok` (2xx), `statusText`, `redirectCount`, `redirectChain` (`[{url, statusCode, location}]`), `isRedirectLoop`, `redirectLocation` (only when *Follow redirects* is off and the answer is a redirect), `method` (HEAD or GET, whichever gave the final answer), `responseTimeMs`, `contentType`, `contentLength`, `server`, `lastModified`, `ipAddress`, `error`, `checkedAt`. - **SSL** (https only): `sslIssuer` (organisation, or common name if there's none), `sslIssuerCommonName`, `sslSubject`, `sslSans`, `sslValidFrom`, `sslExpiresAt`, `sslDaysUntilExpiry`, `sslValidForHost`, `sslValid`, `sslError`. - **Security headers** (when on): `hsts`, `contentSecurityPolicy`, `xFrameOptions`, `xContentTypeOptions`, `referrerPolicy`, `permissionsPolicy` (the header value, or null if missing), `securityHeadersPresent`, `securityHeadersScore` ("x/6"), `securityHeadersMissing`. - **Metadata** (when on): `title`, `metaDescription`, `canonical`, `robotsMeta`, `lang`, `h1`, `ogTitle`, `ogDescription`, `ogImage`, `ogType`, `twitterCard`, `favicon`, `jsonLdTypes`. Relative links are made absolute. `error` values include `DNS lookup failed`, `Timed out`, `Connection refused`, `TLS error: `, `Redirect loop`, `Too many redirects (more than N)`, `Blocked: private address` and `Invalid URL: `. A run summary (counts of 2xx, 4xx, 5xx, unreachable, invalid SSL, certificates expiring within 30 days…; the 30-day summary count is fixed, while your own alerts can use any threshold, such as the 21 days in the use case above) is saved as `OUTPUT` in the key-value store. ### How much does it cost? Pay per event: **$0.0025 per checked URL** (one output row), plus **$0.005 per run start**. Apify platform usage is included. **Apify Store discounts:** the price above is for Apify's Free plan. Paid plans pay less per 1,000 checked URLs: $2.18 on Starter, $1.75 on Scale, $1.38 on Business. The "from" price at the top of the Store page is the Business price. A dead link is a result, so URLs that return 404 or 500, or fail with a DNS error, timeout or refused connection, **are charged**. These are **free**: rows for input that isn't a valid http(s) URL, URLs refused by the private-address guard, and repeats skipped by *Check each URL once*. **Worked examples:** *Examples use the Free-plan rate. Where an example doesn't add it, each run also has its run-start charge, counted per GB of Actor memory (minimum one; one at the default memory); Apify Proxy (where used) is billed separately.* - 1,000 URLs: 1,000 × $0.0025 + $0.005 = **$2.505**. - A weekly SSL and status check of 100 URLs: $0.255 per run, about **$1.10 per month**. If you set a maximum cost per run, the Actor stops cleanly at that limit. If a run is restarted (for example after a timeout or migration), it continues after the rows already in its dataset. ### FAQ and limits #### Does it render JavaScript? No. It makes plain HTTP requests, so JavaScript redirects, `` redirects and "soft 404s" (an error page served with status 200) aren't detected, and metadata is what the server's HTML contains before any script runs. #### A site returns 403 or 429 but works in my browser. Some sites block automated requests or requests from data-centre IP addresses (Cloudflare and similar bot protection). The Actor doesn't use proxies or solve challenges, so it reports the status it actually got. You can set a different User-Agent, but that won't get past bot protection. #### What does "auto" method do? It sends HEAD first (fast, no body). If the server answers HEAD with 403, 405 or 501, or drops the connection, it retries that URL with GET. Turning on page metadata uses GET for every request, because the HTML is needed. #### Why is a URL with a bad certificate shown with no status code? The HTTP request verifies the certificate like a browser does, so an expired, self-signed or untrusted certificate gives `error: "TLS error: …"`. The SSL columns still show the certificate details and the reason in `sslError`. Trust is checked against Mozilla's CA list (certifi). For URLs that end on plain http, the last https URL in the chain is the one whose certificate is checked. #### Can I check internal or localhost URLs? No. Private, loopback, link-local and cloud-metadata addresses are always refused (free rows with `Blocked: private address`), including when a public URL redirects to one. #### Other limits - Page metadata reads at most 2 MB of HTML per page; `favicon` is the icon the page declares (it isn't fetched or verified). - Cookies are kept within one URL's redirect chain only (some sites set a cookie and redirect), never shared between URLs. - Each request (each redirect hop) waits at most *Timeout per request*; a whole URL, with all its hops, retries and the HEAD→GET fallback, gets at most 4× that (at least 60 seconds) before it's reported as `Timed out`. `Retry-After` waits are capped at 10 seconds. - Page metadata asks for gzip or deflate only; a page sent with another compression (such as Brotli) gets empty metadata columns. - HTTP/1.1 only. Response times are measured from Apify's servers, not your location. ### Other tools by the same author | Actor | Use it to | |---|---| | [HTTP Request & API Calls](https://apify.com/charlesmarkette/http-request-api-calls-bulk-pagination) | Call an API for each URL (GET or POST, headers, JSON body) and keep the full JSON response. | | [PageSpeed & Lighthouse Audit](https://apify.com/charlesmarkette/pagespeed-lighthouse-audit-core-web-vitals) | Get Lighthouse scores and Core Web Vitals for the URLs that load. | | [Sitemap URL Extractor & New Page Monitor](https://apify.com/charlesmarkette/sitemap-url-extractor-new-page-monitor) | Get the full URL list of a site from its sitemaps, then check them all here. | | [Domain WHOIS & DNS Lookup](https://apify.com/charlesmarkette/domain-whois-dns-lookup-bulk-rdap) | Check domain registration and expiry dates next to SSL expiry. | | [Merge & Deduplicate Datasets](https://apify.com/charlesmarkette/merge-deduplicate-datasets-only-new-items) | Keep only links you haven't checked before, between runs. | | [Notify Step](https://apify.com/charlesmarkette/send-email-report-slack-discord-teams-telegram-alerts) | Email or Slack a report when a scheduled check finds broken links or expiring certificates. | | [Google Sheets Import & Export](https://apify.com/charlesmarkette/google-sheets-sync) | Send the results to a Google Sheet. | | [Dataset Delivery](https://apify.com/charlesmarkette/dataset-delivery-export-to-sftp-ftp-dropbox-google-drive) | Deliver the results as CSV or Excel to SFTP, Dropbox or Google Drive. | | [Broken Link Checker](https://apify.com/charlesmarkette/broken-link-checker-crawl-website-404) | Crawl a whole website and find every broken link (404s, dead external links, redirects) with the page it's on. | | [Website Tech Stack Detector](https://apify.com/charlesmarkette/website-tech-stack-detector-shopify-cms-analytics) | Detect the platform, CMS and tags behind each live URL. | # Actor input Schema ## `urls` (type: `array`): One per line. Full URLs (https://example.com/page) or bare domains (example.com, which become https://example.com/). ## `datasetId` (type: `string`): Optional. A dataset from another run (scraped links, sitemap URLs, company websites…). For each item, the first field below that holds a URL is checked and the result columns are added next to the item's fields. Use {{resource.defaultDatasetId}} in an integration. ## `urlFields` (type: `array`): Item fields to take the URL from, in order of preference. Dotted paths work (company.website). Default: url, website, link. ## `keepItemFields` (type: `boolean`): On: each output row is the original item plus the check columns (check columns win if a name clashes). Off: only the check columns. ## `method` (type: `string`): Auto: a HEAD request first, then GET if the server answers HEAD with 403, 405 or 501 or drops the connection. HEAD only or GET only are also available. Page metadata always uses GET. ## `followRedirects` (type: `boolean`): Follow 301/302/303/307/308 redirects and record every hop in redirectChain. Off: the first response is the result and its Location goes in redirectLocation. ## `maxRedirects` (type: `integer`): Stop with 'Too many redirects' after this many hops (0–30). ## `timeoutSecs` (type: `integer`): How long to wait for each request (each redirect hop) before reporting 'Timed out'. A whole URL, with all its redirect hops and retries, is given at most 4 times this (at least 60 seconds). ## `userAgent` (type: `string`): Optional. By default the requests identify this tool: Mozilla/5.0 (compatible; BulkUrlStatusChecker/1.0; +https://apify.com/charlesmarkette). ## `checkSsl` (type: `boolean`): For https URLs: issuer, subject, SANs, valid-from and expiry dates, days until expiry, whether it's trusted and valid for the host. One check per host, reused for every URL on it. ## `checkSecurityHeaders` (type: `boolean`): Value of HSTS, Content-Security-Policy, X-Frame-Options, X-Content-Type-Options, Referrer-Policy and Permissions-Policy on the final response, and a score of how many of the six are present. ## `extractMetadata` (type: `boolean`): Download the final HTML page (up to 2 MB) and extract title, meta description, canonical, robots meta, lang, first H1, Open Graph and Twitter card tags, favicon and JSON-LD types. Uses GET for every request. ## `dedupe` (type: `boolean`): Skip repeats of a URL already checked in this run (compared after normalising, e.g. EXAMPLE.com and https://example.com/ are the same). Skipped repeats are free and not output. ## `concurrency` (type: `integer`): How many URLs to check at the same time (1–50). ## `perHostConcurrency` (type: `integer`): At most this many requests at a time to any one host name, to stay polite to the sites you check (1–10). ## `maxResults` (type: `integer`): Optional cap on output rows (each row is one URL). ## Actor input object example ```json { "urls": [ "https://apify.com", "http://wikipedia.org", "https://example.com/this-page-does-not-exist" ], "keepItemFields": true, "method": "auto", "followRedirects": true, "maxRedirects": 10, "timeoutSecs": 15, "checkSsl": true, "checkSecurityHeaders": false, "extractMetadata": false, "dedupe": true, "concurrency": 10, "perHostConcurrency": 2 } ``` # Actor output Schema ## `items` (type: `string`): No description ## `summary` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "urls": [ "https://apify.com", "http://wikipedia.org", "https://example.com/this-page-does-not-exist" ] }; // Run the Actor and wait for it to finish const run = await client.actor("charlesmarkette/bulk-url-status-checker-redirects-ssl").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "urls": [ "https://apify.com", "http://wikipedia.org", "https://example.com/this-page-does-not-exist", ] } # Run the Actor and wait for it to finish run = client.actor("charlesmarkette/bulk-url-status-checker-redirects-ssl").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "urls": [ "https://apify.com", "http://wikipedia.org", "https://example.com/this-page-does-not-exist" ] }' | apify call charlesmarkette/bulk-url-status-checker-redirects-ssl --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,charlesmarkette/bulk-url-status-checker-redirects-ssl" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/kJl6GDwOl54mQea1x/builds/qqaDst5NSWUmaLUTW/openapi.json