Domain Security Audit (TLS, HTTP headers, redirects, robots) avatar

Domain Security Audit (TLS, HTTP headers, redirects, robots)

Pricing

from $3.00 / 1,000 results

Try for free
Go to Apify Store
Domain Security Audit (TLS, HTTP headers, redirects, robots)

Domain Security Audit (TLS, HTTP headers, redirects, robots)

Try for free

Domain Security Audit checks TLS certificate validity, HSTS/CSP and other HTTP security headers, the redirect chain and robots.txt/llms.txt AI-bot rules for any list of domains — one scored, graded row per domain.

Pricing

from $3.00 / 1,000 results

Rating

0.0

(0)

Developer

Murat Uzun

Murat Uzun

Maintained by Community

Actor stats

0

Bookmarked

1

Total users

0

Monthly active users

2 days ago

Last modified

Categories

Developer tools

SEO tools

Business

Share

What is Domain Security Audit?

Domain Security Audit is an Apify Actor that checks a TLS certificate, the HTTP → HTTPS redirect chain, HTTP security headers, cookie flags, and robots.txt / llms.txt AI-crawler rules for any list of domains, and returns one scored, graded row per domain. TLS comes straight from a node:tls handshake on port 443; everything else is a plain fetch — no proxies, no headless browser, no anti-bot wall. Each row gets a securityScore (0-100), a securityGrade (A-F) and an issues array in plain English ("No HSTS header", "TLS certificate expires in 9 day(s)") that drops straight into a client report or a sales-prospecting sheet.

Why use Domain Security Audit?

  • Security consultants and MSPs — score a prospect list in minutes instead of running curl -I and openssl s_client on each site by hand.
  • Sales and lead-scoring teams — a low securityGrade or an expiring certificate is a concrete opener for an outbound e-mail.
  • SEO and content teams — aiBotsDisallowed and llmsTxtExists show whether a site blocks GPTBot/ClaudeBot/Google-Extended or publishes an llms.txt, which matters for AI-search visibility.
  • Internal audits — run it on a schedule against your own domain portfolio to catch a certificate about to expire or a header that regressed after a deploy.

How to use Domain Security Audit

  1. Paste your domains into Domains. Bare domains, full URLs and www. prefixes are all accepted and normalised: https://www.apify.com/store becomes apify.com.
  2. Leave Check TLS certificate, Check HTTP security headers, Follow redirect chain and Check robots.txt and llms.txt all on for the full audit; turn any of them off to skip that section and save a couple of seconds per domain.
  3. Raise Max concurrency for large lists, click Start, then export the dataset as JSON, CSV, Excel or HTML from the Overview, TLS certificate or Headers, redirects & robots views.

Input

ParameterTypeDefaultDescription
domainsarray["apify.com"]Domains to audit, one row each
checkTlsbooleantrueConnect on port 443 and read the peer certificate
checkHeadersbooleantrueRead HSTS, CSP, X-Frame-Options and other headers on the final URL
followRedirectsbooleantrueFollow the http:// → final-URL redirect chain (up to 10 hops)
checkRobotsbooleantrueFetch /robots.txt (AI-bot rules) and /llms.txt
maxConcurrencyinteger5Domains audited in parallel (1-20)

Output

Domain Security Audit extracts 40+ fields per domain across four areas. You can download the dataset in various formats such as JSON, HTML, CSV or Excel.

Field groupExample fieldsDescription
TLS certificatecertValidFrom, certValidTo, certDaysUntilExpiry, certIssuerOrg, certIssuerCN, certSubjectCN, certSanCount, certAuthorized, tlsProtocolPeer certificate read from a raw node:tls handshake, chain-validity flag and negotiated TLS version
RedirectsfinalUrl, redirectHops, redirectChain, httpsRedirect, wwwRedirectThe hop-by-hop path from http://<domain>/ to the final URL
HTTP headershasHsts, hstsMaxAge, hstsIncludeSubdomains, hstsPreload, hasCsp, cspHasUnsafeInline, xFrameOptions, xContentTypeOptions, referrerPolicy, permissionsPolicy, server, xPoweredBy, cacheControl, cookieCount, secureCookieCount, httpOnlyCookieCount, sameSiteCookieCountSecurity headers and cookie flags on the final response
robots.txt / llms.txtrobotsStatus, robotsDisallowAll, robotsSitemapCount, aiBotsDisallowed, llmsTxtExists, llmsTxtTitle, llmsTxtSizeWhich AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot…) are blocked, and whether an llms.txt exists
ScoresecurityScore, securityGrade, issues0-100 score, A-F grade and a plain-English findings list
Metadomain, error, scrapedAtNormalised hostname, failure reason if the domain was unreachable, and the audit timestamp

Example input

{
    "domains": ["apify.com", "example.com", "neverssl.com"],
    "checkTls": true,
    "checkHeaders": true,
    "followRedirects": true,
    "checkRobots": true,
    "maxConcurrency": 5
}

Example output

{
    "domain": "apify.com",
    "certAuthorized": true,
    "certDaysUntilExpiry": 62,
    "certIssuerOrg": "Let's Encrypt",
    "tlsProtocol": "TLSv1.3",
    "finalUrl": "https://apify.com/",
    "redirectHops": 1,
    "httpsRedirect": true,
    "hasHsts": true,
    "hstsMaxAge": 63072000,
    "hasCsp": false,
    "xFrameOptions": "SAMEORIGIN",
    "robotsDisallowAll": false,
    "aiBotsDisallowed": [],
    "llmsTxtExists": false,
    "securityScore": 75,
    "securityGrade": "B",
    "issues": ["No Content-Security-Policy header"],
    "scrapedAt": "2026-09-12T15:45:00.000Z"
}

Pricing

Domain Security Audit uses pay-per-event pricing: $0.005 per domain result, i.e. $5 per 1,000 domains, plus a negligible actor-start fee, platform usage included. Each domain is one TLS handshake, up to 10 HTTP redirect hops, one headers fetch and two text fetches (robots.txt, llms.txt), so compute cost stays in the cents even for a few hundred domains. Set Maximum cost per run and the Actor trims the domain list to what the budget covers instead of overspending.

Domain Security Audit vs. manual curl/openssl checks

Checking one domain's TLS expiry, headers and robots.txt by hand means openssl s_client, curl -I and a manual read of /robots.txt — three tools per domain with no structured output. Domain Security Audit runs all of that for an entire list in parallel and returns one flat, scored dataset row per domain, ready to filter, sort or pipe into a CRM.

Using Domain Security Audit with AI agents and MCP

Domain Security Audit is pay-per-event with limited permissions — the two requirements for an Actor to be callable through the Apify MCP server at mcp.apify.com. An agent passes domains and gets back a scored, graded row per domain it can reason over directly ("which of these ten prospects have the weakest header hygiene?"). The same run works from n8n, Make, Zapier and LangChain through Apify's integrations.

FAQ

How is the score calculated? HTTPS redirect +20, valid/trusted certificate +20 (certificate has more than 14 days left +5), HSTS +15, CSP +10, X-Frame-Options +5, X-Content-Type-Options +5, Referrer-Policy +5, Permissions-Policy +5, no X-Powered-By leak +5, all cookies carry the Secure flag +5. Grades: A ≥ 85, B ≥ 70, C ≥ 50, D ≥ 30, else F.

Why does neverssl.com score low? It intentionally serves plain HTTP with no TLS certificate at all (it exists to test captive portals), so it loses every TLS- and HTTPS-redirect point.

What happens with a domain that doesn't resolve? The row still comes back — with error set to the failure reason and every other field null — instead of failing the whole run.

Is this legal to run? Yes. TLS certificates, HTTP response headers, robots.txt and llms.txt are all public information a server sends to anyone who connects; no personal data is collected.

Can I export to CSV or Excel? Yes, from the Output tab or the API, with ready-made Overview, TLS certificate and Headers, redirects & robots views.

Part of the webdatatools web-intelligence suite — every Actor is pay-per-event, reads public data without a login, and returns one clean row per entity:

Browse the whole suite at webdatatools, or call ten of these Actors straight from Claude, Cursor or Cline with the webdatatools MCP server.

Website & domain intelligence

Content for AI, LLMs and RAG

Search, video and social

Leads, jobs and company data

Developer, app and research data

Support and feedback

Found a header worth tracking, an AI crawler that should be added to the robots.txt check, or a parsing bug? Open an issue on the Issues tab.

You might also like

HTTP & Security Headers Auditor avatar

HTTP & Security Headers Auditor

bgfc97/http-security-headers-audit

Audit any list of URLs: status, final redirect, HTTPS, server, security headers (HSTS, CSP, X-Frame-Options...), a security score, page title and CMS/generator. No key, no proxy.

User avatar

Bruno

3

Redirect Contract Verifier avatar

Redirect Contract Verifier

plutonus/redirect-contract-verifier

Check whether a URL redirects to the destination you expect. Get a deterministic PASS/WARN/FAIL result with the final URL, HTTP status, redirect chain, and reason code.

User avatar

Plutonus

3

On-Page SEO Audit avatar

On-Page SEO Audit

webdatatools/seo-page-audit

On-Page SEO Audit checks title, meta description, headings, images, links, Open Graph, Twitter Card and Schema.org data for any list of URLs — one scored row per page.

User avatar

Murat Uzun

3

Website Security Headers & HTTPS Audit avatar

Website Security Headers & HTTPS Audit

moonweil/website-security-audit

Crawl your website for missing or weak security headers (HSTS, CSP, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, frame protection), insecure cookies, mixed content, and HTTP-to-HTTPS gaps. Passive and safe: only reads responses, never probes. Per-page score plus site summary.

User avatar

Aleksandr Jelohhin

2

Domain Security Auditor — DMARC, SPF, DKIM, TLS & DNS in bulk avatar

Domain Security Auditor — DMARC, SPF, DKIM, TLS & DNS in bulk

datadeltas/domain-security-auditor

Audit any list of domains for email spoofing risk and web security: SPF (with real lookup counting), DKIM, DMARC policy, MTA-STS, TLS expiry, DNSSEC, CAA and security headers — scored, graded, with the fix for every finding.

User avatar

Daniel Matthee

2

Domain Security Checker avatar

Domain Security Checker

automation-lab/bulk-domain-security-posture-checker

Audit public domains across DNS, SPF, optional DKIM, DMARC, TLS, HTTPS, and security headers with severity-tagged remediation findings.

User avatar

Automation Lab

2

SH

Security Headers Grader — one row per domain

artifact-machine/security-headers-enrichment

Grades the security headers of every domain in a list: HSTS, CSP, X-Frame-Options and more, plus TLS. No browser needed, so it runs fast and cheap over large lists.

User avatar

The Artifact Machine

2

BO

Bulk On-Page SEO Audit - 26 Checks, Meta Robots, Hreflang

smiling_jubilance/website-seo-tech-audit

On-page SEO audit and technical SEO audit at scale: meta robots, canonical tag, hreflang, XML sitemap, redirect chain, heading hierarchy, image alt and page speed. 0-100 score, severity-ranked issues, 235-signature tech detection. Pure HTTP, no browser.

User avatar

Lake Sky

2

Domain Expiration, DNS & SSL Intelligence Auditor avatar

Domain Expiration, DNS & SSL Intelligence Auditor

unrivaled_fortress/domain-whois-expiration-dns-intel

Audit domain age, expiration dates, DNS records (A, MX, NS), and SSL certificates. Built for domain investors, SEO drop-catchers, and cybersecurity analysts.

User avatar

David A

3

DA

Domain Attack Surface & Subdomain Scanner

cypherstone/domain-attack-surface-scanner

Find a domain's subdomains from Certificate Transparency logs, resolve each to its IP, and check the live SSL certificate (issuer, expiry, days left). Open-source recon, no API key.

User avatar

Jeff Ralston

2