HTTP Header Inspector: Audit HTTP Response Headers for SEO and Security

HTTP Header Inspector fetches response headers from any URL and shows what the server actually sends back. Give it a URL (or a list) and you get every header, a security score based on which recommended headers are missing, and the full redirect chain with status codes at each hop. No curl, no shell scripts, no manual checking.

Use cases

• SEO auditing: check content-type, cache-control, and redirect chains across pages that may be dragging down crawlability or load times
• Security auditing: find absent headers like CSP, HSTS, X-Frame-Options, and Referrer-Policy before they show up in a security review
• Redirect monitoring: trace redirect chains and catch double redirects or http-to-https loops that eat crawl budget
• Cache configuration review: inspect Cache-Control, ETag, and Last-Modified to understand how pages are being cached by browsers and CDNs
• Server fingerprint checks: see what server software and version headers are exposed in responses

What data does this actor extract?

Each URL produces one result:

{
    "inputUrl": "https://apify.com",
    "url": "https://apify.com/",
    "statusCode": 200,
    "statusText": "OK",
    "redirectChain": [],
    "redirectCount": 0,
    "contentType": "text/html; charset=utf-8",
    "server": "nginx",
    "cacheControl": "max-age=3600, public",
    "etag": "\"abc123\"",
    "lastModified": "Mon, 01 Jan 2024 00:00:00 GMT",
    "contentSecurityPolicy": "default-src 'self'",
    "strictTransportSecurity": "max-age=31536000; includeSubDomains",
    "xFrameOptions": "DENY",
    "xContentTypeOptions": "nosniff",
    "referrerPolicy": "strict-origin-when-cross-origin",
    "permissionsPolicy": "camera=(), microphone=()",
    "xXssProtection": "1; mode=block",
    "accessControlAllowOrigin": "",
    "allHeaders": { "content-type": "text/html; charset=utf-8" },
    "securityScore": 80,
    "missingSecurityHeaders": ["cross-origin-opener-policy", "cross-origin-resource-policy"],
    "checkedAt": "2024-06-01T12:00:00+00:00"
}

Input

Parameter	Type	Default	Description
url	string		Single URL to inspect.
urls	array		List of URLs to inspect in batch.
maxUrls	integer	100	Cap on URLs processed per run.
requestTimeoutSecs	integer	30	Per-request timeout in seconds.
timeoutSecs	integer	300	Overall actor timeout in seconds.
proxyConfiguration	object	Datacenter (Anywhere)	Proxy type and location for requests. Supports Datacenter, Residential, Special, and custom proxies. Optional.

Example input

{
    "urls": ["https://apify.com", "https://apify.com/store"],
    "maxUrls": 50,
    "requestTimeoutSecs": 30,
    "proxyConfiguration": { "useApifyProxy": true }
}

Output

The actor stores one result per URL in a dataset.

Field	Type	Description
inputUrl	string	Original URL as provided in the input.
url	string	Final URL after following all redirects.
statusCode	integer	HTTP status code returned by the server.
statusText	string	HTTP status phrase (e.g., OK, Not Found).
redirectChain	array	Each redirect hop, with url and statusCode.
redirectCount	integer	Number of redirects followed.
contentType	string	Content-Type header value.
contentEncoding	string	Content-Encoding header (e.g., gzip).
server	string	Server header value.
cacheControl	string	Cache-Control directive.
etag	string	ETag header value.
lastModified	string	Last-Modified header value.
contentSecurityPolicy	string	Content-Security-Policy header.
strictTransportSecurity	string	Strict-Transport-Security (HSTS) header.
xFrameOptions	string	X-Frame-Options header.
xContentTypeOptions	string	X-Content-Type-Options header.
referrerPolicy	string	Referrer-Policy header.
permissionsPolicy	string	Permissions-Policy header.
xXssProtection	string	X-XSS-Protection header.
accessControlAllowOrigin	string	Access-Control-Allow-Origin (CORS) header.
allHeaders	object	All response headers as key-value pairs.
securityScore	integer	Score 0-100: percentage of recommended security headers present.
missingSecurityHeaders	array	Security headers absent from the response.
checkedAt	string	ISO 8601 timestamp of when the check ran.
error	string	Error message if the request failed at the network level.

How it works

1. Accepts one URL or a list of URLs as input
2. Sends an HTTP GET request to each URL with a realistic browser User-Agent
3. Follows redirects up to 10 hops, recording each hop's URL and status code
4. Extracts all response headers from the final response
5. Scores the security headers and lists which recommended ones are absent
6. Pushes one result per URL to the dataset

Integrations

Connect HTTP Header Inspector with other tools using Apify integrations. Export results to Google Sheets, trigger Make or Zapier workflows, or use webhooks to alert your team when runs complete.

FAQ

What security headers does the actor check for?

It checks for: Content-Security-Policy, Strict-Transport-Security (HSTS), X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, X-XSS-Protection, Cross-Origin-Opener-Policy, Cross-Origin-Resource-Policy, and Cross-Origin-Embedder-Policy. The securityScore shows what percentage of these are present.

Can I check URLs that return 404 or 500 responses?

Yes. The actor captures headers for any HTTP response, including error codes. Only network-level failures (connection refused, DNS errors, timeouts) produce an error field instead of headers.

How many URLs can I check per run?

The default is 100. Raise the limit using the maxUrls input, up to 1000.

Does it follow redirects?

Yes, up to 10 hops. Each redirect is recorded in redirectChain with the URL and status code at that step.

Can I avoid getting blocked by rate limiting?

Enable the proxy configuration to route requests through Apify's datacenter or residential proxies. Residential proxies are less likely to be blocked by sites that filter datacenter traffic.

Run it across one page or a batch of hundreds. Results land in a dataset you can export, pipe into Google Sheets, or hook into any workflow that reads from Apify.