Content-Type Header Validator

Validate whether a website returns the expected Content-Type header (MIME type + optional charset) and optionally warn if X-Content-Type-Options: nosniff is missing.

This is useful for:

• SEO / crawling correctness (HTML should typically be text/html)
• API correctness (JSON should typically be application/json)
• Security hardening (recommend nosniff)

What the Actor does

For each URL it:

1. Fetches headers using the selected strategy (HEAD then optional GET fallback)
2. Parses Content-Type into mediaType and charset
3. Infers or uses the configured expected kind (HTML/JSON/XML/PDF/IMAGE/TEXT)
4. Produces a per-URL result item + aggregate SUMMARY and REPORT

Input

• Start URLs (startUrls): Request List Sources format
• Request strategy (requestStrategy): HEAD-only, GET-only, or HEAD→GET fallback
• Expected kind (expectedKind): AUTO (infer from file extension) or a fixed kind
• Require charset (requireCharset): warn when charset is missing for text/*
• Warn if no nosniff (warnIfNoNosniff): warn when X-Content-Type-Options: nosniff is missing

Plus: maxUrls, timeoutSecs, followRedirects, maxRedirects, maxConcurrency, and proxyConfiguration.

Output

Dataset (per-URL results)

Each item includes:

• startUrl, finalUrl, statusCode, redirected, checkedAt
• contentTypeRaw, mediaType, charset
• expectedKind, sniffedKind (best-effort when GET is used)
• xContentTypeOptions (value of X-Content-Type-Options)
• score (0–100 heuristic), issues, warningCount, errorCount, ok

Key-value store

• SUMMARY: aggregate counts and top issue codes
• REPORT: same as summary (structured JSON)

Example input

{
  "startUrls": [{"url":"https://example.com"},{"url":"https://httpbin.org/json"}],
  "maxUrls": 2,
  "timeoutSecs": 20,
  "requestStrategy": "HEAD_THEN_GET",
  "expectedKind": "AUTO",
  "warnIfNoNosniff": true,
  "maxConcurrency": 2,
  "proxyConfiguration": {"useApifyProxy": false}
}

Quick start

Store page: https://apify.com/scrappy_garden/content-type-header-validator

Paste this into Input and click Run:

{
  "startUrls": [
    {
      "url": "https://example.com/"
    }
  ],
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Outputs (what you get)

• Dataset: Dataset items typically include fields like: startUrl, finalUrl, statusCode, contentTypeRaw, mediaType, charset, expectedKind, sniffedKind, xContentTypeOptions, score.
• Key-value store: REPORT, SUMMARY

Tips (trust + predictable results)

• Start with 1–3 URLs to validate behavior, then scale up.
• If a target blocks requests, enable Proxy and/or slow down concurrency in Input.
• Use the SUMMARY / REPORT keys (when present) for automation pipelines and monitoring.

Related actors

• security-headers-checker (https://apify.com/scrappy_garden/security-headers-checker)
• hsts-header-checker (https://apify.com/scrappy_garden/hsts-header-checker)
• cache-control-checker (https://apify.com/scrappy_garden/cache-control-checker)
• x-frame-options-header-checker (https://apify.com/scrappy_garden/x-frame-options-header-checker)

Search keywords

content type header validator, content-type header validator - check mime type, charset, nosniff, website audit, seo, http headers