Pricing

Pay per usage

Referrer-Policy Auditor

Fetch one public URL and deeply audit its Referrer-Policy HTTP header and meta referrer tag. Validates directives, detects deprecated values, flags conflicts, and returns a privacy posture score, grade, issues, and recommendations.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Sanskar Jaiswal

Actor stats

Bookmarked

Total users

Monthly active users

3 days ago

Last modified

HTTP & Security Headers Auditor

bgfc97/http-security-headers-audit

Audit any list of URLs: status, final redirect, HTTPS, server, security headers (HSTS, CSP, X-Frame-Options...), a security score, page title and CMS/generator. No key, no proxy.

Bruno

Security Headers Auditor

junipr/security-headers-auditor

Audit website security headers such as CSP, HSTS, X-Frame-Options, Referrer-Policy, and Permissions-Policy.

junipr

HTTP Security Headers Auditor

second_coming/http-headers-auditor

Bulk-check URLs for HTTP security headers. Grades sites A+ through F based on HSTS, CSP, XFO, XCTO, Referrer-Policy, and Permissions-Policy presence.

Richard P

Website SEO, Accessibility & AI Readiness Auditor

feeng/website-seo-ai-readiness-auditor

🔎 Crawl websites and turn technical SEO, accessibility, security, performance, broken-link, structured-data, robots.txt, llms.txt, and AI crawler signals into prioritized 0–100 audit scores and client-ready reports.

Daniel S.

HTTP Security Headers & OWASP Posture Auditor

gp005/security-headers-auditor

Audit HTTP security response headers (CSP, HSTS, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy, COOP/COEP/CORP) for any URL. Returns a 0-100 grade, letter rating, and a prioritized fix list with exact header values to add. Single polite GET per URL, no crawling.

Geo

HTTP Security Headers Auditor

phoenix2810/http-security-headers-auditor

Audit a public website's HTTP security response headers in one API call. Returns a letter grade, numeric score, per-header analysis, and actionable recommendations. Built for security teams, devops, and site migration QA.

Sanskar Jaiswal

HTTP Security Headers Auditor

junipr/http-security-headers-auditor

Audit public URLs or supplied header snapshots for security-relevant HTTP response headers and weak values.

junipr

Bulk HTTP Security Headers Analyzer - CSP, HSTS

logiover/bulk-http-security-headers

Analyze HTTP security headers for thousands of URLs. Checks CSP, HSTS, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy and more. Returns pass/fail per header plus a numeric security score. No API key, export to CSV/JSON. Ideal for security audits and compliance scanning.

Logiover

Security Headers Checker — OWASP Audit & Grading

accurate_pouch/security-headers

Audit 12 HTTP security headers (HSTS, CSP, X-Frame-Options, Referrer-Policy, Permissions-Policy, COOP, CORP, COEP). A-F grading, actionable recommendations. 5 URLs free.

Manchitt Sanan

🛡️ Security Headers Checker

taroyamada/security-headers-checker

Audit HTTP security headers in bulk across hundreds of websites. Extract OWASP compliance grades and detect missing HSTS or CSP directives instantly.

naoki anzai

{ "name": "referrer-policy-auditor", "version": "0.1.0", "description": "Apify Actor that fetches one public URL and deeply audits its Referrer-Policy HTTP response header and HTML meta referrer tag. Validates policy directives against the Referrer Policy spec, detects deprecated values, flags conflicting multiple headers, evaluates effective policy precedence, and returns a privacy posture readiness score, letter grade, issues, and recommendations.", "type": "module", "private": true, "main": "src/main.js", "scripts": { "start": "node src/main.js", "test": "node --test test/*.test.js", "lint": "node --check src/main.js" }, "dependencies": { "apify": "^3.4.5" }, "devDependencies": {} }

{ "actorSpecification": 1, "name": "referrer-policy-auditor", "title": "Referrer-Policy Auditor", "description": "Fetch one public URL and deeply audit its Referrer-Policy HTTP header and meta referrer tag. Validates directives, detects deprecated values, flags conflicts, and returns a privacy posture score, grade, issues, and recommendations.", "version": "0.1", "buildTag": "latest", "meta": { "templateId": "js-start" }, "input": "./input_schema.json", "output": "./output_schema.json", "storages": { "dataset": "./dataset_schema.json" }, "dockerfile": "./Dockerfile", "defaultRunOptions": { "build": "latest", "timeoutSecs": 120, "memoryMbytes": 512 } }

{ "actorSpecification": 1, "views": { "overview": { "title": "Referrer-Policy audits", "transformation": { "fields": [ "inputUrl", "finalUrl", "ok", "hasHeader", "hasMetaReferrer", "effectivePolicy", "privacyPosture", "score", "grade", "checkedAt", "error" ] }, "display": { "component": "table", "properties": { "inputUrl": { "label": "Input URL", "format": "link" }, "finalUrl": { "label": "Final URL", "format": "link" }, "ok": { "label": "OK", "format": "boolean" }, "hasHeader": { "label": "Header", "format": "boolean" }, "hasMetaReferrer": { "label": "Meta", "format": "boolean" }, "effectivePolicy": { "label": "Effective policy" }, "privacyPosture": { "label": "Posture" }, "score": { "label": "Score" }, "grade": { "label": "Grade" }, "checkedAt": { "label": "Checked at", "format": "date" }, "error": { "label": "Error" } } } } }, "fields": { "inputUrl": { "type": "string", "title": "Input URL" }, "normalizedInputUrl": { "type": "string", "title": "Normalized input URL" }, "finalUrl": { "type": "string", "title": "Final URL" }, "https": { "type": "boolean", "title": "HTTPS" }, "ok": { "type": "boolean", "title": "OK" }, "checkedAt": { "type": "string", "title": "Checked at", "format": "date-time" }, "httpStatus": { "type": ["integer", "null"], "title": "HTTP status" }, "hasHeader": { "type": "boolean", "title": "Referrer-Policy header present" }, "headerValues": { "type": "array", "title": "Raw Referrer-Policy header values" }, "headerValueCount": { "type": "integer", "title": "Header value count" }, "parsedPolicies": { "type": "array", "title": "Parsed policy directives from header" }, "hasInvalidHeaderDirective": { "type": "boolean", "title": "Has invalid header directive" }, "hasDeprecatedHeaderDirective": { "type": "boolean", "title": "Has deprecated header directive" }, "hasConflictingHeaders": { "type": "boolean", "title": "Has conflicting multiple headers" }, "hasMetaReferrer": { "type": "boolean", "title": "Meta referrer tag present" }, "metaReferrerValues": { "type": "array", "title": "Meta referrer tag values" }, "metaReferrerCount": { "type": "integer", "title": "Meta referrer tag count" }, "hasInvalidMetaDirective": { "type": "boolean", "title": "Has invalid meta directive" }, "hasDeprecatedMetaDirective": { "type": "boolean", "title": "Has deprecated meta directive" }, "effectivePolicy": { "type": ["string", "null"], "title": "Effective policy" }, "effectivePolicySource": { "type": ["string", "null"], "title": "Effective policy source" }, "privacyPosture": { "type": ["string", "null"], "title": "Privacy posture" }, "specCompliant": { "type": "boolean", "title": "Spec compliant" }, "score": { "type": "integer", "title": "Score" }, "grade": { "type": "string", "title": "Grade" }, "issues": { "type": "array", "title": "Issues" }, "recommendations": { "type": "array", "title": "Recommendations" }, "error": { "type": ["string", "null"], "title": "Error" } } }

{ "title": "Referrer-Policy Auditor input", "description": "Fetch one public URL and deeply audit its Referrer-Policy HTTP response header and HTML meta referrer tag. The actor validates the policy value against the Referrer Policy spec, detects deprecated values, flags conflicting multiple headers, evaluates effective policy precedence (header over meta), and returns a privacy posture readiness score, letter grade, issues, and recommendations. URLs with credentials and private network targets are rejected.", "type": "object", "schemaVersion": 1, "properties": { "startUrl": { "title": "URL", "type": "string", "description": "Public HTTP or HTTPS URL to audit. The actor fetches this page and inspects the Referrer-Policy header and meta referrer tag.", "editor": "textfield", "prefill": "https://example.com/" }, "timeoutSeconds": { "title": "Timeout seconds", "type": "integer", "description": "Request timeout from 3 to 30 seconds.", "editor": "number", "default": 10, "minimum": 3, "maximum": 30 } }, "required": ["startUrl"] }

{ "actorOutputSchemaVersion": 1, "title": "Referrer-Policy Audit Results", "properties": { "results": { "type": "string", "title": "Results", "template": "{{links.apiDefaultDatasetUrl}}/items" } } }

1import { Actor } from 'apify'; 2import dns from 'node:dns/promises'; 3import net from 'node:net'; 4import { fileURLToPath } from 'node:url'; 5 6const USER_AGENT = 'ReferrerPolicyAuditor/0.1 (+https://apify.com)'; 7const DEFAULT_TIMEOUT_SECONDS = 10; 8const MAX_BODY_BYTES = 2 * 1024 * 1024; // 2 MB cap for HTML body 9 10/* --------------------------------------------------------------------------- 11 * SSRF protection — shared pattern used across URL-fetching factory actors. 12 * Allows only HTTP/HTTPS, rejects URL credentials, rejects localhost/private 13 * IP literals, DNS-resolves hostnames, and rejects DNS that resolves to 14 * private IP ranges. 15 * ------------------------------------------------------------------------- */ 16 17function isPrivateIPv4(ip) { 18 const parts = ip.split('.').map(Number); 19 if (parts.length !== 4 || parts.some((n) => Number.isNaN(n))) return false; 20 const [a, b] = parts; 21 return a === 10 22 || (a === 172 && b >= 16 && b <= 31) 23 || (a === 192 && b === 168) 24 || a === 127 25 || a === 0 26 || (a === 169 && b === 254); 27} 28 29function isPrivateIPv6(ip) { 30 const normalized = ip.toLowerCase(); 31 return normalized === '::1' 32 || normalized.startsWith('fc') 33 || normalized.startsWith('fd') 34 || normalized.startsWith('fe80:'); 35} 36 37export async function normalizeAndValidateUrl(rawUrl) { 38 if (!rawUrl || typeof rawUrl !== 'string') throw new Error('startUrl is required'); 39 if (/^[a-z][a-z0-9+.-]*:/i.test(rawUrl) && !/^https?:\/\//i.test(rawUrl)) { 40 throw new Error('Only HTTP and HTTPS URLs are supported'); 41 } 42 const withScheme = /^https?:\/\//i.test(rawUrl) ? rawUrl : `https://${rawUrl}`; 43 const url = new URL(withScheme); 44 if (!['http:', 'https:'].includes(url.protocol)) throw new Error('Only HTTP and HTTPS URLs are supported'); 45 if (!url.hostname || url.username || url.password) throw new Error('URL must be public and must not include credentials'); 46 47 const literalType = net.isIP(url.hostname); 48 if (literalType === 4 && isPrivateIPv4(url.hostname)) throw new Error('Private IPv4 targets are blocked'); 49 if (literalType === 6 && isPrivateIPv6(url.hostname)) throw new Error('Private IPv6 targets are blocked'); 50 51 const records = literalType ? [{ address: url.hostname, family: literalType }] : await dns.lookup(url.hostname, { all: true }); 52 for (const record of records) { 53 if (record.family === 4 && isPrivateIPv4(record.address)) throw new Error('DNS resolves to a private IPv4 address; blocked for SSRF safety'); 54 if (record.family === 6 && isPrivateIPv6(record.address)) throw new Error('DNS resolves to a private IPv6 address; blocked for SSRF safety'); 55 } 56 return url; 57} 58 59/* --------------------------------------------------------------------------- 60 * HTTP fetch with manual redirect revalidation (SSRF-safe). 61 * Returns response headers and optional body text. 62 * ------------------------------------------------------------------------- */ 63 64async function fetchUrl(initialUrl, timeoutSeconds, fetchBody, redirectsRemaining = 3) { 65 await normalizeAndValidateUrl(initialUrl.href); 66 const controller = new AbortController(); 67 const timeout = setTimeout(() => controller.abort(), timeoutSeconds * 1000); 68 try { 69 const response = await fetch(initialUrl, { 70 redirect: 'manual', 71 signal: controller.signal, 72 headers: { 'user-agent': USER_AGENT, accept: 'text/html,application/xhtml+xml,*/*;q=0.1' }, 73 }); 74 75 if ([301, 302, 303, 307, 308].includes(response.status)) { 76 if (redirectsRemaining <= 0) throw new Error('Too many redirects'); 77 const location = response.headers.get('location'); 78 if (!location) throw new Error('Redirect without Location header'); 79 const nextUrl = new URL(location, initialUrl.href); 80 try { await response.arrayBuffer(); } catch { /* drain */ } 81 return fetchUrl(nextUrl, timeoutSeconds, fetchBody, redirectsRemaining - 1); 82 } 83 84 const contentType = response.headers.get('content-type') || null; 85 86 if (!response.ok) { 87 try { await response.arrayBuffer(); } catch { /* drain */ } 88 return { 89 ok: false, 90 status: response.status, 91 finalUrl: response.url || initialUrl.href, 92 https: (response.url || initialUrl.href).startsWith('https://'), 93 contentType, 94 headers: response.headers, 95 body: null, 96 error: null, 97 }; 98 } 99 100 let body = null; 101 if (fetchBody) { 102 const buffer = await response.arrayBuffer(); 103 const bytes = new Uint8Array(buffer.slice(0, MAX_BODY_BYTES)); 104 body = new TextDecoder('utf-8', { fatal: false }).decode(bytes); 105 } 106 107 return { 108 ok: true, 109 status: response.status, 110 finalUrl: response.url || initialUrl.href, 111 https: (response.url || initialUrl.href).startsWith('https://'), 112 contentType, 113 headers: response.headers, 114 body, 115 error: null, 116 }; 117 } catch (error) { 118 return { 119 ok: false, 120 status: null, 121 finalUrl: initialUrl.href, 122 https: initialUrl.protocol === 'https:', 123 contentType: null, 124 headers: null, 125 body: null, 126 error: error.message, 127 }; 128 } finally { 129 clearTimeout(timeout); 130 } 131} 132 133/* --------------------------------------------------------------------------- 134 * Referrer Policy spec constants. 135 * 136 * Valid policy directives per the Referrer Policy spec (W3C / fetch standard): 137 * - no-referrer 138 * - no-referrer-when-downgrade 139 * - same-origin 140 * - origin 141 * - strict-origin 142 * - origin-when-cross-origin 143 * - strict-origin-when-cross-origin (browser default since 2020) 144 * - unsafe-url 145 * 146 * "no-referrer-when-downgrade" and "unsafe-url" are not deprecated per spec, 147 * but "no-referrer-when-downgrade" is the pre-2020 default and leaks full URLs 148 * on same-origin requests. "unsafe-url" always leaks full URLs. 149 * 150 * Multiple comma-separated directives: the last valid directive wins per spec. 151 * ------------------------------------------------------------------------- */ 152 153const VALID_DIRECTIVES = new Set([ 154 'no-referrer', 155 'no-referrer-when-downgrade', 156 'same-origin', 157 'origin', 158 'strict-origin', 159 'origin-when-cross-origin', 160 'strict-origin-when-cross-origin', 161 'unsafe-url', 162]); 163 164// Directives that leak full URLs (privacy risk) 165const FULL_URL_LEAKING = new Set(['unsafe-url', 'origin-when-cross-origin', 'no-referrer-when-downgrade']); 166 167// Directives that are privacy-strong (no cross-origin URL leakage) 168const PRIVACY_STRONG = new Set(['no-referrer', 'same-origin', 'strict-origin', 'strict-origin-when-cross-origin']); 169 170// The spec-recommended default (browser default since Chrome 85 / Firefox 87) 171const SPEC_DEFAULT = 'strict-origin-when-cross-origin'; 172 173export function parseReferrerPolicy(rawValue) { 174 if (!rawValue || typeof rawValue !== 'string' || rawValue.trim() === '') { 175 return { directives: [], valid: [], invalid: [], deprecated: [], effective: null }; 176 } 177 178 // Per spec, multiple policies can be comma-separated; last valid one wins. 179 const parts = rawValue.split(',').map((p) => p.trim().toLowerCase()).filter((p) => p !== ''); 180 const valid = []; 181 const invalid = []; 182 let effective = null; 183 184 for (const part of parts) { 185 if (VALID_DIRECTIVES.has(part)) { 186 valid.push(part); 187 effective = part; // last valid wins 188 } else { 189 invalid.push(part); 190 } 191 } 192 193 // "deprecated" in this context = privacy-leaking or pre-2020 default 194 const deprecated = valid.filter((d) => FULL_URL_LEAKING.has(d)); 195 196 return { 197 directives: parts, 198 valid, 199 invalid, 200 deprecated, 201 effective, 202 }; 203} 204 205/* --------------------------------------------------------------------------- 206 * Classify the privacy posture of an effective policy. 207 * Returns: 'strong' | 'balanced' | 'weak' | 'none' 208 * ------------------------------------------------------------------------- */ 209 210export function classifyPosture(policy) { 211 if (!policy) return 'none'; 212 if (PRIVACY_STRONG.has(policy)) { 213 if (policy === 'no-referrer' || policy === 'same-origin') return 'strong'; 214 return 'strong'; // strict-origin and strict-origin-when-cross-origin are also strong 215 } 216 if (policy === 'origin') return 'balanced'; // leaks origin but not full URL 217 return 'weak'; // unsafe-url, origin-when-cross-origin, no-referrer-when-downgrade 218} 219 220/* --------------------------------------------------------------------------- 221 * Extract <meta name="referrer" content="..."> tags from HTML. 222 * Per spec, the meta tag is honored only if no Referrer-Policy header is present. 223 * The first meta referrer tag wins (earliest in document order). 224 * ------------------------------------------------------------------------- */ 225 226export function extractMetaReferrer(html) { 227 if (!html || typeof html !== 'string') return []; 228 229 const values = []; 230 // ponytail: regex-based extraction avoids a full DOM parser dependency. 231 // Matches <meta name="referrer" content="..."> case-insensitively, 232 // tolerating single/double quotes and attribute order. 233 const metaRegex = /<meta\s+[^>]*>/gi; 234 let match; 235 while ((match = metaRegex.exec(html)) !== null) { 236 const tag = match[0]; 237 if (/\bname\s*=\s*["']?referrer["']?/i.test(tag)) { 238 const contentMatch = /\bcontent\s*=\s*["']([^"']*)["']/i.exec(tag); 239 if (contentMatch) { 240 values.push(contentMatch[1].trim()); 241 } else { 242 values.push(''); 243 } 244 } 245 } 246 return values; 247} 248 249/* --------------------------------------------------------------------------- 250 * Score and grade the Referrer-Policy posture. 251 * ------------------------------------------------------------------------- */ 252 253export function scoreReferrerPolicy(analysis) { 254 let score = 0; 255 const issues = []; 256 const recommendations = []; 257 258 const { hasHeader, headerParsed, hasMetaReferrer, metaParsed, effectivePolicy, privacyPosture, hasInvalidHeaderDirective, hasDeprecatedHeaderDirective, hasConflictingHeaders, hasInvalidMetaDirective, hasDeprecatedMetaDirective } = analysis; 259 260 if (!hasHeader && !hasMetaReferrer) { 261 // No policy at all — browser defaults to strict-origin-when-cross-origin, 262 // but the site should set it explicitly. 263 score = 40; 264 issues.push('No Referrer-Policy header and no meta referrer tag detected. Browsers default to "strict-origin-when-cross-origin", but the policy is not explicitly declared and may change with browser updates.'); 265 recommendations.push('Set Referrer-Policy: strict-origin-when-cross-origin as an HTTP response header to explicitly control referrer leakage.'); 266 return finalize(score, issues, recommendations); 267 } 268 269 score = 100; 270 271 // Header presence is better than meta-only 272 if (hasHeader) { 273 // Full marks for header 274 } else if (hasMetaReferrer) { 275 // Meta-only: deduct because meta is weaker (can be stripped by intermediaries) 276 score -= 15; 277 issues.push('Referrer-Policy is set via meta tag only, not as an HTTP response header. HTTP headers are more reliable because meta tags can be stripped by intermediaries or missed if the HTML is not fully parsed.'); 278 recommendations.push('Set Referrer-Policy as an HTTP response header instead of (or in addition to) the meta tag for reliability.'); 279 } 280 281 // Invalid header directives 282 if (hasInvalidHeaderDirective) { 283 score -= 15; 284 issues.push(`Referrer-Policy header contains invalid directive(s): ${headerParsed.invalid.join(', ')}. Invalid directives are ignored by browsers.`); 285 recommendations.push(`Remove invalid directives from the Referrer-Policy header. Valid values: ${[...VALID_DIRECTIVES].join(', ')}.`); 286 } 287 288 // Deprecated / privacy-leaking header directives 289 if (hasDeprecatedHeaderDirective) { 290 score -= 20; 291 issues.push(`Referrer-Policy header uses a privacy-leaking directive: ${headerParsed.deprecated.join(', ')}. These directives may leak full URLs to third parties.`); 292 recommendations.push('Use "strict-origin-when-cross-origin" (the browser default) or "no-referrer" for stronger privacy.'); 293 } 294 295 // Conflicting multiple headers 296 if (hasConflictingHeaders) { 297 score -= 10; 298 issues.push('Multiple Referrer-Policy header values detected. Per spec, the last valid directive wins, but this is fragile and can cause unexpected behavior.'); 299 recommendations.push('Use a single Referrer-Policy header value to avoid ambiguity.'); 300 } 301 302 // Invalid meta directives 303 if (hasInvalidMetaDirective) { 304 score -= 5; 305 issues.push(`Meta referrer tag contains invalid directive(s): ${metaParsed.invalid.join(', ')}.`); 306 recommendations.push(`Remove invalid directives from the meta referrer tag. Valid values: ${[...VALID_DIRECTIVES].join(', ')}.`); 307 } 308 309 // Deprecated meta directives 310 if (hasDeprecatedMetaDirective) { 311 score -= 10; 312 issues.push(`Meta referrer tag uses a privacy-leaking directive: ${metaParsed.deprecated.join(', ')}.`); 313 recommendations.push('Use "strict-origin-when-cross-origin" or "no-referrer" in the meta referrer tag for stronger privacy.'); 314 } 315 316 // Effective policy quality 317 if (effectivePolicy) { 318 if (effectivePolicy === 'unsafe-url') { 319 score -= 25; 320 issues.push('Effective policy is "unsafe-url", which sends full URLs as Referer headers on all requests, including cross-origin. This is the most privacy-leaking policy.'); 321 recommendations.push('Change the effective policy from "unsafe-url" to "strict-origin-when-cross-origin" or "no-referrer".'); 322 } else if (effectivePolicy === 'no-referrer-when-downgrade') { 323 score -= 15; 324 issues.push('Effective policy is "no-referrer-when-downgrade", which sends full URLs on same-origin requests and cross-origin HTTPS requests. This is the pre-2020 browser default and leaks more than necessary.'); 325 recommendations.push('Change to "strict-origin-when-cross-origin" (the current browser default) to reduce referrer leakage.'); 326 } else if (effectivePolicy === 'origin-when-cross-origin') { 327 score -= 10; 328 issues.push('Effective policy is "origin-when-cross-origin", which sends full URLs on same-origin requests but only the origin cross-origin.'); 329 recommendations.push('Consider "strict-origin-when-cross-origin" for reduced same-origin URL leakage in logs.'); 330 } else if (effectivePolicy === 'no-referrer') { 331 // Strong but may break analytics 332 if (!issues.some((i) => i.includes('no-referrer'))) { 333 issues.push('Effective policy is "no-referrer", which sends no Referer header at all. This is the strongest privacy setting but may break analytics and downstream referrer-dependent features.'); 334 } 335 } else if (effectivePolicy === 'strict-origin-when-cross-origin') { 336 // Optimal — no issue 337 } else if (effectivePolicy === 'same-origin') { 338 // Strong 339 } else if (effectivePolicy === 'strict-origin') { 340 // Strong 341 } else if (effectivePolicy === 'origin') { 342 score -= 5; 343 issues.push('Effective policy is "origin", which sends only the origin (scheme+host+port) but not the full URL. This is balanced but less protective than strict-origin-when-cross-origin.'); 344 recommendations.push('Consider "strict-origin-when-cross-origin" for better same-origin referrer detail with cross-origin protection.'); 345 } 346 } 347 348 // Bonus: both header and meta present and consistent 349 if (hasHeader && hasMetaReferrer && headerParsed.effective === metaParsed.effective) { 350 score = Math.min(100, score + 5); 351 } 352 353 // Penalize header + meta with different values (conflict) 354 if (hasHeader && hasMetaReferrer && headerParsed.effective !== metaParsed.effective) { 355 score -= 10; 356 issues.push('Referrer-Policy header and meta referrer tag have different values. Per spec, the HTTP header takes precedence, but the mismatch can cause confusion.'); 357 recommendations.push('Use the same policy value in both the HTTP header and the meta referrer tag.'); 358 } 359 360 return finalize(score, issues, recommendations); 361} 362 363function finalize(score, issues, recommendations) { 364 const bounded = Math.max(0, Math.min(100, score)); 365 const grade = bounded >= 95 ? 'A+' 366 : bounded >= 85 ? 'A' 367 : bounded >= 75 ? 'B' 368 : bounded >= 65 ? 'C' 369 : bounded >= 50 ? 'D' 370 : bounded >= 30 ? 'E' 371 : 'F'; 372 373 if (bounded >= 95 && issues.length === 0) { 374 recommendations.push('Referrer-Policy posture is strong and spec-compliant. Continue monitoring after deploys and CDN cutovers.'); 375 } 376 377 if (recommendations.length === 0 && issues.length === 0) { 378 recommendations.push('No Referrer-Policy issues detected.'); 379 } 380 381 return { score: bounded, grade, issues, recommendations }; 382} 383 384/* --------------------------------------------------------------------------- 385 * Main audit pipeline. 386 * ------------------------------------------------------------------------- */ 387 388export async function auditReferrerPolicy(input) { 389 const timeoutSeconds = Math.min(Math.max(Number(input.timeoutSeconds || DEFAULT_TIMEOUT_SECONDS), 3), 30); 390 391 let startUrl; 392 try { 393 startUrl = await normalizeAndValidateUrl(input.startUrl); 394 } catch (validationError) { 395 return errorResult(input.startUrl, validationError.message); 396 } 397 398 // Fetch headers + body (need body for meta referrer tag) 399 let fetched; 400 try { 401 fetched = await fetchUrl(startUrl, timeoutSeconds, true); 402 } catch (fetchError) { 403 return errorResult(input.startUrl, fetchError.message); 404 } 405 406 if (fetched.error) { 407 return errorResult(input.startUrl, fetched.error, fetched.finalUrl, fetched.https, fetched.status); 408 } 409 410 // Extract Referrer-Policy header(s) 411 // getSetCookie returns array; for other headers, get() returns comma-joined. 412 // We need to handle multiple Referrer-Policy headers properly. 413 const headerValues = []; 414 if (fetched.headers) { 415 // fetch API doesn't expose getSetCookie for non-cookie headers. 416 // headers.get() joins multiple values with ", " — which is also the 417 // spec separator for directives within a single header. 418 // We treat each comma-separated value as a potential separate directive 419 // and let the parser sort it out (last valid wins). 420 const raw = fetched.headers.get('referrer-policy'); 421 if (raw) { 422 headerValues.push(raw); 423 } 424 } 425 426 const hasHeader = headerValues.length > 0; 427 const headerParsed = headerValues.length > 0 428 ? parseReferrerPolicy(headerValues.join(', ')) 429 : { directives: [], valid: [], invalid: [], deprecated: [], effective: null }; 430 431 // Detect conflicting multiple header values 432 // Multiple distinct valid directives in the header = conflict 433 const hasConflictingHeaders = headerParsed.valid.length > 1; 434 435 // Extract meta referrer tags from HTML body 436 const metaReferrerValues = extractMetaReferrer(fetched.body); 437 const hasMetaReferrer = metaReferrerValues.length > 0; 438 const metaParsed = hasMetaReferrer 439 ? parseReferrerPolicy(metaReferrerValues[0]) 440 : { directives: [], valid: [], invalid: [], deprecated: [], effective: null }; 441 442 // Effective policy: header takes precedence over meta (per spec) 443 let effectivePolicy = null; 444 let effectivePolicySource = null; 445 if (headerParsed.effective) { 446 effectivePolicy = headerParsed.effective; 447 effectivePolicySource = 'header'; 448 } else if (metaParsed.effective) { 449 effectivePolicy = metaParsed.effective; 450 effectivePolicySource = 'meta'; 451 } 452 453 const privacyPosture = classifyPosture(effectivePolicy); 454 const specCompliant = effectivePolicy !== null && VALID_DIRECTIVES.has(effectivePolicy); 455 456 const analysis = { 457 hasHeader, 458 headerParsed, 459 hasMetaReferrer, 460 metaParsed, 461 effectivePolicy, 462 privacyPosture, 463 hasInvalidHeaderDirective: headerParsed.invalid.length > 0, 464 hasDeprecatedHeaderDirective: headerParsed.deprecated.length > 0, 465 hasConflictingHeaders, 466 hasInvalidMetaDirective: hasMetaReferrer && metaParsed.invalid.length > 0, 467 hasDeprecatedMetaDirective: hasMetaReferrer && metaParsed.deprecated.length > 0, 468 }; 469 470 const scored = scoreReferrerPolicy(analysis); 471 472 return { 473 inputUrl: input.startUrl, 474 normalizedInputUrl: startUrl.href, 475 finalUrl: fetched.finalUrl, 476 https: fetched.https, 477 ok: true, 478 checkedAt: new Date().toISOString(), 479 httpStatus: fetched.status, 480 hasHeader, 481 headerValues, 482 headerValueCount: headerValues.length, 483 parsedPolicies: headerParsed.valid, 484 hasInvalidHeaderDirective: headerParsed.invalid.length > 0, 485 hasDeprecatedHeaderDirective: headerParsed.deprecated.length > 0, 486 hasConflictingHeaders, 487 hasMetaReferrer, 488 metaReferrerValues, 489 metaReferrerCount: metaReferrerValues.length, 490 hasInvalidMetaDirective: hasMetaReferrer && metaParsed.invalid.length > 0, 491 hasDeprecatedMetaDirective: hasMetaReferrer && metaParsed.deprecated.length > 0, 492 effectivePolicy, 493 effectivePolicySource, 494 privacyPosture, 495 specCompliant, 496 score: scored.score, 497 grade: scored.grade, 498 issues: scored.issues, 499 recommendations: scored.recommendations, 500 error: null, 501 }; 502} 503 504function errorResult(inputUrl, errorMsg, finalUrl, https, httpStatus) { 505 return { 506 inputUrl, 507 normalizedInputUrl: null, 508 finalUrl: finalUrl || (typeof inputUrl === 'string' ? inputUrl : null), 509 https: https ?? false, 510 ok: false, 511 checkedAt: new Date().toISOString(), 512 httpStatus: httpStatus ?? null, 513 hasHeader: false, 514 headerValues: [], 515 headerValueCount: 0, 516 parsedPolicies: [], 517 hasInvalidHeaderDirective: false, 518 hasDeprecatedHeaderDirective: false, 519 hasConflictingHeaders: false, 520 hasMetaReferrer: false, 521 metaReferrerValues: [], 522 metaReferrerCount: 0, 523 hasInvalidMetaDirective: false, 524 hasDeprecatedMetaDirective: false, 525 effectivePolicy: null, 526 effectivePolicySource: null, 527 privacyPosture: 'none', 528 specCompliant: false, 529 score: 0, 530 grade: 'F', 531 issues: ['The request failed before the Referrer-Policy could be inspected. Verify the URL is reachable and try again.'], 532 recommendations: ['Verify the URL is public, reachable, and returns a response.'], 533 error: errorMsg, 534 }; 535} 536 537/* --------------------------------------------------------------------------- 538 * Apify runtime entry point. 539 * ------------------------------------------------------------------------- */ 540 541const isExecutedDirectly = process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]; 542 543if (process.env.NODE_ENV !== 'test' && isExecutedDirectly) { 544 await Actor.init(); 545 try { 546 const input = await Actor.getInput(); 547 const result = await auditReferrerPolicy(input || {}); 548 await Actor.pushData(result); 549 await Actor.setValue('OUTPUT', result); 550 Actor.log.info('Referrer-Policy audit complete', { 551 hasHeader: result.hasHeader, 552 effectivePolicy: result.effectivePolicy, 553 score: result.score, 554 grade: result.grade, 555 }); 556 } finally { 557 await Actor.exit(); 558 } 559}

1import test from 'node:test'; 2import assert from 'node:assert/strict'; 3import { 4 parseReferrerPolicy, 5 classifyPosture, 6 extractMetaReferrer, 7 scoreReferrerPolicy, 8 normalizeAndValidateUrl, 9 auditReferrerPolicy, 10} from '../src/main.js'; 11 12/* --- parseReferrerPolicy: parsing -------------------------------------- */ 13 14test('parseReferrerPolicy parses a single valid directive', () => { 15 const r = parseReferrerPolicy('strict-origin-when-cross-origin'); 16 assert.equal(r.effective, 'strict-origin-when-cross-origin'); 17 assert.equal(r.valid.length, 1); 18 assert.equal(r.invalid.length, 0); 19}); 20 21test('parseReferrerPolicy handles empty input', () => { 22 const r = parseReferrerPolicy(''); 23 assert.equal(r.effective, null); 24 assert.equal(r.valid.length, 0); 25}); 26 27test('parseReferrerPolicy handles null input', () => { 28 const r = parseReferrerPolicy(null); 29 assert.equal(r.effective, null); 30 assert.equal(r.valid.length, 0); 31}); 32 33test('parseReferrerPolicy is case-insensitive', () => { 34 const r = parseReferrerPolicy('No-Referrer'); 35 assert.equal(r.effective, 'no-referrer'); 36}); 37 38test('parseReferrerPolicy trims whitespace', () => { 39 const r = parseReferrerPolicy(' strict-origin '); 40 assert.equal(r.effective, 'strict-origin'); 41}); 42 43test('parseReferrerPolicy flags invalid directives', () => { 44 const r = parseReferrerPolicy('none'); 45 assert.equal(r.valid.length, 0); 46 assert.equal(r.invalid.length, 1); 47 assert.equal(r.invalid[0], 'none'); 48 assert.equal(r.effective, null); 49}); 50 51test('parseReferrerPolicy handles comma-separated directives (last valid wins)', () => { 52 const r = parseReferrerPolicy('no-referrer, strict-origin-when-cross-origin'); 53 assert.equal(r.valid.length, 2); 54 assert.equal(r.effective, 'strict-origin-when-cross-origin'); 55}); 56 57test('parseReferrerPolicy handles mixed valid and invalid directives', () => { 58 const r = parseReferrerPolicy('invalid-val, no-referrer, another-bad'); 59 assert.equal(r.valid.length, 1); 60 assert.equal(r.invalid.length, 2); 61 assert.equal(r.effective, 'no-referrer'); 62}); 63 64test('parseReferrerPolicy marks unsafe-url as deprecated', () => { 65 const r = parseReferrerPolicy('unsafe-url'); 66 assert.equal(r.deprecated.length, 1); 67 assert.equal(r.deprecated[0], 'unsafe-url'); 68}); 69 70test('parseReferrerPolicy marks no-referrer-when-downgrade as deprecated', () => { 71 const r = parseReferrerPolicy('no-referrer-when-downgrade'); 72 assert.equal(r.deprecated.length, 1); 73}); 74 75test('parseReferrerPolicy does not mark strict-origin-when-cross-origin as deprecated', () => { 76 const r = parseReferrerPolicy('strict-origin-when-cross-origin'); 77 assert.equal(r.deprecated.length, 0); 78}); 79 80/* --- classifyPosture ---------------------------------------------------- */ 81 82test('classifyPosture returns none for null policy', () => { 83 assert.equal(classifyPosture(null), 'none'); 84}); 85 86test('classifyPosture returns strong for no-referrer', () => { 87 assert.equal(classifyPosture('no-referrer'), 'strong'); 88}); 89 90test('classifyPosture returns strong for same-origin', () => { 91 assert.equal(classifyPosture('same-origin'), 'strong'); 92}); 93 94test('classifyPosture returns strong for strict-origin', () => { 95 assert.equal(classifyPosture('strict-origin'), 'strong'); 96}); 97 98test('classifyPosture returns strong for strict-origin-when-cross-origin', () => { 99 assert.equal(classifyPosture('strict-origin-when-cross-origin'), 'strong'); 100}); 101 102test('classifyPosture returns balanced for origin', () => { 103 assert.equal(classifyPosture('origin'), 'balanced'); 104}); 105 106test('classifyPosture returns weak for unsafe-url', () => { 107 assert.equal(classifyPosture('unsafe-url'), 'weak'); 108}); 109 110test('classifyPosture returns weak for origin-when-cross-origin', () => { 111 assert.equal(classifyPosture('origin-when-cross-origin'), 'weak'); 112}); 113 114test('classifyPosture returns weak for no-referrer-when-downgrade', () => { 115 assert.equal(classifyPosture('no-referrer-when-downgrade'), 'weak'); 116}); 117 118/* --- extractMetaReferrer ----------------------------------------------- */ 119 120test('extractMetaReferrer extracts a meta referrer tag with double quotes', () => { 121 const html = '<html><head><meta name="referrer" content="no-referrer"></head><body></body></html>'; 122 const values = extractMetaReferrer(html); 123 assert.equal(values.length, 1); 124 assert.equal(values[0], 'no-referrer'); 125}); 126 127test('extractMetaReferrer extracts a meta referrer tag with single quotes', () => { 128 const html = "<html><head><meta name='referrer' content='strict-origin-when-cross-origin'></head></html>"; 129 const values = extractMetaReferrer(html); 130 assert.equal(values.length, 1); 131 assert.equal(values[0], 'strict-origin-when-cross-origin'); 132}); 133 134test('extractMetaReferrer handles reversed attribute order (content before name)', () => { 135 const html = '<meta content="same-origin" name="referrer">'; 136 const values = extractMetaReferrer(html); 137 assert.equal(values.length, 1); 138 assert.equal(values[0], 'same-origin'); 139}); 140 141test('extractMetaReferrer is case-insensitive', () => { 142 const html = '<META NAME="Referrer" CONTENT="origin">'; 143 const values = extractMetaReferrer(html); 144 assert.equal(values.length, 1); 145 assert.equal(values[0], 'origin'); 146}); 147 148test('extractMetaReferrer returns empty array for null input', () => { 149 assert.equal(extractMetaReferrer(null).length, 0); 150}); 151 152test('extractMetaReferrer returns empty array for HTML without meta referrer', () => { 153 const html = '<html><head><meta name="viewport" content="width=device-width"></head></html>'; 154 assert.equal(extractMetaReferrer(html).length, 0); 155}); 156 157test('extractMetaReferrer extracts multiple meta referrer tags', () => { 158 const html = '<meta name="referrer" content="no-referrer"><meta name="referrer" content="origin">'; 159 const values = extractMetaReferrer(html); 160 assert.equal(values.length, 2); 161 assert.equal(values[0], 'no-referrer'); 162 assert.equal(values[1], 'origin'); 163}); 164 165/* --- scoreReferrerPolicy ------------------------------------------------ */ 166 167test('scoreReferrerPolicy returns low score when no policy is set', () => { 168 const analysis = { 169 hasHeader: false, 170 headerParsed: { valid: [], invalid: [], deprecated: [], effective: null }, 171 hasMetaReferrer: false, 172 metaParsed: { valid: [], invalid: [], deprecated: [], effective: null }, 173 effectivePolicy: null, 174 privacyPosture: 'none', 175 hasInvalidHeaderDirective: false, 176 hasDeprecatedHeaderDirective: false, 177 hasConflictingHeaders: false, 178 hasInvalidMetaDirective: false, 179 hasDeprecatedMetaDirective: false, 180 }; 181 const scored = scoreReferrerPolicy(analysis); 182 assert.ok(scored.score <= 40); 183 assert.ok(['D', 'E', 'F'].includes(scored.grade)); 184 assert.ok(scored.issues.length > 0); 185}); 186 187test('scoreReferrerPolicy returns high score for strict-origin-when-cross-origin header', () => { 188 const analysis = { 189 hasHeader: true, 190 headerParsed: { valid: ['strict-origin-when-cross-origin'], invalid: [], deprecated: [], effective: 'strict-origin-when-cross-origin' }, 191 hasMetaReferrer: false, 192 metaParsed: { valid: [], invalid: [], deprecated: [], effective: null }, 193 effectivePolicy: 'strict-origin-when-cross-origin', 194 privacyPosture: 'strong', 195 hasInvalidHeaderDirective: false, 196 hasDeprecatedHeaderDirective: false, 197 hasConflictingHeaders: false, 198 hasInvalidMetaDirective: false, 199 hasDeprecatedMetaDirective: false, 200 }; 201 const scored = scoreReferrerPolicy(analysis); 202 assert.ok(scored.score >= 95); 203 assert.equal(scored.grade, 'A+'); 204}); 205 206test('scoreReferrerPolicy penalizes unsafe-url', () => { 207 const analysis = { 208 hasHeader: true, 209 headerParsed: { valid: ['unsafe-url'], invalid: [], deprecated: ['unsafe-url'], effective: 'unsafe-url' }, 210 hasMetaReferrer: false, 211 metaParsed: { valid: [], invalid: [], deprecated: [], effective: null }, 212 effectivePolicy: 'unsafe-url', 213 privacyPosture: 'weak', 214 hasInvalidHeaderDirective: false, 215 hasDeprecatedHeaderDirective: true, 216 hasConflictingHeaders: false, 217 hasInvalidMetaDirective: false, 218 hasDeprecatedMetaDirective: false, 219 }; 220 const scored = scoreReferrerPolicy(analysis); 221 assert.ok(scored.score < 60); 222 assert.ok(scored.issues.some((i) => i.includes('unsafe-url'))); 223}); 224 225test('scoreReferrerPolicy penalizes meta-only without header', () => { 226 const analysis = { 227 hasHeader: false, 228 headerParsed: { valid: [], invalid: [], deprecated: [], effective: null }, 229 hasMetaReferrer: true, 230 metaParsed: { valid: ['strict-origin-when-cross-origin'], invalid: [], deprecated: [], effective: 'strict-origin-when-cross-origin' }, 231 effectivePolicy: 'strict-origin-when-cross-origin', 232 privacyPosture: 'strong', 233 hasInvalidHeaderDirective: false, 234 hasDeprecatedHeaderDirective: false, 235 hasConflictingHeaders: false, 236 hasInvalidMetaDirective: false, 237 hasDeprecatedMetaDirective: false, 238 }; 239 const scored = scoreReferrerPolicy(analysis); 240 assert.ok(scored.score < 95); 241 assert.ok(scored.issues.some((i) => i.includes('meta tag only'))); 242}); 243 244test('scoreReferrerPolicy penalizes conflicting header and meta values', () => { 245 const analysis = { 246 hasHeader: true, 247 headerParsed: { valid: ['no-referrer'], invalid: [], deprecated: [], effective: 'no-referrer' }, 248 hasMetaReferrer: true, 249 metaParsed: { valid: ['unsafe-url'], invalid: [], deprecated: ['unsafe-url'], effective: 'unsafe-url' }, 250 effectivePolicy: 'no-referrer', 251 privacyPosture: 'strong', 252 hasInvalidHeaderDirective: false, 253 hasDeprecatedHeaderDirective: false, 254 hasConflictingHeaders: false, 255 hasInvalidMetaDirective: false, 256 hasDeprecatedMetaDirective: true, 257 }; 258 const scored = scoreReferrerPolicy(analysis); 259 assert.ok(scored.issues.some((i) => i.includes('different values'))); 260}); 261 262test('scoreReferrerPolicy gives bonus for consistent header and meta', () => { 263 const analysis = { 264 hasHeader: true, 265 headerParsed: { valid: ['strict-origin-when-cross-origin'], invalid: [], deprecated: [], effective: 'strict-origin-when-cross-origin' }, 266 hasMetaReferrer: true, 267 metaParsed: { valid: ['strict-origin-when-cross-origin'], invalid: [], deprecated: [], effective: 'strict-origin-when-cross-origin' }, 268 effectivePolicy: 'strict-origin-when-cross-origin', 269 privacyPosture: 'strong', 270 hasInvalidHeaderDirective: false, 271 hasDeprecatedHeaderDirective: false, 272 hasConflictingHeaders: false, 273 hasInvalidMetaDirective: false, 274 hasDeprecatedMetaDirective: false, 275 }; 276 const scored = scoreReferrerPolicy(analysis); 277 assert.equal(scored.score, 100); 278 assert.equal(scored.grade, 'A+'); 279}); 280 281test('scoreReferrerPolicy penalizes conflicting multiple header directives', () => { 282 const analysis = { 283 hasHeader: true, 284 headerParsed: { valid: ['no-referrer', 'strict-origin-when-cross-origin'], invalid: [], deprecated: [], effective: 'strict-origin-when-cross-origin' }, 285 hasMetaReferrer: false, 286 metaParsed: { valid: [], invalid: [], deprecated: [], effective: null }, 287 effectivePolicy: 'strict-origin-when-cross-origin', 288 privacyPosture: 'strong', 289 hasInvalidHeaderDirective: false, 290 hasDeprecatedHeaderDirective: false, 291 hasConflictingHeaders: true, 292 hasInvalidMetaDirective: false, 293 hasDeprecatedMetaDirective: false, 294 }; 295 const scored = scoreReferrerPolicy(analysis); 296 assert.ok(scored.issues.some((i) => i.includes('Multiple Referrer-Policy'))); 297}); 298 299test('scoreReferrerPolicy penalizes invalid header directives', () => { 300 const analysis = { 301 hasHeader: true, 302 headerParsed: { valid: ['no-referrer'], invalid: ['none', 'whatever'], deprecated: [], effective: 'no-referrer' }, 303 hasMetaReferrer: false, 304 metaParsed: { valid: [], invalid: [], deprecated: [], effective: null }, 305 effectivePolicy: 'no-referrer', 306 privacyPosture: 'strong', 307 hasInvalidHeaderDirective: true, 308 hasDeprecatedHeaderDirective: false, 309 hasConflictingHeaders: false, 310 hasInvalidMetaDirective: false, 311 hasDeprecatedMetaDirective: false, 312 }; 313 const scored = scoreReferrerPolicy(analysis); 314 assert.ok(scored.score < 90); 315 assert.ok(scored.issues.some((i) => i.includes('invalid directive'))); 316}); 317 318/* --- normalizeAndValidateUrl: SSRF guards ------------------------------- */ 319 320test('normalizeAndValidateUrl blocks private IPv4 literals', async () => { 321 await assert.rejects(() => normalizeAndValidateUrl('http://10.0.0.1/'), /Private IPv4/); 322 await assert.rejects(() => normalizeAndValidateUrl('http://192.168.1.1/'), /Private IPv4/); 323}); 324 325test('normalizeAndValidateUrl blocks localhost DNS resolution', async () => { 326 await assert.rejects(() => normalizeAndValidateUrl('http://localhost/'), /private|Private|DNS/); 327}); 328 329test('normalizeAndValidateUrl rejects credentials in URL', async () => { 330 await assert.rejects(() => normalizeAndValidateUrl('https://user:pass@example.com/'), /credentials/); 331}); 332 333test('normalizeAndValidateUrl rejects non-HTTP schemes', async () => { 334 await assert.rejects(() => normalizeAndValidateUrl('file:///etc/passwd'), /Only HTTP/); 335}); 336 337test('normalizeAndValidateUrl defaults missing scheme to https', async () => { 338 const url = await normalizeAndValidateUrl('apify.com'); 339 assert.equal(url.protocol, 'https:'); 340 assert.equal(url.hostname, 'apify.com'); 341}); 342 343/* --- auditReferrerPolicy: end-to-end error path ------------------------- */ 344 345test('auditReferrerPolicy returns an F-grade error object when the fetch fails', async () => { 346 const result = await auditReferrerPolicy({ startUrl: 'https://this-host-does-not-exist-cron-test.invalid/' }); 347 assert.equal(result.ok, false); 348 assert.equal(result.score, 0); 349 assert.equal(result.grade, 'F'); 350 assert.equal(result.hasHeader, false); 351 assert.ok(Array.isArray(result.issues)); 352 assert.ok(result.issues.length > 0); 353});