Website Technology Detector

Fingerprint the technologies a website exposes on the pages its own robots.txt permits — web server, CDN, CMS / site builder, JavaScript frameworks, analytics and marketing tags — for buyer-supplied domains. One scan per domain, and every detection carries an evidence field: the concrete response header or HTML substring that triggered it, and where it was seen.

Differentiation — the fleet-2-standard rebuild of the classic tech-lookup scraper (fleet-1 sibling: tech-stack-analyzer). The trust deltas are the whole point:

• Robots honored. The origin's robots.txt is fetched and honored per origin before every fetch — only pages the site permits are read. An origin that disallows the target pages is reported ROBOTS_BLOCKED and skipped, never billed.
• Evidence, not guesses. Each technology is reported with the exact header value or HTML fragment that matched, so you can audit every fingerprint instead of trusting an opaque label. Detection is from what the site publicly serves only — no exploit probing.
• Fail loud. SSRF-refused targets, all-failed fetches, and unparseable inputs are surfaced and never billed; a batch with no completed scan fails loud.
• Exact billing. You are charged only for a completed fingerprint scan on permitted pages (FINGERPRINTED) — including a permitted fetch that legitimately matches no signature (still a completed scan). Blocked and faulted origins are never billed.
• SSRF-guarded. Only public hosts are fetched; private / loopback / link-local / metadata addresses are refused.

What it returns

Per domain: technologies (each {technology, category, evidence, source}), technology_names, technology_count, pages_fetched, pages_blocked_by_robots, robots_status, and a coverage_summary.

Scope & PII

This reads technology fingerprints a site publicly serves — not personal data. Buyer-supplied domains only. It is not a person-discovery tool. Inputs are not retained beyond the run.

Billing

Pay-per-event. website_tech_check is charged once per completed fingerprint scan on robots-permitted pages — $0.05. Volume tiers apply a GOLD-floored −10 / −20 / −33% ladder. apify-actor-start ($0.02) is platform-reserved, never charged in code. Robots-blocked, SSRF-blocked, faulted and unparseable inputs are never billed.