Website Tech Stack & Hosting Detector

what it does

Turn a list of public websites into structured technology observations for prospecting, audits, or market research. Supply up to 1,000 domain names or HTTP(S) URLs. Bare domains use HTTPS; supplied paths are preserved. The actor follows redirects, captures the final URL and HTTP status, and inspects server headers, generator metadata, script URLs, stylesheet links, cookie names, and HTML markers. It does not save cookie values or execute JavaScript.

The vendored, original MIT-licensed fingerprint set covers CMS products, frameworks, analytics, tag managers, CDNs, hosting, commerce, payments, chat, email providers, and marketing tools (211 original product/category rules). Optional network enrichment uses Google DNS-over-HTTPS for MX and Team Cymru ASN DNS records. The networkName field is kept for compatibility and is always null. Failures in optional enrichment are warnings. An ASN identifies the visible edge network; it does not establish where a site's hidden origin is hosted.

The default input contains Wikipedia, Python.org, and WordPress.org, with homepage-only mode enabled. Disabling that mode fetches robots.txt and at most one same-origin internal page permitted for TechStackDetector. Robots failures other than 404/410 prevent the internal fetch. Redirect targets are checked against the same policy. Concurrency is fixed at ten domains.

Use cases

Find CMS migration prospects, identify payment integrations, segment leads by commerce platform, or compare public infrastructure signals across a list of sites. Each submitted entry produces one dataset item, including failures, so downstream workflows can join results back to inputs. Duplicate inputs remain separate entries. Arrays summarize categories while technologies retains the evidence source and confidence for each finding. Unknown technologies produce empty arrays rather than invented results.

• Sales development teams can group prospect domains by observed CMS before assigning migration outreach.
• Web agencies can screen client homepages for visible analytics and tag-manager integrations before an audit.
• Partnership teams can identify commerce-platform signals when building an integration prospect list.
• Infrastructure teams can compare visible CDN and edge-network observations across a managed domain inventory.

sample output

Illustrative abbreviated output, not a measurement of the example domain:

{
  "input": "example.org",
  "domain": "example.org",
  "finalUrl": "https://example.org/",
  "statusCode": 200,
  "success": true,
  "server": "cloudflare",
  "cms": ["WordPress"],
  "cdn": ["Cloudflare"],
  "technologies": [
    {"name": "WordPress", "category": "cms", "confidence": 100,
     "evidence": ["meta.generator"]}
  ],
  "confidence": 100,
  "warnings": [],
  "error": null
}

Confidence ranges from zero to 100 and expresses rule strength, not measured probability. The top-level value is the maximum finding confidence, not a statement that the entire stack was identified. HTTP error pages can still contain useful CDN evidence; inspect success and statusCode first.

pricing

The proposed price is $0.01 per submitted domain entry, including failed lookups. Three defaults cost $0.03 at this event price; 1,000 entries cost $10. The runner calls Actor.charge(event_name='domain-analyzed', count=1) once after storing each item. Local runs do not collect payment. The declaration is in .actor/pay_per_event.json; it is publication configuration, not a claim that the CLI automatically deploys pricing. Before publishing, configure this event in Console and remove synthetic dataset/start charges to preserve the advertised single-event price. No account, token, login, push, or billing configuration was performed for this local build.

FAQ

How do I run it? From this folder in PowerShell, use the existing Python environment (no Apify CLI is required):

$env:APIFY_LOCAL_STORAGE_DIR = Join-Path $PWD 'storage/local-run'
New-Item -ItemType Directory -Force "$env:APIFY_LOCAL_STORAGE_DIR/key_value_stores/default" | Out-Null
Copy-Item INPUT.json "$env:APIFY_LOCAL_STORAGE_DIR/key_value_stores/default/INPUT.json"
.\.venv\Scripts\python.exe -m src
.\.venv\Scripts\python.exe -m unittest discover -s tests -v

On a fresh checkout only, first create the environment with python -m venv .venv and install dependencies with .\.venv\Scripts\python.exe -m pip install -r requirements.txt. Use a fresh storage directory when you want results separated from earlier runs. The SDK reads the local key-value-store INPUT, so copying the requested JSON matters.

Apify stores local dataset output under $env:APIFY_LOCAL_STORAGE_DIR/datasets/default/. The Python module is src. Input defaults also exist in code so an absent local input still uses the three public sites. .actor/input_schema.json provides Store prefill. timeoutSeconds is the per-request socket timeout, from one to 60 seconds; DNS resolution and total domain work can take longer.

Does one failure stop the run? Website and enrichment failures become result errors or warnings. Invalid top-level input and SDK storage/charging failures still fail the run, so infrastructure problems are not disguised as website failures. Dataset storage and charging are sequential, not a single transaction; forced termination between them requires reconciliation.

Where do the fingerprints come from? vendor/PROVENANCE.md describes the original collection and license. It contains 211 independently authored product/category rules, not the full Wappalyzer database. Patterns and their confidence values live in each rule's signals list. A product can appear in multiple relevant categories (for example, Shopify in CMS and commerce). New signals can be added with regression fixtures.

limitations

No headless browser, login, proxy rotation, CAPTCHA solving, or JavaScript execution is included. Consent-gated and dynamically injected tags may be missed. Cookie matching only sees response Set-Cookie names. Sites can spoof headers and metadata. Responses are capped at two MiB. MX reflects the final website hostname, which can have no mail records even when its parent does. Free enrichment endpoints may throttle or fail; ASN data may lag DNS lookups are not cached. Public-address checks are a basic guard, not DNS-rebinding isolation.

Local validation status is documented in VALIDATION.md. The default sites were successfully checked locally with ten additional sites on 2026-09-26. This is not a hosted Store test or validation of production billing. Network availability and website behavior cannot be guaranteed by prefilled input.

Example output

One real dataset row from a platform run on 2026-09-26, trimmed by omitting fields without changing retained values:

{
  "input": "https://www.wordpress.org",
  "finalUrl": "https://wordpress.org/",
  "statusCode": 200,
  "success": true,
  "server": "nginx",
  "metaGenerators": ["WordPress 7.2-alpha-63914"],
  "technologies": [
    {"name": "WordPress", "category": "cms", "confidence": 100, "evidence": ["html", "meta.generator"]},
    {"name": "Automattic (edge network)", "category": "hosting", "confidence": 85, "evidence": ["asn"]},
    {"name": "Google Tag Manager", "category": "tagManagers", "confidence": 95, "evidence": ["html"]}
  ],
  "cms": ["WordPress"],
  "tagManagers": ["Google Tag Manager"],
  "hosting": ["Automattic (edge network)"],
  "network": {"mx": ["smtp1-dca.wordpress.org", "smtp2-dca.wordpress.org"], "asn": 2635, "asnName": "AUTOMATTIC - Automattic, Inc, US"},
  "confidence": 100,
  "error": null
}

Every detection lists the evidence it came from. A site that matches no rule returns empty arrays, which means "nothing recognised", not "uses nothing".

Pricing example: 1,000 submitted domain entries, including failures x $0.01 per domain-analyzed event = $10.00 in event fees, using .actor/pay_per_event.json. Local runs do not bill.