Website Tech-Stack Scanner + Change Alerts

Scan any list of domains, fingerprint each site's technology stack — CMS, framework, analytics, payments, live chat, hosting hints — and get change events every time a site adds or drops a technology:

"site X added Shopify" · "site Y dropped Intercom"

The first run establishes a baseline per domain. Every later run diffs against the stored snapshot and emits added / removed change records, optionally pushed straight to Slack or any webhook. Put it on a weekly Schedule over your prospect or competitor list and the change feed becomes a signal machine: new store platforms, dropped chat vendors, switched analytics — each one is a sales trigger or a competitive insight.

What it does

• Fingerprints 7,500+ technologies across 109 categories using the actively maintained community fork of the Wappalyzer fingerprint corpus (enthec/webappanalyzer), vendored into the actor so scans are fast and self-contained.
• Detects from real signals: response headers, Set-Cookie names, HTML content, script src URLs, inline script content, <meta> tags, the final URL, and statically-evaluable DOM rules (e.g. link[href*='shopify.com']) — all over plain HTTP with Cheerio. No headless browser, so scans are cheap and fast.
• Resolves implication chains: detecting WooCommerce implies WordPress, which implies PHP and MySQL — you get the whole stack, not just the top layer.
• Snapshots + diffs per domain in a named key-value store you control. First scan = baseline (no changes). Every subsequent scan emits precise added / removed events with the previous scan timestamp.
• Alerts: compact Slack summary and/or full JSON payload POSTed to your webhook whenever a run detects changes.
• Polite by design: one page per domain per run, robots.txt respected (Disallow: / for * skips the domain), exponential-backoff retries that honor 429 Retry-After, per-domain sequential requests.

Who it's for

• Agencies & freelancers prospecting by platform ("show me sites that just adopted Shopify / dropped their chat widget").
• Competitive & market intelligence teams tracking rivals' tooling moves week over week.
• Sales teams using technology changes as buying signals (a dropped Intercom is an open door for your chat product).
• Data teams enriching domain lists with current, structured tech-stack data.

Input

Field	Type	Default	Description
domains	array of strings	required	Bare domains (example.com) or full URLs (https://example.com/pricing). Only the hostname is used; each domain's homepage is fetched once per run. Duplicates are deduped, invalid entries are reported and skipped.
storeName	string	stack-detector-snapshots	Named key-value store holding per-domain snapshots between runs. Use different names for independent watchlists (one per client, one per market segment…).
slackWebhookUrl	string (secret)	—	Slack Incoming Webhook. If set and changes are found, a compact per-domain summary is posted.
alertWebhookUrl	string (secret)	—	Any HTTP(S) endpoint. If set and changes are found, the full JSON change payload is POSTed (shape below).
concurrency	integer	10 (max 50)	Domains scanned in parallel. Each domain still receives at most one request at a time.
proxyConfiguration	proxy	Apify Proxy (auto)	Proxy for all outgoing requests. Recommended so scans aren't blocked or rate-limited.

Example:

{
    "domains": ["shopify.com", "wordpress.org", "stripe.com"],
    "storeName": "stack-detector-snapshots",
    "slackWebhookUrl": "https://hooks.slack.com/services/T000/B000/XXXX",
    "concurrency": 10,
    "proxyConfiguration": { "useApifyProxy": true }
}

Output

One dataset record per scanned domain. This is a real record from a real local run (wordpress.org, where the previous snapshot contained an Intercom entry that is no longer detected):

{
    "domain": "wordpress.org",
    "status": "ok",
    "statusCode": 200,
    "finalUrl": "https://wordpress.org/",
    "scannedAt": "2026-08-13T12:42:58.092Z",
    "technologies": [
        { "name": "Google Font API", "category": "Font scripts", "confidence": 100 },
        { "name": "Google Tag Manager", "category": "Tag managers", "confidence": 100 },
        { "name": "Gutenberg", "category": "WordPress plugins", "confidence": 100 },
        { "name": "HSTS", "category": "Security", "confidence": 100 },
        { "name": "MySQL", "category": "Databases", "confidence": 100 },
        { "name": "Nginx", "category": "Web servers", "confidence": 100 },
        { "name": "Open Graph", "category": "Miscellaneous", "confidence": 100 },
        { "name": "PHP", "category": "Programming languages", "confidence": 100 },
        { "name": "Priority Hints", "category": "Performance", "confidence": 100 },
        { "name": "RSS", "category": "Miscellaneous", "confidence": 100 },
        { "name": "WordPress", "category": "CMS", "confidence": 100 },
        { "name": "WordPress Block Editor", "category": "Page builders", "confidence": 100 },
        { "name": "WordPress Site Editor", "category": "Page builders", "confidence": 100 }
    ],
    "technologyNames": "Google Font API, Google Tag Manager, Gutenberg, HSTS, MySQL, Nginx, Open Graph, PHP, Priority Hints, RSS, WordPress, WordPress Block Editor, WordPress Site Editor",
    "changes": [
        {
            "domain": "wordpress.org",
            "type": "removed",
            "technology": "Intercom",
            "category": "Live chat",
            "previousScanAt": "2026-08-13T12:42:18.359Z"
        }
    ],
    "changeSummary": "-Intercom",
    "error": null
}

For comparison, the same build detects Shopify [Ecommerce], Cloudflare [CDN], HTTP/3, HSTS, and Cart Functionality on shopify.com, and Amazon Web Services [PaaS], Amazon S3 [CDN], and Nginx on stripe.com.

Record statuses

status	Meaning	Charged?	Snapshot updated?
ok	Page fetched (HTTP < 400), stack detected, diffed against snapshot	✅	✅
http_error	Site answered with 4xx/5xx after retries (bot wall, outage). Detected tech (often the CDN/WAF) is still reported	❌	❌
unreachable	DNS/TLS/network failure after retries	❌	❌
skipped_robots	robots.txt disallows / for all user agents	❌	❌
failed	Internal error while scanning this domain (never kills the run)	❌	❌

Snapshots are only written on ok, so a temporarily down site can never produce a bogus "removed everything" storm on its next successful scan.

The run also pushes a final (run summary) record with totals, alert outcomes, and the change-artifact location, and stores all changes aggregated as CHANGES-{runId}.json in the run's default key-value store.

Alert payloads

Slack (slackWebhookUrl) receives a compact summary grouped by domain:

:mag: Tech-stack changes detected — 1 change(s) across 1 domain(s) (run local-1786624978092)
• *wordpress.org* — − Intercom (Live chat) _(previous scan 2026-08-13T12:42:18.359Z)_

Generic webhook (alertWebhookUrl) receives the full change list (real captured payload):

{
    "source": "stack-change-detector",
    "runId": "local-1786624978092",
    "runUrl": null,
    "generatedAt": "2026-08-13T12:43:00.208Z",
    "totalChanges": 1,
    "domains": [
        {
            "domain": "wordpress.org",
            "previousScanAt": "2026-08-13T12:42:18.359Z",
            "added": [],
            "removed": ["Intercom (Live chat)"]
        }
    ],
    "changes": [
        {
            "domain": "wordpress.org",
            "type": "removed",
            "technology": "Intercom",
            "category": "Live chat",
            "previousScanAt": "2026-08-13T12:42:18.359Z"
        }
    ]
}

Pricing (pay-per-event)

Event	When it's charged	Price
domain-scanned	Once per successfully scanned domain (status ok), only after its record is stored	$0.002 ($2 per 1,000 scans)
change-alert	Once per change record (technology added or removed), only after the record is stored	$0.002

You are never charged for unreachable domains, HTTP errors, robots-skipped domains, or failures. A weekly scan of a 500-domain watchlist costs about $1/week plus $0.002 per detected change.

The recurring pattern: schedule it

This actor is built to run on a Schedule. Create one in Apify Console (e.g. every Monday 07:00), point it at your prospect or competitor list, and add your Slack webhook. From then on:

1. Monday morning, every domain is re-scanned and diffed against last week.
2. Only changes reach your Slack — no noise when nothing moved.
3. The full history stays queryable: every run's dataset holds the complete stack per domain, and CHANGES-{runId}.json is your machine-readable change feed for downstream automation.

The change feed is the product. A one-off scan tells you what a site runs; the schedule tells you what they just decided.

Three concrete use cases

1. Agency prospecting — Watch 1,000 local-business domains weekly. The Monday Slack digest lists sites that just adopted Shopify or WooCommerce: they're actively investing in e-commerce and are warm leads for design, apps, or CRO services.
2. Competitor monitoring — Track your 20 competitors. When one drops Intercom and adds Zendesk, or swaps Google Analytics for a server-side setup, you hear about it days after it ships — not months later in a case study.
3. Churn/expansion signals for SaaS vendors — Scan your customers' and prospects' domains. A customer removing your script is a churn red flag worth a same-week call; a prospect adding a competitor's widget moves them to nurture.

Honest limitations (and roadmap)

• HTTP-only, by design. The actor evaluates everything detectable from the raw HTML and response headers. Technologies that only reveal themselves at JavaScript runtime (e.g. a global __NEXT_DATA__ variable, runtime-injected pixels) will be missed on sites that leave no static traces — stripe.com, for example, fingerprints as its hosting stack (AWS, S3, Nginx) because its Next.js markers are runtime-only. This keeps scans fast and ~10× cheaper than browser scans.
• Roadmap — "deep mode": an optional headless-browser pass for domains you flag, unlocking JS-runtime rules (js, xhr, DOM properties) from the same corpus. Planned for a future version.
• Homepage only. Tech used exclusively on inner pages (e.g. checkout-only payment scripts) may not appear. Point the input at a specific URL's domain to scan that hostname's root.
• Fingerprints ship with the actor (snapshot of enthec/webappanalyzer) and are refreshed with each actor release via npm run update-fingerprints.

FAQ

What happens on the very first run? Every domain gets a baseline snapshot; no changes are emitted (there is nothing to compare against yet). Changes start with the second run.

How do I reset the baseline for a domain? Delete that domain's key from the snapshot store (storeName, default stack-detector-snapshots) in Apify Console → Storage → Key-value stores, or delete the whole store to reset everything.

Can I keep separate watchlists? Yes — run the actor (or several Schedules) with different storeName values. Snapshots never mix.

Why is a site http_error / why do I only see Cloudflare on it? The site (or its WAF) refused the request. The detected CDN/WAF is reported for free, the snapshot is untouched, and nothing is charged. Using Apify Proxy (default) usually reduces these.

Do confidence changes create change events? No. Only the appearance or disappearance of a technology counts — the feed stays quiet unless something actually changed.

Is scanning polite/legal? The actor fetches one public page per domain per run (plus robots.txt), honors Disallow: / for all-agents, backs off on 429s, and identifies with standard browser headers. It never logs your webhook URLs. Internal/private hosts (localhost, RFC-1918 IPs, .local…) are refused.

A domain in my list was skipped with a validation warning — why? Entries that aren't valid public hostnames (bad URLs, IPv6 literals, single-label names, private IPs) are skipped and listed in the log; the rest of the run proceeds.

Running locally

npm install
npx apify-cli run          # uses storage/key_value_stores/default/INPUT.json
npm test                   # builds + runs the offline test suite (50 tests, saved real fixtures)
npm run update-fingerprints   # refresh src/data/*.json from enthec/webappanalyzer
npm run update-fixtures       # re-capture the real HTML test fixtures (network)

Local runs need no Apify account: charging no-ops with a log line, and proxyConfiguration: { "useApifyProxy": false } (as in the shipped sample input) skips Apify Proxy.

Deployment

apify login
apify push

Then in Apify Console:

1. Open the actor → Monetization → pay-per-event → create the two events exactly as in the pricing table above (domain-scanned at $0.002, change-alert at $0.002).
2. Do a test run with a few domains; check the dataset, then run again to confirm the zero-change steady state.
3. Create a weekly Schedule with your real watchlist + Slack webhook.
4. Publish to the Store with this README as the store page.

Attribution & data license

Technology fingerprints are vendored from enthec/webappanalyzer (the community-maintained fork of the original Wappalyzer corpus), licensed GPL-3.0. This actor's use of the corpus is limited to the JSON fingerprint data in src/data/; see that repository for the data's license terms.

Changelog

0.1.0 (2026-08-13)

• Initial release: 7,575 technology fingerprints, snapshot + diff engine, Slack/webhook change alerts, pay-per-event pricing (domain-scanned, change-alert), robots.txt respect, offline test suite with real-page fixtures.