# Website Tech Stack Scraper & Change Monitor (`automa-flow/website-tech-stack-monitor`) Actor Detect website technologies in bulk and monitor when prospects or competitors add, remove, or switch SaaS tools. Get deterministic fingerprints, confirmed-removal semantics, CRM IDs, DNS/TLS enrichment, and explicit per-domain failures without a BuiltWith or Wappalyzer account. - **URL**: https://apify.com/automa-flow/website-tech-stack-monitor.md - **Developed by:** [Vadim Bezrukov](https://apify.com/automa-flow) (community) - **Categories:** Developer tools, Lead generation, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing $1.00 / 1,000 successful domain checks This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Website Tech Stack Scraper & Change Monitor Detect technologies used by websites in bulk and monitor when prospects or competitors add, remove, or switch SaaS tools. Use it as a website technology checker, technology stack detector API, technographic data feed, or confirmed technology change monitor. The Actor analyzes each target website directly. It does **not** scrape BuiltWith or Wappalyzer, does not require an account with either service, and is not affiliated with them. The detector uses a small independently authored set of high-confidence signatures rather than a large third-party fingerprint dump. ### What it returns Every requested domain gets exactly one record, including invalid, blocked, and failed inputs. A successful record contains: - the normalized current technology stack with category, version when explicit, confidence, and evidence surface names; - optional normalized MX/nameserver providers and TLS certificate metadata; - a deterministic semantic fingerprint and versioned detector metadata; - `BASELINE`, `UNCHANGED`, `TECH_ADDED`, `MISSING_PENDING`, `TECH_REMOVED_CONFIRMED`, or `TECH_VERSION_CHANGED` events; - your own `external_id` for CRM/account/watchlist joins. `technologies: []` with `SUCCESS` means a complete public page was fetched and none of the supported high-confidence signatures matched. A timeout, 403, challenge page, invalid response, or truncated page is never converted into that answer; it is `FAILED`, `BLOCKED`, or `PARTIAL`. If a pay-per-event run reaches the user's maximum charge, untouched inputs receive `SKIPPED_BILLING_LIMIT` and their monitor state is retained. ### Pricing The Actor costs **$0.001 per distinct complete `SUCCESS` domain check** — $1 per 1,000 successful websites. Platform usage is included and there is no start fee. Snapshot and monitor checks have the same predictable price, including an `UNCHANGED` result because a fresh live verification was performed. Duplicate aliases, retries, `PARTIAL`, `FAILED`, `BLOCKED`, `INVALID_DOMAIN`, and `SKIPPED_BILLING_LIMIT` are free. DNS queries, TLS enrichment and emitted change events are not separate billable events. The Actor charges while the successful row is stored; monitor state advances only after that row was delivered and billed. ### Ready-to-run examples - [Website Tech Stack Detector for Sales Leads](https://apify.com/automa-flow/website-tech-stack-monitor/examples/detect-website-tech-stacks-for-sales-leads) enriches prospect domains and preserves CRM IDs. - [Competitor Technology Stack Change Monitor](https://apify.com/automa-flow/website-tech-stack-monitor/examples/monitor-competitor-technology-stack-changes) creates a reusable baseline and emits confirmed change events. - [Shopify & Ecommerce Platform Migration Monitor](https://apify.com/automa-flow/website-tech-stack-monitor/examples/monitor-ecommerce-platform-migrations) watches supported commerce and CMS platform changes. - [Website SaaS Adoption Signals for Sales Teams](https://apify.com/automa-flow/website-tech-stack-monitor/examples/monitor-saas-adoption-signals-for-sales) turns supported public SaaS additions into sales signals. - [Email Hosting & DNS Provider Checker for Domains](https://apify.com/automa-flow/website-tech-stack-monitor/examples/check-email-and-dns-providers) resolves normalized MX and nameserver provider signals. ### Quick start: one-domain lookup ```json { "domains": [{ "domain": "example.com" }], "mode": "snapshot", "outputMode": "all", "includeDns": true, "includeTls": false } ``` Run from the CLI after the Actor is deployed: ```bash apify call YOUR_USERNAME/website-tech-stack-monitor --input-file input.json ``` The full input and representative output are in `examples/`. ### CRM domain batch ```json { "domains": [ { "domain": "prospect-a.com", "externalId": "salesforce-001" }, { "domain": "prospect-b.com", "externalId": "salesforce-002" }, { "domain": "prospect-c.com", "externalId": "salesforce-003" } ], "mode": "snapshot", "outputMode": "all" } ``` The Actor fetches duplicate normalized domains once, but still emits one row per input so every external ID receives an answer. ### Competitor watchlist ```json { "domains": [ { "domain": "competitor-a.com", "externalId": "competitor-a" }, { "domain": "competitor-b.com", "externalId": "competitor-b" } ], "mode": "monitor", "outputMode": "changesOnly", "watchTechnologies": ["Shopify", "WooCommerce", "Magento", "HubSpot", "Klaviyo"], "includeDns": true } ``` The current stack is always present because the per-domain status row must remain unambiguous. `changesOnly` suppresses the redundant `UNCHANGED` object inside `events`, while keeping the main row with `change_type: UNCHANGED`; it never suppresses failure or invalid-domain rows. This preserves the batch invariant: 100 inputs produce 100 explicit statuses. `all` includes the explicit `UNCHANGED` event as well. ### Sales and competitive signals #### Shopify migration A switch from Shopify to another platform is reported as exact raw events, not an invented intent score: ```json [ { "type": "TECH_ADDED", "technology": "Magento", "before": null, "after": { "name": "Magento" } }, { "type": "TECH_REMOVED_CONFIRMED", "technology": "Shopify", "before": { "name": "Shopify" }, "after": null, "missing_successful_scans": 2 } ] ``` The events may arrive on different runs: addition is immediate on strong evidence, while removal needs two independent complete successful missing scans. #### HubSpot or Klaviyo adoption Use `watchTechnologies: ["HubSpot", "Klaviyo"]`. A new strong vendor-script or cookie signal produces `TECH_ADDED` with exact `after` technology data. That row can go directly to a CRM webhook as an adoption trigger. ### Safe change semantics Monitor state lives only in Apify KVS and is keyed by normalized domain plus the relevant scan configuration. It stores the last confirmed technology set, compact evidence/confidence, missing counters, fingerprint, detector versions, and observation time. ```text present -> first complete successful scan missing -> MISSING_PENDING -> second complete successful scan missing -> TECH_REMOVED_CONFIRMED ``` `FAILED`, `BLOCKED`, and `PARTIAL` scans never increment the counter and never overwrite good state. If the detector or signature database version changes, the first complete run emits `DETECTOR_REBASELINE` and adopts a fresh baseline; it does not manufacture a wave of additions/removals. ### Daily schedule First save the monitor input as an Apify Task and test it. Then create a daily 08:00 schedule with the API (replace placeholders; keep the token in an environment variable, never in source control): ```bash curl -X POST "https://api.apify.com/v2/schedules" \ -H "Authorization: Bearer $APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "name": "Daily website technology watchlist", "isEnabled": true, "isExclusive": true, "cronExpression": "0 8 * * *", "timezone": "UTC", "actions": [{ "type": "RUN_ACTOR_TASK", "actorTaskId": "YOUR_TASK_ID" }] }' ``` Apify schedule syntax and timezone behavior are documented at . ### Webhook sales trigger Create a run-succeeded webhook on the saved Task. The payload carries the run and dataset IDs; your receiver then reads dataset rows whose `events` contain `TECH_ADDED`, `TECH_REMOVED_CONFIRMED`, or `TECH_VERSION_CHANGED`: ```bash curl -X POST "https://api.apify.com/v2/webhooks" \ -H "Authorization: Bearer $APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "requestUrl": "https://crm.example/webhooks/technology-change", "eventTypes": ["ACTOR.RUN.SUCCEEDED"], "condition": { "actorTaskId": "YOUR_TASK_ID" }, "payloadTemplate": "{\"run_id\":\"{{resource.id}}\",\"dataset_id\":\"{{resource.defaultDatasetId}}\"}", "shouldInterpolateStrings": true, "description": "Website technology sales signals" }' ``` Webhook behavior is documented at . ### Use with AI agents through Apify MCP Expose the Actor as a typed MCP tool: ```text https://mcp.apify.com?tools=automa-flow/website-tech-stack-monitor ``` Example prompt: ```text Run automa-flow/website-tech-stack-monitor for these company domains. Return the detected ecommerce, analytics, CRM, email, and infrastructure technologies, separate FAILED or PARTIAL scans, and cite each source_url and evidence surface. ``` For recurring sales or competitor workflows, use `mode: monitor` and act only on confirmed `TECH_ADDED`, `TECH_VERSION_CHANGED`, and `TECH_REMOVED_CONFIRMED` events. A failed scan never confirms a removal. ### Detection surfaces and supported scope The default path is one public homepage fetch. The Actor inspects final URL, HTTP headers, Set-Cookie names, generator/meta values, script/link URLs, and a small number of explicit inline identifiers. Optional DNS makes two queries (MX and NS). TLS is off by default. Raw HTML, cookie values, arbitrary TXT records, personal contacts, and giant evidence snippets are never returned. Every initial target and redirect hop is resolved once and the connection is made to that exact verified public IP while retaining the original HTTP `Host` and TLS SNI. The optional proxy CONNECT target is pinned the same way. Private, loopback, link-local, and other non-public addresses are rejected before any request bytes are sent; transient resolver failures remain explicit failures rather than being mislabeled as invalid input. The curated MVP covers 40+ commercially important technologies across CMS, ecommerce, frameworks, analytics, advertising, CRM/marketing automation, payments, live chat, CDN/hosting, and consent management. The exact selectable names are listed in the Input Schema. ### Limitations - No JavaScript execution: runtime-only tools may be missed. - Normally one page: technologies absent from the homepage are not discovered. - Consent, geo/A-B variants, and personalized templates can hide scripts. This is why removals require confirmation. - Confidence is a deterministic rule strength, not a machine-learning probability and not a claim of complete coverage. - Direct HTTP is normal. An optional Apify datacenter proxy is attempted only after a failure/block when explicitly configured. Residential proxy, CAPTCHA solving, login, and access-control circumvention are out of scope. - Fingerprints are intentionally curated for precision. This Actor will detect fewer obscure technologies than a multi-thousand-rule browser database. ### Economics and benchmark The 2026-09-01 direct-HTTP PoC tested 100 diverse domains. At concurrency 10 it completed in 18.4 seconds: 81 `SUCCESS`, 11 `BLOCKED`, 8 transport failures, about 52.1 MB inspected, one normal request/domain, no browser, and no proxy. Concurrency 25/50 did not improve useful throughput, so the Actor caps work at 10 domains. Full research and raw per-domain measurements are under `experiments/tech-stack/` in the source repository. An official Tranco top-1,000 economics run completed in 154.5 seconds, inspected 197.4 MB, and measured 72.7 MB of peak traced Python allocations: 525 web successes, 89 blocks, and 386 failures/non-web endpoints. It made exactly 1,000 homepage requests and used no proxy/browser. Optional DNS adds two queries per successfully fetched domain; TLS remains opt-in. The pay-per-event contract bills only a distinct complete `SUCCESS` domain check. Charging per technology change would make quiet daily monitors unpredictable, so exact change events remain included in the domain-check price. ### Legal and responsible use Source-risk classification is **MODERATE** because every target domain has its own terms. The Actor accesses only public homepage technical signals, normally once per check. It does not log in, visit admin/private paths, bypass access controls, collect personal contacts, or build a copy of target content. Users must ensure their domain list and schedule comply with applicable terms and law. ### Output contract Records are history-ready from the first run: stable `source_id`, UTC `scraped_at`, additive `schema_version`, semantic `fingerprint`, `detector_version`, and `fingerprint_database_version`. Datasets remain append-only observations. The Actor does not build a historical database; KVS holds only compact monitoring state. # Actor input Schema ## `domains` (type: `array`): One to 500 bare public domains. externalId is echoed into the result for CRM and watchlist joins and is never sent to the target. ## `mode` (type: `string`): snapshot returns the current observed stack without state. monitor compares complete scans with the last successful state in Apify KVS. ## `outputMode` (type: `string`): Every input always receives one status row. changesOnly emphasizes webhook-ready events; all is intended for full stack exports. The stable main record shape is the same in both modes. ## `watchTechnologies` (type: `array`): Optional monitoring filter. The current stack still includes every detected technology, while events and state focus on these names. Leave empty to monitor all supported technologies. Changing this starts a separate baseline. ## `includeDns` (type: `boolean`): Resolve MX and nameserver records and return only normalized known providers. Required when watchTechnologies contains an email or DNS provider. Arbitrary TXT values and raw DNS records are not exposed. A DNS failure makes the domain PARTIAL and does not advance monitor state. ## `includeTls` (type: `boolean`): Include normalized issuer, subject, and expiry metadata. Off by default because TLS is not needed for core technology detection. A TLS failure makes the scan PARTIAL. ## `proxyConfiguration` (type: `object`): Optional Apify proxy fallback after direct HTTP is blocked or fails. Direct HTTP is always tried first. Residential proxy escalation is not automatic. ## Actor input object example ```json { "domains": [ { "domain": "example.com", "externalId": "account-001" }, { "domain": "shop.example.org", "externalId": "account-002" } ], "mode": "monitor", "outputMode": "changesOnly", "watchTechnologies": [], "includeDns": true, "includeTls": false, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `results` (type: `string`): No description ## `changes` (type: `string`): Change-oriented projection. Filter events by type in your webhook or downstream workflow. ## `failures` (type: `string`): No description ## `runSummary` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "domains": [ { "domain": "example.com", "externalId": "account-001" }, { "domain": "shop.example.org", "externalId": "account-002" } ], "proxyConfiguration": { "useApifyProxy": false } }; // Run the Actor and wait for it to finish const run = await client.actor("automa-flow/website-tech-stack-monitor").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "domains": [ { "domain": "example.com", "externalId": "account-001", }, { "domain": "shop.example.org", "externalId": "account-002", }, ], "proxyConfiguration": { "useApifyProxy": False }, } # Run the Actor and wait for it to finish run = client.actor("automa-flow/website-tech-stack-monitor").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "domains": [ { "domain": "example.com", "externalId": "account-001" }, { "domain": "shop.example.org", "externalId": "account-002" } ], "proxyConfiguration": { "useApifyProxy": false } }' | apify call automa-flow/website-tech-stack-monitor --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,automa-flow/website-tech-stack-monitor" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/sVe32u8ehPO03SHdo/builds/CMBXjOvSP1Msp9qGu/openapi.json