# Open Tech Stack Detector (`opusnuma/open-tech-stack-detector`) Actor

Find out what any website is built with: CMS, ecommerce, analytics, JS frameworks, CDN, hosting, payments and chat tools. 7,600+ open-source fingerprints. Pay only for successfully analysed sites.

- **URL**: https://apify.com/opusnuma/open-tech-stack-detector.md
- **Developed by:** [Jako Kaups](https://apify.com/opusnuma) (community)
- **Categories:** Developer tools, Lead generation, SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Tech Stack Detector: open Wappalyzer alternative

Find out what any website is built with. Give it a list of domains and get back each site's CMS, ecommerce platform, analytics, JavaScript frameworks, CDN, hosting, payment providers, chat widgets and more, with versions and confidence scores where they can be detected.

- **7,600+ technologies** from the open-source [webappanalyzer](https://github.com/enthec/webappanalyzer) fingerprint database. This is the community-maintained, GPL-3.0 continuation of Wappalyzer's fingerprints after Wappalyzer went closed-source.
- **Fast and light.** It makes one plain HTTP request per site. There's no headless browser and no proxies, and it handles hundreds of sites per minute.
- **You pay only for successful analyses.** Sites that time out, don't resolve, block bots or disallow us in robots.txt cost nothing.
- **Polite and privacy-friendly.** It respects robots.txt and sends an identifiable user agent. It never logs in and collects no personal data. It reads cookie *names* only and never their values.
- **Open source** under GPL-3.0.

### What you can use it for

- **Lead generation for agencies.** Find prospects running an outdated WordPress, a competitor's platform, or no analytics at all.
- **Sales prospecting.** Qualify a list of company domains by stack, e.g. "Shopify stores using Klaviyo" or "sites on HubSpot".
- **Competitor research.** See which frameworks, CDNs, A/B testing and marketing tools your competitors use.
- **Market research.** Measure technology adoption across an industry, a country's top sites, or a startup list.
- **Security and IT inventory.** Get a quick inventory of server software and versions across your own domains. The output includes CPE identifiers you can match against vulnerability feeds.

### How it works

For each website the Actor does the following:

1. It fetches `/robots.txt` once per host. If our user agent (`OpenTechStackBot`) is disallowed from the page, the site is skipped and you aren't charged.
2. It sends **one GET request** to the URL you gave, following redirects (up to 10). Every redirect hop is also checked against that host's robots.txt.
3. It matches the response against the fingerprint database using these signals:

   - the HTML,
   - `<script src>` URLs and inline scripts,
   - `<meta>` tags,
   - inline CSS,
   - visible text,
   - DOM selectors,
   - response headers,
   - cookie names,
   - the final URL,
   - robots.txt.

   It then resolves the `implies`, `excludes` and `requires` relationships between technologies (for example, WordPress implies PHP and MySQL).
4. It writes one dataset row per site, and charges one `site-analyzed` event only if the site answered with HTTP 2xx.

Bare domains (`example.com`) are tried over HTTPS first. If HTTPS fails at the network or TLS level, the Actor retries once over plain HTTP. Network errors and timeouts are retried once. A single bad site never stops the run.

### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `startUrls` | array of strings | (required) | Domains (`example.com`) or URLs (`https://example.com/shop`). Objects like `{ "url": "..." }` are also accepted. Duplicates are removed. |
| `maxConcurrency` | integer | `10` | Sites analysed in parallel (1–100). |
| `requestTimeoutSecs` | integer | `20` | Hard time limit per site, including redirects (3–120). |
| `respectRobotsTxt` | boolean | `true` | Fetch robots.txt, skip disallowed sites, and use robots.txt as a detection signal. |
| `headersOnly` | boolean | `false` | Download only headers and cookie names, not the HTML. Faster, but detects far fewer technologies. |
| `ignoreSslErrors` | boolean | `true` | Analyse sites with expired or self-signed certificates instead of failing them. |
| `selfTest` | boolean | `false` | Monitoring mode: ignore `startUrls`, check reference sites, and fail if quality drops. See [Self-test and monitoring](#self-test-and-monitoring). |
| `selfTestMinPassPercent` | integer | `80` | Share of reference sites (in %) that must pass in self-test mode. |

Example:

```json
{
    "startUrls": ["wordpress.org", "https://www.shopify.com", "nextjs.org", "https://www.drupal.org"],
    "maxConcurrency": 10,
    "requestTimeoutSecs": 20
}
```

### Output

Each website gets one dataset item. The example below comes from a real local run, shortened to the first three technologies:

```json
{
    "url": "wordpress.org",
    "finalUrl": "https://wordpress.org/",
    "domain": "wordpress.org",
    "statusCode": 200,
    "contentType": "text/html; charset=UTF-8",
    "technologies": [
        {
            "name": "WordPress",
            "categories": ["CMS", "Blogs"],
            "version": "7.2",
            "confidence": 100,
            "website": "https://wordpress.org",
            "cpe": "cpe:2.3:a:wordpress:wordpress:*:*:*:*:*:*:*:*",
            "matchedOn": ["dom", "headers:link", "html", "meta:generator", "scriptSrc", "scripts"]
        },
        {
            "name": "WordPress Block Editor",
            "categories": ["Page builders"],
            "version": null,
            "confidence": 100,
            "website": "https://wordpress.org/gutenberg/",
            "cpe": null,
            "matchedOn": ["html"]
        },
        {
            "name": "MySQL",
            "categories": ["Databases"],
            "version": null,
            "confidence": 100,
            "website": "https://mysql.com",
            "cpe": "cpe:2.3:a:mysql:mysql:*:*:*:*:*:*:*:*",
            "matchedOn": ["implied:WordPress"]
        }
    ],
    "technologyNames": ["WordPress 7.2", "WordPress Block Editor", "WordPress Site Editor", "Gutenberg 24.1.0", "MySQL", "PHP", "Nginx", "Google Font API", "Google Tag Manager", "HSTS", "Priority Hints", "Open Graph", "RSS"],
    "technologyCount": 13,
    "error": null,
    "errorCode": null,
    "warning": null,
    "robotsTxt": "allowed",
    "mode": "full",
    "responseTimeMs": 589,
    "charged": true,
    "checkedAt": "2026-10-03T14:21:16.081Z"
}
```

A site that failed is still recorded, and it isn't charged:

```json
{
    "url": "this-domain-does-not-exist-4821.com",
    "finalUrl": null,
    "statusCode": null,
    "technologies": [],
    "error": "Domain does not resolve (ENOTFOUND)",
    "errorCode": "DNS_NOT_FOUND",
    "charged": false,
    "checkedAt": "2026-10-03T14:21:18.407Z"
}
```

The output fields are:

- **`technologies[].confidence`** (0–100): a sum of the matched patterns' confidence, capped at 100. Technologies that are only *implied* inherit the lower of their parent's confidence and the implied confidence.
- **`technologies[].matchedOn`**: which signals triggered the detection, so you can audit any result.
- **`technologyNames`**: a flat `"Name version"` list, handy for spreadsheets and CSV exports.
- **`errorCode`**: one of `INVALID_URL`, `HTTP_ERROR`, `TIMEOUT`, `DNS_NOT_FOUND`, `CONNECTION_FAILED`, `TLS_ERROR`, `TOO_MANY_REDIRECTS`, `INVALID_REDIRECT`, `ROBOTS_DISALLOWED`, `NETWORK_ERROR` or `INTERNAL_ERROR`.
- **`warning`**: non-fatal notes, for example a non-HTML response (only headers were analysed) or a page larger than 3 MB that was truncated.
- **`charged`**: whether this row was billed.

In Apify Console the dataset has two table views:

- **Overview**: one row per site.
- **One row per technology**: handy for filters like "all sites using Shopify".

The key-value store record `RUN_SUMMARY` holds counts of analysed, failed and charged sites plus the fingerprint database version.

### Pricing

This Actor uses **pay-per-event** pricing:

| Event | Price | When it's charged |
|---|---|---|
| `site-analyzed` | **$0.002** ($2 per 1,000 sites) | Once per site whose homepage answered with HTTP 2xx and was analysed. |

**Not charged:**

- invalid URLs,
- DNS failures,
- timeouts, connection and TLS failures,
- redirect loops,
- sites that disallow us in robots.txt,
- non-2xx responses such as 403 bot walls, 404s and 5xx errors.

Every row has a `charged` field so you can check your bill against your data.

Set a **maximum cost per run** in Apify Console or through the API. The Actor stops cleanly when the limit is reached and never pushes results it hasn't charged for.

### Self-test and monitoring

Set `selfTest: true` (preset: [`presets/self-test-input.json`](presets/self-test-input.json)) and the Actor runs two checks:

1. **An offline fixture check.** A synthetic page with known markers must yield WordPress, PHP, Nginx, jQuery, Google Analytics and MySQL. This catches a broken engine or a corrupted fingerprint database independently of the network.
2. **Reference sites.** These are wordpress.org, jquery.com, nextjs.org, drupal.org, joomla.org, shopify.com, cloudflare.com and php.net. Each one must show its expected technologies.

The run **fails** (exit code 1, status `FAILED`) if the offline check fails or fewer than `selfTestMinPassPercent` % of the reference sites pass. A full report is saved as `SELF_TEST_REPORT` in the key-value store. Self-test runs charge nothing.

To monitor daily on Apify:

1. Create a **Task** for the Actor with input `{ "selfTest": true }`.
2. Add a **Schedule** that runs it daily, for example `0 6 * * *`.
3. In **Settings → Notifications**, enable email alerts for failed runs. Or attach a webhook to the task for the `ACTOR.RUN.FAILED` event.

If a reference site legitimately changes its stack, update `SELF_TEST_SITES` in [`src/self-test.ts`](src/self-test.ts).

### Limitations

- **No JavaScript execution.** About 1,050 of the 7,628 fingerprints have no static signal at all; they rely on browser JavaScript globals (`window.*`), XHR requests or DNS. Those technologies show up only when another detected technology implies them. Another ~2,300 fingerprints combine JS patterns with static ones, and only the static part is used. Most popular tools leave static traces. Single-page apps that render everything client-side reveal less. A small curated supplement ([`data/supplements.json`](data/supplements.json)) adds static patterns for some popular tools that upstream detects only through JS, such as Next.js, Remix and Drift.
- **No DNS, TLS-certificate or extra-page probes.** Detection is limited to the homepage response: no MX/TXT records, no certificate issuer, no extra paths.
- **Bot protection.** Some large sites (news publishers, big marketplaces) answer non-browser clients with HTTP 403. Those rows are reported as `HTTP_ERROR` and aren't charged. Any technologies listed for them describe the protection page, typically Cloudflare or Akamai.
- **Only one page per site.** Technologies used only on inner pages, such as a checkout provider, may be missed.
- **Cookie values are never read.** So the few fingerprints that need a cookie's value can't match.
- **Pages are cut at 3 MB.** Larger HTML is truncated, and the row gets a `warning`.
- **Versions** are reported only when the site exposes them, for example in a `generator` meta tag, `X-Powered-By` or versioned script URLs.

In a 40-site local test against well-known homepages, 35 sites were analysed successfully. Four returned bot-protection 403s and one disallowed bots in robots.txt.

### Responsible use

- The user agent is `Mozilla/5.0 (compatible; OpenTechStackBot/1.0; +<info URL>)`. Site owners can block it in robots.txt with `User-agent: OpenTechStackBot`.
- Each site gets at most one robots.txt request plus one page request (with redirects), and one retry on network errors.
- No login, no form submission, no proxies, no attempts to bypass bot protection.
- No personal data is collected. The output contains only technology names, public URLs, HTTP status and header-derived facts.

***

### Development

#### Requirements

- Node.js 20.18+ (22 recommended)
- npm

#### Commands

```bash
npm ci                          # install pinned dependencies
npm run build                   # compile TypeScript to dist/
npm test                        # unit tests (fixtures + local HTTP server, no internet needed)
npm run test:integration        # live tests against 5 public sites; skipped offline or with SKIP_NETWORK_TESTS=1
npm run typecheck               # type-check src, tests and scripts
npm run fingerprints:update     # refresh data/ from upstream webappanalyzer (optionally pass a git ref)
```

#### Run locally

With the [Apify CLI](https://docs.apify.com/cli) (`npm i -g apify-cli`):

```bash
npm run build
apify run -p --input-file presets/sample-input.json
## results:  storage/datasets/default/*.json
## summary:  storage/key_value_stores/default/RUN_SUMMARY.json
```

Without the CLI, put the input in `storage/key_value_stores/default/INPUT.json` and run `npm start`.

To simulate pay-per-event charging locally:

```bash
ACTOR_TEST_PAY_PER_EVENT=true ACTOR_USE_CHARGING_LOG_DATASET=true \
  apify run -p --input-file presets/sample-input.json
## each charge is logged to storage/datasets/charging_log/
```

Add `ACTOR_MAX_TOTAL_CHARGE_USD=2` to test the spending cap. Locally every event costs $1, so the run stops after 2 charged sites.

#### Project layout

```
.actor/            actor.json, input/dataset/output schemas, Dockerfile
data/              vendored fingerprint DB (GPL-3.0, from webappanalyzer) + SOURCE.json + supplements.json
presets/           sample input and self-test input
scripts/           update-fingerprints.ts
src/engine/        fingerprint compiler, signal extraction, matcher (implies/excludes/requires)
src/http/          HTTP fetcher (redirects, decompression, charset, limits) and robots.txt cache
src/analyze-site.ts  one site → one result
src/main.ts        Actor entry point (Crawlee BasicCrawler, charging, self-test)
test/              unit tests, HTML fixtures, live integration test
```

#### Updating fingerprints

`npm run fingerprints:update` downloads `src/technologies/*.json` and `src/categories.json` from [enthec/webappanalyzer](https://github.com/enthec/webappanalyzer). It resolves the current commit SHA, strips fields that aren't used for detection (icons, descriptions, pricing), checks that every pattern compiles, and writes `data/technologies.json`, `data/categories.json` and `data/SOURCE.json`. It refuses to overwrite data/ if fewer than 5,000 technologies are downloaded.

After updating:

1. Run `npm test`.
2. Run a self-test (`apify run -p --input-file presets/self-test-input.json`).
3. Commit, then `apify push`.

### Publishing to Apify

You need an Apify account and an API token from **Console → Settings → API & Integrations**.

```bash
npm i -g apify-cli          # once
apify login                 # paste your API token when prompted (or: apify login -t <TOKEN>)
apify push                  # uploads this directory and builds it on Apify using .actor/Dockerfile
```

`apify push` creates the Actor `open-tech-stack-detector` under your account, or updates it if it already exists. Then:

1. **Before setting up monetization, update the bot info URL.** Edit `BOT_INFO_URL` in `src/constants.ts` to point at the Actor's public Store page, then `apify push` again. This lets site owners identify the bot.
2. **Run it once in Console** with the prefilled input, and check the Output tab shows both dataset views.
3. **Make the source public.** The fingerprint database is GPL-3.0, so link this repository from the Store listing (Actor → Settings → Source / Git repository). Also keep the license section of this README.

#### Set up pay-per-event pricing

Pricing is configured in Apify Console, not in code.

1. Open the Actor → **Publication** → **Monetization** and choose **Pay per event**.
2. Add a custom event:
   - **Event name:** `site-analyzed` (must match `CHARGE_EVENT` in `src/constants.ts`)
   - **Title:** Site analyzed
   - **Description:** One website homepage fetched and analysed successfully (HTTP 2xx). Failed sites are free.
   - **Price:** `$0.002`
   - Mark it as the **primary event**.
3. **Remove the `apify-default-dataset-item` synthetic event.** Apify enables it by default, and it would charge for *every* dataset row, including failed sites, which this Actor promises are free.
4. **Keep the `apify-actor-start` synthetic event** at its default price ($0.00005 per start per GB). Apify then covers the first 5 seconds of compute for each run.
5. Optionally, set a minimum for the per-run spending limit (`minimalMaxTotalChargeUsd`), e.g. $0.01.
6. Review and publish. Read Apify's [pay-per-event docs](https://docs.apify.com/platform/actors/publishing/monetize/pay-per-event) before going live.

The code charges through `Actor.pushData(result, 'site-analyzed')`. That call pushes the row and charges the event together, and it respects the user's spending limit. If no pay-per-event pricing is configured (local runs, or before monetization is set up), nothing is charged and `charged` is `false`.

**Rough economics** (an estimate, not measured on the platform): locally, a 40-site run took about 0.2 CPU-seconds per site. At the default 1 GB memory (a quarter of a CPU core on Apify), that's roughly 1 second of compute per site, or about 0.3 compute units per 1,000 sites. Revenue at $0.002 is $2.00 per 1,000 sites, before Apify's 20% share.

### License and attribution

This Actor is free software, licensed under the **GNU General Public License v3.0** ([LICENSE](LICENSE)).

- Technology fingerprints and categories in [`data/`](data) come from [**enthec/webappanalyzer**](https://github.com/enthec/webappanalyzer), licensed GPL-3.0. That project continues the fingerprints originally created by [Wappalyzer](https://github.com/wappalyzer/wappalyzer) and its contributors. The exact upstream commit is recorded in [`data/SOURCE.json`](data/SOURCE.json). The JSON was minified, and icon, description and pricing fields were removed. Nothing else was changed.
- [`data/supplements.json`](data/supplements.json) contains additional patterns written for this project, also under GPL-3.0.
- The detection engine (`src/engine`) is an independent TypeScript implementation of the Wappalyzer fingerprint format.

"Wappalyzer" is a trademark of its owner. This project isn't affiliated with or endorsed by Wappalyzer or Enthec.

# Changelog

This Actor's version history is a separate document: https://apify.com/opusnuma/open-tech-stack-detector/changelog.md

# Actor input Schema

## `startUrls` (type: `array`):

Domains (example.com) or full URLs (https://example.com/shop). One per line. Bare domains are tried over HTTPS first, then HTTP. Duplicates are removed. Each entry is fetched with a single GET request.

## `maxConcurrency` (type: `integer`):

Maximum number of sites analysed in parallel. Higher is faster; 10-30 is plenty for most runs.

## `requestTimeoutSecs` (type: `integer`):

Hard limit for downloading one site's homepage, including redirects. Slow sites are retried once, then reported as failed (and not charged).

## `respectRobotsTxt` (type: `boolean`):

Fetch each site's robots.txt and skip sites that disallow our user agent (OpenTechStackBot) from the homepage. The robots.txt content is also used as a detection signal. Skipped sites are not charged.

## `headersOnly` (type: `boolean`):

Only download response headers and cookie names; skip the HTML body. Faster and lighter, but detects far fewer technologies (servers, CDNs, hosting, some frameworks).

## `ignoreSslErrors` (type: `boolean`):

Analyse sites with expired or self-signed certificates instead of failing them. Only public homepage HTML is read, so this is safe for detection purposes.

## `selfTest` (type: `boolean`):

Ignore 'Websites' and analyse a fixed set of well-known reference sites, then compare against expected technologies. The run FAILS if detection quality drops below the threshold. Nothing is charged. Schedule a task with this enabled to monitor the Actor daily.

## `selfTestMinPassPercent` (type: `integer`):

Percentage of reference sites that must be detected correctly for the self-test to pass. An offline fixture check must always pass regardless.

## Actor input object example

```json
{
  "startUrls": [
    "wordpress.org",
    "https://www.shopify.com",
    "nextjs.org",
    "https://www.drupal.org"
  ],
  "maxConcurrency": 10,
  "requestTimeoutSecs": 20,
  "respectRobotsTxt": true,
  "headersOnly": false,
  "ignoreSslErrors": true,
  "selfTest": false,
  "selfTestMinPassPercent": 80
}
```

# Actor output Schema

## `results` (type: `string`):

One item per website with detected technologies, versions, confidence and errors.

## `technologies` (type: `string`):

The same data with one row per detected technology.

## `runSummary` (type: `string`):

Counts of analysed, failed and charged sites plus the fingerprint database version.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "wordpress.org",
        "https://www.shopify.com",
        "nextjs.org",
        "https://www.drupal.org"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("opusnuma/open-tech-stack-detector").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [
        "wordpress.org",
        "https://www.shopify.com",
        "nextjs.org",
        "https://www.drupal.org",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("opusnuma/open-tech-stack-detector").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "wordpress.org",
    "https://www.shopify.com",
    "nextjs.org",
    "https://www.drupal.org"
  ]
}' |
apify call opusnuma/open-tech-stack-detector --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,opusnuma/open-tech-stack-detector"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/6mOEqCThY6fNIQtnT/builds/fSAr7WMMUl8dcjDNQ/openapi.json