# Website Technology Lookup: CMS, Framework and Hosting Detector (`madrasco/website-tech-stack-detector`) Actor

Find which CMS, ecommerce platform, JavaScript framework, analytics, CDN and hosting a website uses. One polite HTTP request per URL, with evidence for each finding. Accepts URLs or bare domains; works from the API and AI agents (MCP). Honours robots.txt; blocked sites are reported, not bypassed.

- **URL**: https://apify.com/madrasco/website-tech-stack-detector.md
- **Developed by:** [Madrasco](https://apify.com/madrasco) (community)
- **Categories:** Developer tools, SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Website Tech Stack Detector

Give it a list of website URLs; get back the technologies each site visibly uses: CMS, ecommerce platform, JavaScript frameworks and libraries, analytics and tag managers, CDN, hosting, web server, payment and chat widgets, and more — with the evidence for each finding.

### What it does

For each URL you give, the actor makes **one ordinary HTTP request** to that page (after reading the site's `robots.txt` once; redirects are followed) and looks at:

- response headers (for example `Server`, `X-Powered-By`, CDN headers)
- cookie **names** set by the page (values are discarded immediately and never stored)
- `<meta name="generator">` and similar meta tags
- `<script src>` and `<link href>` URLs
- characteristic patterns in the HTML

It matches these against about 170 hand-written signatures and reports each technology with its category, version (when the page states it) and what matched.

### What it does not do

- **No browser.** JavaScript is not executed, so technologies that are only loaded later by scripts, or only appear after login or consent, are not seen.
- **No bypassing of blocks.** If a site answers with a recognised bot challenge or refusal — HTTP 403 or 429, HTTP 401 or 503 with a known challenge page, or Cloudflare's challenge header — the URL is reported as `blocked` (with any facts visible in the response headers, e.g. the CDN) and is **not retried** through proxies or other tricks. It will stay free when per-website pricing is introduced (see Pricing). A challenge page that is served with HTTP 200 and no challenge header may not be recognised; it is then counted as analysed (and, once per-website pricing is introduced, charged), usually with few or no technologies found.
- **robots.txt is honoured** by default, including for the target of a redirect (a redirect to another site is checked against that site's `robots.txt`). Disallowed pages are reported as `disallowed-by-robots` and not fetched. A `Crawl-delay` in `robots.txt` is respected between requests to the same site (waits are capped at 10 seconds). You can switch robots.txt handling off only for sites you own or have permission to test.
- **No crawling.** Only the URLs you list (and any redirects they answer with) are fetched, at most one request at a time per host.
- It is not a security scanner and does not probe for vulnerabilities.
- Detection is passive and can miss technologies that leave no trace in the first HTML response (for example a Laravel or Django site whose home page sets no framework cookie and has no form). It can also be fooled by sites that deliberately mimic or hide markers.

### Accuracy (measured by us, September 2026)

These figures apply only to this version of the detector (the one described under "Changes in this version"); earlier or later versions can score differently.

We tested this version on **197 websites** we had never tuned on: 150 randomly sampled from the Tranco top-sites list (popular, mid-ranked and long-tail sites) and 47 from vendor showcase pages. We left out sites that block bots, fail to load or disallow crawling in robots.txt. Reference labels came from **separate blind AI labelling passes (the labellers could not see the detector or each other's labels)** over a full browser load of each home page. Every label cites evidence a script re-checked. Where the passes disagreed, a third AI pass decided. **The labels were not reviewed by a person.** We ran this version's detector on saved copies of each home page's first response, captured with a browser on 25–26 September 2026, not on live actor runs. A site may answer the actor's own request differently.

- For the six categories we scored (CMS, e-commerce, framework, analytics, CDN, hosting), when the actor reported a technology it matched the reference labels **96%** of the time (707 of 736; 95% interval 94–97%). Other things it reports (web servers, languages, fonts, payments and so on) were not scored.
- Of the technologies visible without running JavaScript (the home page's first response headers, cookies and HTML, or its DNS records; the actor doesn't read DNS records, so those count as misses), it found **72%** (646 of 891). Counting everything a full browser load reveals, including tags loaded later by JavaScript, it found **56%** (707 of 1262). The actor makes one plain request and does not run JavaScript, so it misses those.
- By category (correct when reported / found of those visible without JavaScript): CMS 96% / 79%, framework 99% / 91%, CDN 99% / 80%, analytics 92% / 62%, hosting 92% / 62%, e-commerce 100% / 71% (only 18 e-commerce labels, so treat that one loosely; hosting could be decided on only 89 of the 197 sites).
- **Developer test set:** 31 saved page snapshots (headers + HTML, including decoy pages that mention products in plain text) with hand-assigned labels: precision 1.00, recall 1.00 over 141 labels. These snapshots were written by the same developer as the signatures, so treat this as an upper bound, not a promise.

Results on your sites will differ. Treat a missing technology as "not seen", not "not used", and check important results by hand.

### Changes in this version

- `assets.adobedtm.com` is now reported as **Adobe Launch** (tag manager), not Adobe Analytics; Adobe Analytics is reported only from its own `AppMeasurement` script.
- New: **HubSpot Analytics** (the HubSpot tracking script and cookies).
- `preconnect` and `dns-prefetch` links no longer count as evidence, so, for example, Google Fonts isn't reported from a preconnect link alone.
- WordPress is no longer detected from `/wp-content/uploads/` links alone (those often point to media on another WordPress site).
- The HTML scan limit rose from 1 million to 4 million characters, so tags near the end of very large pages are seen.
- 20 new signatures in all (173 in total), mostly analytics tools, hosting providers and public library CDNs.

### Input

| Field | Default | Meaning |
|---|---|---|
| `urls` | — | Up to 1,000 URLs (bare domains are fetched over HTTPS). |
| `startUrls`, `domains`, `websites`, `url` | — | Other names for the same thing, for API callers and AI agents: `startUrls` takes Apify's request-list format (`[{"url": "..."}]`) or plain strings, `url` takes one string. All given URL fields are merged and duplicates removed. |
| `respectRobotsTxt` | `true` | Skip pages disallowed by robots.txt. |
| `timeoutSecs` | `20` | Give up on a page after this many seconds. |
| `maxConcurrency` | `10` | Different sites fetched in parallel (never more than one request at a time per host). |
| `includeEvidence` | `true` | Include what matched for each technology. |

### Use with AI agents

The only input the actor needs is a list of websites. Any of the field names above works, so an agent that guesses `startUrls` or `url` still gets results; if no URL is given, the run fails with a message naming the accepted fields.

- **Apify MCP server:** add `https://mcp.apify.com?tools=madrasco/website-tech-stack-detector` to your MCP client (Claude, Cursor and others; see [Apify's MCP docs](https://docs.apify.com/platform/integrations/mcp)). The agent then has a tool for this actor and calls it with, for example, `{"urls": ["example.com", "https://shop.example.org"]}`.
- **HTTP API, one call:** `POST https://api.apify.com/v2/acts/madrasco~website-tech-stack-detector/run-sync-get-dataset-items` with your Apify token in the `Authorization: Bearer` header and a JSON body such as `{"urls": ["example.com"]}` returns the dataset rows (one per URL) when the run finishes. For long lists, start a normal run and read the dataset afterwards.

Each row carries `status`, so an agent can tell "no technologies found" apart from `blocked`, `disallowed-by-robots` and `error`.

### Output (one dataset row per URL)

Real output for `https://wordpress.org/` (run by us on 2026-09-24), **shortened**: 3 of the 7 technologies found are shown, the long evidence list for WordPress is cut to two entries, and `categories` lists only the shown technologies.

```json
{
  "url": "https://wordpress.org/",
  "finalUrl": "https://wordpress.org/",
  "httpStatus": 200,
  "status": "ok",
  "error": null,
  "technologies": [
    {"name": "Nginx", "categories": ["Web servers", "Reverse proxies"], "version": null,
     "evidence": ["header: server: nginx"], "confidence": "direct"},
    {"name": "PHP", "categories": ["Programming languages"], "version": null,
     "evidence": ["implied by WordPress"], "confidence": "implied"},
    {"name": "WordPress", "categories": ["CMS", "Blogs"], "version": "7.2",
     "evidence": ["meta: generator=WordPress 7.2-alpha-63914", "html: /wp-content/plugins/"],
     "confidence": "direct"}
  ],
  "categories": {"Web servers": ["Nginx"], "Reverse proxies": ["Nginx"],
                 "Programming languages": ["PHP"], "CMS": ["WordPress"], "Blogs": ["WordPress"]},
  "techCount": 7,
  "fetchedAt": "2026-09-24T14:40:15+00:00"
}
```

`status` is one of `ok`, `blocked`, `disallowed-by-robots`, `error` (DNS failure, timeout, HTTP 4xx/5xx), or `skipped` (only once per-website pricing is introduced: your run's maximum charge was reached before this URL was processed; every URL you give still gets a row). `confidence` is `direct` (a marker matched) or `implied` (inferred from another technology, e.g. WordPress implies PHP).

### Pricing

**Now:** no charge from us. You pay only Apify's platform usage for your runs (shown as "pay per usage" on Apify), which is small for this actor: it runs at 256 MB memory with no browser and no proxies, and our own 5-URL test run on 2026-09-24 used about US$0.0002 of platform usage. Your cost depends on how many URLs you give and how quickly the sites answer.

**Later:** we intend to introduce pay-per-event pricing of about **US$0.01 per website analysed** (status `ok`), with blocked, disallowed, failed and skipped URLs free. Apify notifies the actor's users of such a change and applies its notice period before the new price takes effect (currently 14 days for price increases and pricing-model changes on an actor that has paying users). Once that pricing is active, if you set a maximum charge per run, the actor stops starting new URLs when it is reached, returns the remaining URLs as free `skipped` rows, and says so in the run's `OUTPUT` record.

### Privacy

The actor fetches public pages you name. It stores no cookie values, no full page content and no personal data; the output describes software used by websites. When `includeEvidence` is on (the default), the `evidence` field keeps short matched snippets of response headers and HTML (for example a `Server` header value, a script URL or a meta tag) so you can check each finding; switch `includeEvidence` off to omit them. The user agent identifies the actor: `TechStackDetector/0.1 (website technology lookup; one request per page; honours robots.txt)`.

### Support

Questions and bug reports: open an issue in the **Issues** tab on this actor's page. We aim to respond within 14 days. Replies are written with AI assistance; a human owner can be reached on request.

### About

Published by Madrasco and built and maintained with AI assistance. The signatures are our own work; no third-party fingerprint database is bundled. Product names are trademarks of their owners and are used only to identify the technologies detected.

# Actor input Schema

## `urls` (type: `array`):

Pages to analyse (usually home pages). A bare domain like example.com is fetched as https://example.com/. Up to 1,000 URLs per run; duplicates are removed. Calling from the API or an AI agent? You can also pass the same list as startUrls, domains, websites or a single url (see below).

## `startUrls` (type: `array`):

Same as Website URLs, in the Apify request-list format (\[{"url": "https://..."}] or plain strings). Merged with the other URL fields.

## `domains` (type: `array`):

Bare domains such as example.com (fetched over https). Merged with the other URL fields.

## `url` (type: `string`):

One website URL or domain. Merged with the other URL fields.

## `respectRobotsTxt` (type: `boolean`):

Skip pages that the site's robots.txt disallows for our user agent (reported as 'disallowed-by-robots'). Turn off only for sites you own or have permission to test.

## `timeoutSecs` (type: `integer`):

Give up on a page after this long (reported as an error, not charged).

## `maxConcurrency` (type: `integer`):

How many different sites to fetch at once. Never more than one request at a time to the same host.

## `includeEvidence` (type: `boolean`):

For each technology, list what matched (header, cookie name, meta tag, script URL or HTML snippet).

## Actor input object example

```json
{
  "urls": [
    "https://wordpress.org",
    "https://www.shopify.com",
    "https://example.com"
  ],
  "respectRobotsTxt": true,
  "timeoutSecs": 20,
  "maxConcurrency": 10,
  "includeEvidence": true
}
```

# Actor output Schema

## `results` (type: `string`):

One dataset row per URL: detected technologies by category, with the evidence for each.

## `summary` (type: `string`):

Counts of URLs given, processed and skipped.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://wordpress.org",
        "https://www.shopify.com",
        "https://example.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("madrasco/website-tech-stack-detector").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://wordpress.org",
        "https://www.shopify.com",
        "https://example.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("madrasco/website-tech-stack-detector").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://wordpress.org",
    "https://www.shopify.com",
    "https://example.com"
  ]
}' |
apify call madrasco/website-tech-stack-detector --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,madrasco/website-tech-stack-detector"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/awDHsyrbUTrNU6wyg/builds/B3hIhszzZhbaGGoOU/openapi.json