# Website Tech Stack Scraper – CMS, Analytics, CDN Detector (`datataskflow/website-tech-stack-scraper`) Actor

Detect the technology stack of any website: CMS, e-commerce, frameworks, analytics, ad pixels, CDN, hosting and email provider. Plain HTTP, fast, with optional DNS and security-header audit. Pay only for sites with results.

- **URL**: https://apify.com/datataskflow/website-tech-stack-scraper.md
- **Developed by:** [DataTaskFlow](https://apify.com/datataskflow) (community)
- **Categories:** Lead generation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.40 / 1,000 website scanneds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Website Tech Stack Scraper – Detect CMS, Analytics, CDN, Hosting & Email Provider

Find out which technologies any website runs on. Paste a list of domains and get a clean JSON/CSV/Excel table with the **CMS, e-commerce platform, JavaScript frameworks, analytics and ad pixels, tag managers, chat and marketing tools, CDN, hosting and web server** of every site – with a confidence score and the evidence for each detection.

It uses plain HTTP (no browser), so it is fast and cheap: hundreds of sites in a few minutes. Failed or empty sites are returned as clear rows and are **never charged**.

### What you get

| Feature | Details |
|---|---|
| 🧩 Technology detection | 250+ hand-maintained fingerprints in 40+ categories: CMS, website builders, e-commerce, payments, frameworks, UI libraries, analytics, advertising pixels, tag managers, marketing automation, live chat, cookie consent, CDN, hosting, web servers, languages |
| 📊 Confidence + evidence | Every technology has `confidence` (0-100), `version` when visible, and `evidence` (header, cookie, meta tag, script URL, markup) |
| 🏷️ Grouped output | `technologyNames` (flat list) and `categories` (grouped), ready for filters and pivot tables |
| 🎯 Lead-list filter | Enter `requireTechnologies` (e.g. `Shopify`, `Klaviyo`) and every row gets `matchesAllRequired` |
| 📧 DNS & email provider (add-on) | MX / NS / TXT records plus detection of Google Workspace, Microsoft 365, email-sending services and DNS host |
| 🔐 Security header audit (add-on) | HTTPS + six security headers, 0-100 score and list of missing headers |
| 🛡️ Reliability | Automatic retries with fresh proxy IPs, residential fallback, `robots.txt` respected, per-site error rows |

### Use cases

- **Sales & agencies:** build lead lists of sites that use a specific platform ("all Shopify stores without Klaviyo").
- **Competitor research:** see which analytics, ad pixels and tools competitors run.
- **Market sizing:** measure how widely a CMS, framework or CDN is used in a list of domains.
- **Security & compliance:** quickly spot sites without HSTS or CSP, or without a cookie-consent tool.
- **Data enrichment:** add a tech profile to your CRM or spreadsheet via API, n8n, Make or Zapier.

### How to use it

1. Enter **websites to scan** (domains or URLs, one per line).
2. Optionally set **required technologies**, and turn on the **DNS** and **security audit** add-ons.
3. Click **Start** and download the results as JSON, CSV or Excel, or read them through the API.

#### Input example

```json
{
  "urls": ["https://wordpress.org", "shopify.com", "https://vercel.com"],
  "requireTechnologies": ["Cloudflare"],
  "includeDnsRecords": true,
  "includeSecurityAudit": true
}
```

#### Output example (one item per website)

```json
{
  "url": "https://wordpress.org",
  "finalUrl": "https://wordpress.org/",
  "domain": "wordpress.org",
  "statusCode": 200,
  "title": "WordPress.org",
  "generator": null,
  "server": "nginx",
  "technologyCount": 6,
  "technologies": [
    {"name": "Nginx", "category": "Web server", "version": null, "confidence": 60, "evidence": ["header Server"]},
    {"name": "Google Workspace", "category": "Email provider", "version": null, "confidence": 80, "evidence": ["dns MX record"]}
  ],
  "technologyNames": ["Nginx", "Google Workspace"],
  "categories": {"Web server": ["Nginx"], "Email provider": ["Google Workspace"]},
  "hasData": true,
  "dns": {"mx": ["aspmx.l.google.com"], "ns": ["ns1.example.com"], "txt": ["v=spf1 …"]},
  "emailAndDnsProviders": ["Google Workspace"],
  "securityAudit": {"score": 71, "https": true, "headers": {"Strict-Transport-Security": true}, "missing": ["Content-Security-Policy"]},
  "scrapedAt": "2026-10-18T10:00:00Z"
}
```

Sites that cannot be scanned come back as `{"url": "...", "error": "...", "hasData": false, "scrapedAt": "..."}`.

### Run summary

Every run saves a `RUN_STATS` record (Storage → Key-value store) with the number of sites with technologies, sites without any, failures with reasons, and retry counts.

### Pricing

Pay per result, no platform usage fees and no start fee:

| Event | When it is charged |
|---|---|
| Website scanned | Once per website where at least one technology was detected |
| DNS & email provider (add-on) | Per website, only if you turned it on and DNS records exist |
| Security header audit (add-on) | Per website, only if you turned it on and the site was scanned |

Failed websites and sites without detected technologies are **never charged**. Exact prices are on the Pricing tab and get lower on higher Apify plans.

### FAQ

**How accurate is the detection?** The Actor analyses the HTTP response of the page you give it: headers, cookies, HTML, meta tags and script/stylesheet URLs. Technologies that only load after user interaction or only appear on other pages are not seen – scan a specific URL (e.g. the checkout page) if you need those. Technologies marked "implied by …" are inferred and have a lower confidence.

**Does it run JavaScript?** No. That keeps it fast and cheap. Server-rendered sites and most script tags are detected; things injected purely by late client-side code may be missed.

**Which pages are scanned?** Exactly the URL you enter (a bare domain means the homepage). One request per website.

**Are robots.txt rules respected?** Yes, by default. If a site disallows automated access to the page, you get an error row and no charge.

**Is it legal?** The Actor reads only public information that every browser receives when visiting a page, plus public DNS records, and collects no personal data. You are responsible for using the data in line with applicable laws and the terms of the sites you scan.

**Can I schedule it?** Yes. Use Apify Schedules and connect results to Sheets, Slack, n8n, Make or your own API.

### Support

Found a bug, a missing technology or need a feature? Open an issue on the **Issues** tab – fixes usually ship within a few days.

# Actor input Schema

## `urls` (type: `array`):

Websites or domains to analyze, one per line (for example example.com or https://example.com/shop). Up to 5,000 per run. Each website is charged once, only if technologies were found.

## `requireTechnologies` (type: `array`):

Technology names such as 'Shopify' or 'Klaviyo'. Every result gets matchesAllRequired true/false, so you can filter a lead list by the tools a website uses. Names are case-insensitive. Does not change the price.

## `includeDnsRecords` (type: `boolean`):

Looks up the MX, NS and TXT records of the domain and detects email provider (Google Workspace, Microsoft 365, ...), email-sending services and DNS host. Charged as an extra event per website when records exist.

## `includeSecurityAudit` (type: `boolean`):

Checks HTTPS and six security headers (HSTS, CSP, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy) and returns a 0-100 score. Charged as an extra event per website.

## `respectRobotsTxt` (type: `boolean`):

Skip pages that the site's robots.txt disallows for automated access. Recommended.

## `maxConcurrency` (type: `integer`):

How many websites are scanned at the same time (1-20).

## `maxRetries` (type: `integer`):

Each retry uses a fresh proxy IP; later attempts use residential proxies if available.

## `proxyConfiguration` (type: `object`):

Apify Proxy is used by default. Datacenter proxies work for most sites; the Actor escalates to residential automatically when a site blocks them.

## Actor input object example

```json
{
  "urls": [
    "https://wordpress.org",
    "https://www.shopify.com",
    "https://vercel.com"
  ],
  "includeDnsRecords": false,
  "includeSecurityAudit": false,
  "respectRobotsTxt": true,
  "maxConcurrency": 5,
  "maxRetries": 5,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `audit` (type: `string`):

No description

## `runStats` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://wordpress.org",
        "https://www.shopify.com",
        "https://vercel.com"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("datataskflow/website-tech-stack-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": [
        "https://wordpress.org",
        "https://www.shopify.com",
        "https://vercel.com",
    ],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("datataskflow/website-tech-stack-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://wordpress.org",
    "https://www.shopify.com",
    "https://vercel.com"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call datataskflow/website-tech-stack-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datataskflow/website-tech-stack-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/A32CxtIhKEnKb7R97/builds/SZhgBIdQRP18JX0cb/openapi.json