# Website Tech Stack Detector + DMARC, SPF, MX & SSL Checker (`hirewatch/tech-stack-domain-profiler`) Actor

Technology lookup for any list of domains, a BuiltWith/Wappalyzer-style tech stack detector: 3,500+ technologies (CMS, ecommerce, frameworks, analytics, CDN), email provider from MX records, SPF, DMARC and DKIM checks, SSL certificate expiry, llms.txt and security headers. One row per domain.

- **URL**: https://apify.com/hirewatch/tech-stack-domain-profiler.md
- **Developed by:** [HireWatch](https://apify.com/hirewatch) (community)
- **Categories:** Developer tools, Lead generation, SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$2.00 / 1,000 domain profileds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Website Tech Stack Detector + DMARC, SPF, MX & SSL Checker

A fast **tech stack detector and technology lookup** for any list of websites, plus **email and domain checks** in the same run: email provider from **MX records**, **SPF / DMARC / DKIM** validation and **SSL certificate expiry**. Use it as an affordable BuiltWith or Wappalyzer alternative for lead lists and domain audits.

*Independent tool: not affiliated with or endorsed by BuiltWith or Wappalyzer. Names are used only to describe similar functionality.*

Paste a list of domains and get **one clean row per domain**:

- 🧩 **Technologies**: CMS, ecommerce platform, JS and web frameworks, analytics and tag managers, CDN and hosting, payments, live chat, marketing tools and 3,500+ more, with versions where the site exposes them.
- 📧 **E-mail provider**: Google Workspace, Microsoft 365, Zoho, Proton and others, read from MX records. Security gateways such as Proofpoint, Mimecast and Cisco are reported too, along with the mailbox behind them when SPF reveals it.
- 🔐 **E-mail security**: SPF (record, `-all`/`~all`, recursive DNS-lookup count vs the limit of 10, sending services), DMARC (policy, subdomain policy, pct, reporting service), DKIM keys on ~35 common selectors (with key size), MTA-STS, TLS-RPT and BIMI.
- 🔒 **SSL/TLS**: issuer, validity dates, **days until expiry**, hostname coverage, chain verification, TLS version and key type.
- 🤖 **robots.txt, sitemap and llms.txt**: whether each exists, which **AI crawlers** (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot…) are blocked, the sitemap type and URL count, and the llms.txt title and link count.
- 🌐 **HTTP basics**: status, redirects, final URL, server, response time, HTTP/3, HSTS, security headers, and whether HTTP redirects to HTTPS.
- 🧾 **DNS**: DNS host (Cloudflare, Route 53…), CAA, IPs, and **SaaS tools revealed by TXT verification records** (Atlassian, DocuSign, Zoom, OpenAI, HubSpot, Stripe…).
- ✅ **Plain-language findings**: e.g. "No DMARC record", "SPF needs 13 DNS lookups (limit 10)", "TLS certificate expires in 9 days", "No HSTS header".

### Who it's for

- **Sales and lead generation**: qualify accounts by stack ("uses Shopify + Klaviyo", "on Microsoft 365 behind Proofpoint", "WordPress sites without DMARC").
- **Agencies and MSPs**: e-mail deliverability, security and SSL-expiry audits across many client or prospect domains in one run.
- **SEO and AI visibility**: robots.txt, sitemap and llms.txt status, and which AI crawlers a site blocks.
- **Market research**: technology adoption across a list of companies or competitors.

### Input

| Field | What it does |
|---|---|
| `domains` | Domains or URLs, one per line. `stripe.com`, `https://www.example.com/page` and `info@example.com` all work (only the domain is used). |
| `detectTechnologies`, `checkEmailDns`, `checkDkim`, `checkSsl`, `checkSiteFiles`, `checkHttpsRedirect` | Turn individual checks on or off (all on by default). |
| `dkimSelectors` | Extra DKIM selectors to test. |
| `minTechConfidence` | Hide weaker technology matches (0–100). |
| `maxConcurrency` | Domains processed in parallel (default 10). |

```json
{ "domains": ["apify.com", "stripe.com", "shopify.com"] }
```

### Output

One dataset item per domain. The table views are **Overview**, **Technologies**, **E-mail & DNS** and **Audit findings**, and you can export to CSV, Excel or JSON. A shortened example from a real run:

```json
{
  "domain": "apify.com",
  "status": "ok",
  "finalUrl": "https://apify.com/",
  "httpStatus": 200,
  "title": "Apify: Marketplace of ready-to-run tools for AI",
  "hostingCdn": "Amazon Cloudfront, Amazon S3, Amazon Web Services",
  "emailProvider": "Google Workspace",
  "spfAll": "-all",
  "dmarcPolicy": "reject",
  "dkimFound": true,
  "sslIssuer": "Amazon",
  "sslDaysLeft": 100,
  "hasRobotsTxt": true,
  "hasSitemap": true,
  "hasLlmsTxt": true,
  "technologyNames": ["Algolia", "Amazon Cloudfront", "HubSpot", "Mixpanel", "React", "styled-components", "..."],
  "technologies": [{ "name": "Algolia", "categories": ["Search engines"], "version": null, "confidence": 100 }],
  "email": {
    "provider": "Google Workspace",
    "sendingServices": ["Google Workspace", "Mailgun", "Amazon SES", "HubSpot"],
    "spf": { "allQualifier": "-all", "dnsLookups": 10, "tooManyLookups": false },
    "dmarc": { "policy": "reject", "subdomainPolicy": "reject", "record": "v=DMARC1; p=reject; sp=reject; pct=100; rua=mailto:***@apify.com" },
    "dkim": { "found": [{ "selector": "google", "keyBits": 2048 }, { "selector": "pic", "likelySender": "Mailgun", "keyBits": 1024 }] }
  },
  "dns": { "dnsProvider": "Amazon Route 53", "verifiedServices": ["OpenAI", "Google Search Console"] },
  "robotsTxt": { "found": true, "aiCrawlersBlocked": [], "sitemaps": ["https://apify.com/sitemap.xml"] },
  "llmsTxt": { "found": true, "title": "Apify", "linkCount": 73 },
  "issues": []
}
```

`status` takes one of these values:

- `ok`: the full profile.
- `partial`: the website was unreachable, returned an error, or robots.txt disallowed it. DNS, e-mail and SSL data are still returned.
- `domain_not_found`: the domain does not exist. **Not charged.**
- `invalid_input`: the input could not be read as a domain. **Not charged.**

The `RUN_REPORT` record in the key-value store lists the status of every input.

### Pricing

**Pay per event: $0.002 per domain profiled ($2 per 1,000 domains).** There is no extra charge for the extra checks; everything above is included. You are not charged for invalid inputs, duplicates, or domains that don't exist. Set a *maximum cost per run* and the Actor stops cleanly when it reaches it.

### How it works, and why it's polite

- It requests **only** the public homepage, `/robots.txt`, the sitemap (the first one listed in robots.txt, or `/sitemap.xml`), `/llms.txt`, and one request to `http://` to check the HTTPS redirect, whose body is not downloaded. It never crawls deeper pages.
- **robots.txt is checked before every request**, including each redirect hop (RFC 9309). If robots.txt disallows the homepage, the page is skipped and the domain is reported as `partial`.
- At most **one request per second per website**, an honest User-Agent (`HireWatchSiteProfiler`), and size limits on every download.
- DNS and TLS facts come from public DNS and a normal TLS handshake.
- **No personal data is collected.** Page text, e-mail addresses and phone numbers are never output, and the mailbox part of DMARC report addresses is masked (`***@domain`). No WHOIS or registrant data is collected.

### Limits and accuracy

- Technology detection reads the homepage HTML, headers, cookies, script URLs, meta tags, DOM, DNS and the certificate. It does **not** run JavaScript in a browser, so tools that only appear after client-side scripts run can be missed. This keeps it fast and cheap.
- DNS cannot list DKIM selectors, so only common selectors are checked. "Not found" therefore does not prove that DKIM is missing.
- Sites behind aggressive bot protection may return 403. You still get DNS, e-mail, SSL and robots data, and the CDN/WAF is usually detected from the response headers.

### Use with AI agents / MCP

This Actor is built to be called by AI agents (Grok Bot, Claude, ChatGPT, Cursor, LangChain and others) through the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp) or the Apify API.

- **Add only this tool:** `https://mcp.apify.com?tools=hirewatch/tech-stack-domain-profiler`. You can also let your agent find it through `search-actors`.
- **Keep calls small so they finish fast.** The MCP `call-actor` tool waits up to 45 seconds, and a run with 3 domains takes about 20 seconds, so send up to ~10 domains per call. For bigger lists, start the run and read the results when it finishes. Turn off checks you don't need (e.g. `checkDkim: false`) to speed things up, and cap spend with `maxTotalChargeUsd` in `callOptions`.
- **Read results** with `get-dataset-items` using the run's `datasetId`. You get one flat row per domain with a `status` field (`ok`, `partial`, `domain_not_found` or `invalid_input`). A bad domain never fails the whole run, and domains that don't exist or can't be read are not charged.
- **Pay without an Apify account.** The Actor uses pay-per-event pricing with limited permissions, so agents can pay per run with [agentic payments](https://docs.apify.com/agents/payments) (x402 / MPP prepaid tokens or Skyfire).

Minimal agent call:

```json
{
  "domains": ["stripe.com", "shopify.com"],
  "checkDkim": false
}
```

### Attribution and licences

Technology fingerprints come from **Wappalyzer** ([npm `wappalyzer@6.10.54`](https://www.npmjs.com/package/wappalyzer/v/6.10.54)), © Elbert Alias / Wappalyzer, used under the **MIT License**. 6.10.54 is the last MIT-licensed release; later Wappalyzer versions and community forks are GPL-3.0 and are not used. The full licence text ships with the Actor in `src/data/LICENSE-fingerprints.txt`. This Actor is not affiliated with or endorsed by Wappalyzer or BuiltWith.

### Related

- [Workday, Greenhouse, Lever, Ashby & Oracle Jobs Scraper](https://apify.com/hirewatch/career-sites-jobs-tracker): open jobs, new/closed tracking and hiring signals from company career sites on five platforms.
- [Workday Jobs Scraper](https://apify.com/hirewatch/workday-jobs-scraper): every job from Workday career sites, including sites with more than 2,000 jobs.
- [Greenhouse Jobs Scraper](https://apify.com/hirewatch/greenhouse-jobs-scraper): Greenhouse job boards by company name, with salary ranges.
- [Lever Jobs Scraper](https://apify.com/hirewatch/lever-jobs-scraper) and [Ashby Jobs Scraper](https://apify.com/hirewatch/ashby-jobs-scraper): Lever and Ashby job boards by company name, with salaries.

Questions or a missing technology, e-mail provider or check? Open an issue on the Actor's Issues tab.

# Actor input Schema

## `domains` (type: `array`):

One per line. Bare domains (stripe.com), URLs (https://www.example.com/about) and even e-mail-style inputs (info@example.com, only the domain is used) are accepted. Duplicates are removed. AI agents: send up to ~10 domains per call to get results within about 30 seconds.

## `detectTechnologies` (type: `boolean`):

Identify CMS, ecommerce platform, frameworks, analytics, tag managers, CDN/hosting, payment, chat and marketing tools from the homepage, its HTTP headers and DNS (3,500+ fingerprints).

## `checkEmailDns` (type: `boolean`):

MX-based e-mail provider and security gateway, SPF (incl. recursive lookup count), DMARC policy, MTA-STS, TLS-RPT, BIMI, DNS host, CAA, and SaaS tools revealed by TXT verification records.

## `checkDkim` (type: `boolean`):

Query ~35 common DKIM selectors (Google, Microsoft 365, SendGrid, Mailchimp, Mailgun...). DNS cannot list selectors, so 'not found' means none of the common ones exist.

## `dkimSelectors` (type: `array`):

Optional custom selectors to check in addition to the common ones (max 20).

## `checkSsl` (type: `boolean`):

Issuer, validity dates, days until expiry, hostname coverage, chain verification and negotiated TLS version.

## `checkSiteFiles` (type: `boolean`):

Presence and basics of robots.txt (incl. which AI crawlers are blocked), the XML sitemap and llms.txt.

## `checkHttpsRedirect` (type: `boolean`):

Make one extra request to http:// to see whether it redirects to https:// (body not downloaded).

## `minTechConfidence` (type: `integer`):

Only report technologies detected with at least this confidence (0-100). Most detections are 100; some weaker signals are 25-75.

## `maxConcurrency` (type: `integer`):

How many domains to profile at the same time. Each website still gets at most one request per second.

## `requestTimeoutSecs` (type: `integer`):

Timeout for each HTTP request.

## `dnsMode` (type: `string`):

auto = system resolver, switching to DNS-over-HTTPS (Cloudflare/Google) if the system resolver looks unusable.

## Actor input object example

```json
{
  "domains": [
    "apify.com",
    "stripe.com",
    "shopify.com"
  ],
  "detectTechnologies": true,
  "checkEmailDns": true,
  "checkDkim": true,
  "checkSsl": true,
  "checkSiteFiles": true,
  "checkHttpsRedirect": true,
  "minTechConfidence": 0,
  "maxConcurrency": 10,
  "requestTimeoutSecs": 20,
  "dnsMode": "auto"
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

## `technologies` (type: `string`):

No description

## `emailSecurity` (type: `string`):

No description

## `full` (type: `string`):

No description

## `runReport` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "apify.com",
        "stripe.com",
        "shopify.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("hirewatch/tech-stack-domain-profiler").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": [
        "apify.com",
        "stripe.com",
        "shopify.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("hirewatch/tech-stack-domain-profiler").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "apify.com",
    "stripe.com",
    "shopify.com"
  ]
}' |
apify call hirewatch/tech-stack-domain-profiler --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,hirewatch/tech-stack-domain-profiler"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/nrt962iljsSxCAgfe/builds/8Gc91ay5lTmcCpNn6/openapi.json