Bulk Tech Stack Lookup: a Wappalyzer and BuiltWith alternative with DNS, email security and TLS checks

Paste a list of domains and get one clean row per domain:

• Tech stack. CMS, ecommerce platform, JavaScript framework, analytics, tag manager, CDN, hosting, marketing tools and 7,000+ other technologies, with versions.
• Email setup. Email provider (Google Workspace, Microsoft 365, Proofpoint and others), sending services found in SPF (SendGrid, Mailgun, HubSpot and others), and an SPF/DMARC grade with the DMARC reporting vendor.
• SaaS tools from DNS verification records. For example Atlassian, OpenAI, DocuSign or Stripe.
• TLS certificate. Issuer, expiry date and days left.
• Security headers. Score and A–F grade.
• AI crawler policy. Which AI bots robots.txt blocks, and whether llms.txt exists.
• security.txt and ads.txt, plus the sitemaps declared in robots.txt.

No browser is used, so it is fast and cheap. Free while in launch; planned pay-per-event price: $3 per 1,000 domains, everything included, charged only for successful lookups (dead, unreachable and blocked sites will stay free).

What can I use it for?

• Lead qualification and segmentation. Find Shopify stores using Klaviyo, WordPress sites without a CDN, or companies on Microsoft 365 with a weak DMARC policy.
• Agency prospecting. Look for expiring TLS certificates, F-grade security headers, missing DMARC or an outdated CMS version.
• Competitive and market research. Measure technology adoption across a list of sites, such as Next.js versus WordPress, or which CDN is most common in an industry.
• Email deliverability audits. Check SPF lookup counts over the limit of 10, DMARC p=none, missing MTA-STS or BIMI.
• AI and SEO policy tracking. See who blocks GPTBot, ClaudeBot or Google-Extended, who publishes llms.txt, and who uses Content-Signal directives.
• Domain list hygiene. Flag dead domains (NXDOMAIN), unreachable sites and redirects to other domains. Dead and unreachable rows are not charged, now or under the planned pricing.

What data does it return?

Every row has flat summary columns that are easy to use in a spreadsheet, plus nested detail objects.

Field	Example
domain, finalUrl, status	wordpress.org, https://wordpress.org/, ok
cms, cdn, hosting	WordPress, Cloudflare, Vercel
technologyNames	["WordPress", "PHP", "Nginx", "Google Tag Manager", ...]
technologies[]	name, version, confidence, categories, website, detectedBy, CPE
emailProvider, dnsProvider	Google Workspace, Amazon Route 53
emailSecurityGrade	A–F, based on SPF policy and lookups, DMARC policy, MTA-STS and BIMI
dns	A/AAAA/NS/MX, SPF (policy, includes, lookup count), DMARC (policy, pct, reporting vendor), CAA issuers, SaaS verifications
tls	valid, issuer, notBefore/notAfter, daysUntilExpiry, lifetime, SANs, protocol, key type
securityHeaders	score 0–100, grade, present/missing headers, HSTS and CSP details, version leaks
aiCrawlers	blocked and allowed AI bots, bots named in robots.txt, blocksAllAiTraining, Content-Signal, llms.txt / llms-full.txt
wellKnown	security.txt (found, expires, signed), ads.txt (seller records, top ad systems), sitemap URLs
page	title, meta description, language, generator, canonical URL, Open Graph, JSON-LD types
http	status code, HTTP version, redirect chain, response time, server

status is ok, unreachable (no HTTP answer over HTTPS or HTTP), dns_error (the domain does not exist), blocked_by_site (the site answered with 403, 429 or a bot-protection page), disallowed_by_robots_txt (robots.txt does not allow this actor to fetch the homepage) or invalid_input; error explains anything other than ok.

Example output (one row, shortened)

This row is from a real run on 2026-10-03:

{
  "domain": "wordpress.org",
  "status": "ok",
  "finalUrl": "https://wordpress.org/",
  "title": "Blog Tool, Publishing Platform, and CMS – WordPress.org",
  "cms": "WordPress",
  "technologyNames": ["WordPress", "WordPress Block Editor", "Gutenberg", "MySQL", "PHP", "Nginx",
                      "Google Font API", "Google Tag Manager", "HSTS", "Let's Encrypt", "Open Graph", "RSS"],
  "emailSecurityGrade": "A",
  "securityHeadersGrade": "E",
  "tlsDaysUntilExpiry": 79,
  "blocksAiTraining": false,
  "hasLlmsTxt": true,
  "technologies": [
    {"name": "WordPress", "version": "7.2", "confidence": 100, "categories": ["CMS", "Blogs"],
     "detectedBy": ["dom", "headers", "html", "meta", "scriptSrc", "scripts"],
     "cpe": "cpe:2.3:a:wordpress:wordpress:*:*:*:*:*:*:*:*"},
    {"name": "Gutenberg", "version": "24.1.0", "confidence": 100, "categories": ["WordPress plugins", "Editors"],
     "detectedBy": ["dom", "scriptSrc"]}
  ],
  "dns": {
    "mx": ["smtp1-dca.wordpress.org", "smtp2-dca.wordpress.org"],
    "spf": {"policy": "fail", "includes": ["helpscoutemail.com"], "dnsLookupCount": 1, "exceedsLookupLimit": false},
    "dmarc": {"policy": "reject", "pct": 100, "reportingDomains": ["ag.dmarcian.com"], "reportingVendors": ["dmarcian"]},
    "sendingServices": ["Help Scout"]
  },
  "tls": {"valid": true, "issuer": "Let's Encrypt", "notAfter": "2026-12-22T19:43:57+00:00", "daysUntilExpiry": 79,
          "lifetimeDays": 90, "protocol": "TLSv1.3", "keyType": "EC secp256r1"},
  "securityHeaders": {"score": 30, "grade": "E", "present": ["Strict-Transport-Security", "X-Frame-Options"],
                      "missing": ["Content-Security-Policy", "X-Content-Type-Options", "Referrer-Policy", "..."],
                      "hsts": {"maxAge": 3600, "includeSubDomains": false, "preload": false}},
  "aiCrawlers": {"robotsTxtFound": true, "blockedBots": [], "namedBots": ["GPTBot", "ClaudeBot", "Google-Extended", "..."],
                 "blocksAllAiTraining": false, "llmsTxt": {"found": true, "title": "WordPress", "linkCount": 36}},
  "wellKnown": {"securityTxt": {"found": true, "expires": "2027-06-30T15:00:00.000Z", "hasPolicy": true},
                "sitemaps": ["https://wordpress.org/sitemap.xml", "https://wordpress.org/news-sitemap.xml", "..."]}
}

The Output tab offers four ready-made views: Overview, Email & DNS security, TLS & headers and AI crawler policy. You can export any of them as CSV, Excel, JSON or XML, or fetch them through the Apify API.

How much does it cost?

Free while in launch. The actor charges no fee of its own right now; you pay only the Apify platform usage of your runs (the free Apify plan includes $5 of usage a month). The pay-per-event price below is planned and applies only once it is switched on; Apify notifies existing users before a price change takes effect.

Planned pricing, pay per event, only for successful lookups:

Event	Planned price
Domain analyzed (successful lookup, all checks included)	$0.003 (= $3 per 1,000 domains)

• Charged: only rows with status: ok, where the homepage answered and was analysed.
• Not charged: everything else. That covers dead domains (dns_error), sites that gave no HTTP answer (unreachable), sites that refused our request (blocked_by_site), sites whose robots.txt does not allow the homepage fetch (disallowed_by_robots_txt), invalid inputs (for example 10.0.0.1 or localhost), timeouts and internal errors. These rows are still written so you can see what happened, with whatever DNS, email, TLS and robots.txt results could be collected, so list cleaning is free.
• Cost cap: once pricing is on, set Maximum cost per run and the actor stops cleanly when it reaches it. Uncharged rows don't count toward it.

For comparison (prices checked 2026-10-03): Wappalyzer's API starts at $250/month, BuiltWith's plans start at $295/month, and the most-used tech-stack actor on the Apify Store charges $0.10 per domain.

How to use it

1. Paste domains, URLs or email addresses into Domains or URLs. Only the domain part of an email address is used, and duplicates are removed.
2. Optionally switch off checks you don't need. This makes runs faster but not cheaper.
3. Click Start. With the defaults, about 20 domains are checked in parallel.

Input example

{
  "domains": ["apify.com", "https://www.shopify.com", "ops@stripe.com", "wordpress.org"],
  "maxConcurrency": 20,
  "minConfidence": 50
}

API example

curl -X POST "https://api.apify.com/v2/acts/adamhaleworks~bulk-tech-stack-lookup/run-sync-get-dataset-items?token=<TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"domains": ["apify.com", "wordpress.org"]}'

How it works

• Homepage fetch. robots.txt is read first. HTTPS is tried first, then HTTP if HTTPS fails. Redirects are followed one hop at a time, and each new site's robots.txt is checked before the hop. If the homepage is readable, the actor also reads llms.txt, llms-full.txt, security.txt and ads.txt from the same origin, skipping any that robots.txt disallows.
• Polite by design. Every request identifies the actor as bulk-tech-stack-lookup/1.0 (+<link to this Store page>); there is no browser disguise, no proxy and no retry with a different identity. It follows robots.txt rules for bulk-tech-stack-lookup (RFC 9309: a missing robots.txt allows everything, an unreadable one allows nothing). If the site answers with 403, keeps answering 429 or serves a bot-protection page (Cloudflare, DataDome and similar), the row is blocked_by_site and you are not charged.
• Technology detection. It matches the homepage HTML, scripts, meta tags, headers, cookies, DNS records and certificate issuer against the open webappanalyzer fingerprint set (GPL-3.0, 7,000+ technologies, version pinned for reproducible results). An Aho-Corasick prefilter makes detection take about 60 ms per page.
• DNS checks. These go through public resolvers (1.1.1.1 and 8.8.8.8 by default). SPF includes are followed recursively to count DNS lookups.
• TLS check. The certificate is read with a direct handshake. Invalid certificates (expired, self-signed or wrong host) are still parsed and reported with the reason.
• Reliability. Network errors, HTTP 429 and 5xx get retries with exponential backoff, and each domain has a time limit. Progress is saved, so a migrated run continues where it stopped.

Limitations

• Only the homepage is analysed, and no JavaScript is executed. Technologies that appear only after client-side rendering or on inner pages can be missed. Wappalyzer's js and xhr rules are skipped for the same reason.
• detectedBy: ["dns:TXT"] means the company has verified the domain with that SaaS tool. It does not mean the tool runs on the website.
• Sites behind strict bot protection refuse automated clients. Those rows are blocked_by_site, with the DNS, email and TLS results but no tech stack, and they are not charged. The actor does not try to get around bot protection and has no proxy option.

Is it legal? What about personal data?

The actor reads only public, technical, non-personal data: HTTP headers, HTML markup, public DNS records and TLS certificates. These are the same data a browser or dig sees. It does not collect names, email addresses or phone numbers. For example, security.txt is reported as present or absent, without storing contact values, and only the domains in DMARC reporting addresses are kept. Each domain gets only a handful of light requests: the homepage plus robots.txt, llms.txt, llms-full.txt, security.txt and ads.txt, and a retry only when one fails. They identify the actor by name, follow robots.txt, and parallelism is capped by the concurrency setting. You are responsible for using the results in line with the laws that apply to you.

This actor is not affiliated with or endorsed by Wappalyzer or BuiltWith. Their names are used only to describe what it can replace.

FAQ

Why do technology counts differ from Wappalyzer's browser extension? The extension runs JavaScript in your browser and sees inner pages. This actor checks the raw homepage in bulk, which is far cheaper, and adds DNS, TLS, email and AI-policy data that the extension doesn't provide.

Can I monitor domains over time? Yes. Schedule the actor (for example weekly) and compare tlsDaysUntilExpiry, technologyNames or emailSecurityGrade between runs.

Does it work for subdomains? Yes. shop.example.com is fetched as given. Email and DNS records are read from the registrable domain (example.com).

Why is a site blocked_by_site or disallowed_by_robots_txt? The site refused automated clients, or its robots.txt does not allow the homepage to be fetched (for example User-agent: * with Disallow: /). The actor accepts both, and you are not charged for those rows.

I run a website. How do I control this actor? It sends bulk-tech-stack-lookup in its User-Agent. Add a robots.txt group for that name (User-agent: bulk-tech-stack-lookup with Disallow: /), or answer its requests with 403. Either way it records that and fetches nothing else from your site in that run. The TLS certificate check still runs, because it is a handshake rather than a page request.

Something is wrong or missing? Open an issue on the actor's Issues tab and include the domain. Fingerprint fixes are easy to add.

---

Author: Adam Hale