Certificate Transparency Monitor — AI Agent Recon API

Keyless certificate transparency intelligence for AI security/recon agents. Pull TLS certificates from CT logs (crt.sh) for any domain. Track newly-issued certs daily, discover subdomains via wildcard, or batch many domains. Each certificate becomes one structured row with issuer, validity window, SAN hostnames and serial — ideal for detecting new infrastructure, brand impersonation, and attack-surface changes. No API key, no headless browser.

Built for the new wave of AI agents that reason about external infrastructure and attack surface — recon agents that watch a target's new certificates, brand-protection agents that detect lookalike domains, asset-discovery agents that enumerate subdomains via CT logs, and RAG pipelines that ground LLM answers in current infrastructure data instead of stale scans.

---

🎯 What this Actor is for

Every TLS certificate issued by a public CA is logged to Certificate Transparency logs — a real-time firehose of new infrastructure, hostnames and issuer relationships. When an AI agent is asked "what new infrastructure did Acme stand up this week?" or "what subdomains does google.com have?", it needs structured, current, CT-log data — not a raw crt.sh JSON dump. certificate-transparency-monitor is that infra-grounding layer:

• Daily new certificates. watch mode pulls only certs logged in the last N days for a domain — ideal for a scheduled daily monitoring run. Big companies log dozens of new certs a day.
• Subdomain discovery. subdomains mode uses a wildcard query (%.domain) to enumerate all hostnames ever certified for a domain — hundreds to thousands of subdomains for large orgs.
• Batch domains. domain/bulk mode pulls all certificates for a list of domains — build a full CT-log corpus.
• Triage-ready schema. Every cert carries the queried domain, commonName, nameValues (SAN hostnames array), issuerName, serialNumber, notBefore/notAfter, entryTimestamp/loggedDate, and isExpired.
• Filters. excludeExpired drops past-validity certs for a current attack-surface view. dedupBySerial collapses precertificate + final cert duplicates.
• AI-agent friendly. Predictable fields, ISO dates, nullable values. Drop straight into a prompt or a vector store.
• No keys, no browser. Pure HTTP + JSON from crt.sh on a small Node 20 container, with Apify proxy for reliability. Cheap, fast, resilient.

---

✨ Key features

• 📜 crt.sh CT logs — https://crt.sh/?q=…&output=json covering all major CT logs (Google Argon, Cloudflare Nimbus, Let's Encrypt Oak, DigiCert, …).
• 👁️ Watch mode (daily monitoring) — daysBack filter keeps only certs logged in the last N days (by entry_timestamp). 1 = today's new certs.
• 🌐 Subdomain discovery — wildcard %.domain query surfaces every hostname that has ever held a cert for the domain, via SAN (nameValues).
• 🏷️ Rich per-cert fields — commonName, SAN hostnames (nameValues), issuerName, serialNumber, validity window (notBefore/notAfter), log entry timestamp.
• 🧹 Filters — excludeExpired (current attack surface), dedupBySerial (collapse precert+final duplicates).
• 📚 Batch domains — bulk mode loops many domains; each tagged with query/domain.
• 🌐 Proxy-aware — Apify datacenter proxy for reliability (crt.sh can be slow/overloaded).
• 🔁 Retry/backoff — 5xx and timeouts trigger exponential backoff retry (up to 5 attempts).
• 💰 Pay-per-result — charged per saved certificate, not per run. Empty results are free.

---

🤖 Why AI agents need this

CT logs are the highest-signal, real-time source of external infrastructure changes — every new cert means a new hostname, service, or issuer relationship. When an agent monitors a target's CT feed, it sees new infrastructure the moment it's certified (often before DNS or port scans catch it). certificate-transparency-monitor wraps crt.sh into one reliable, pay-per-result Actor:

1. New-infrastructure detection. A recon agent runs watch daily on a target domain and alerts when a cert is logged for a hostname not seen before — catching new services, staging environments, and shadow IT.
2. Subdomain enumeration. An asset-discovery agent runs subdomains mode to build a full hostname inventory via CT SANs — far broader than DNS brute-forcing.
3. Brand-protection / lookalike detection. An agent monitors certs whose commonName or SANs contain a brand string, flagging impersonation domains the moment they're certified.
4. Issuer / CA tracking. An agent tracks which CAs a target uses (Let's Encrypt vs DigiCert vs internal) and alerts on anomalies (e.g. a sudden shift to a cheap CA often used for phishing).
5. Attack-surface diffing. An agent schedules weekly domain runs, diffs the cert sets, and reports new/removed hostnames as attack-surface changes.
6. RAG over infrastructure. Embed the cert feed into a vector store; answer "when did X hostname first appear?" with citations to the exact cert.
7. Threat-intel pipeline. Feed new certs into a SIEM; correlate issuer/hostname patterns with known attacker TTPs.

Each of these is one Actor call (or a scheduled run). The output is a table of certs ready for an LLM to triage, summarize, or alert on.

---

📦 What you get (output schema)

Every run streams one certificate per row to the default dataset. A row looks like:

{
  "query": "stripe.com",
  "domain": "stripe.com",
  "certId": 12345678901,
  "commonName": "*.stripe.com",
  "nameValues": ["stripe.com", "www.stripe.com", "api.stripe.com", "dashboard.stripe.com"],
  "issuerName": "DigiCert TLS RSA SHA256 2020 CA1",
  "serialNumber": "0abc123def456...",
  "notBefore": "2026-06-28T00:00:00.000Z",
  "notAfter": "2027-07-03T23:59:59.000Z",
  "entryTimestamp": "2026-06-28T12:34:56.789Z",
  "loggedDate": "2026-06-28",
  "isExpired": false,
  "scrapedAt": "2026-07-02T12:00:00.000Z"
}

Use the Overview view to scan all certs with issuer + validity, or the Subdomains (SAN) view to read the discovered hostnames per cert.

---

🚀 How to use

1. Daily new certs (schedule this)

{
  "mode": "watch",
  "domain": "google.com",
  "daysBack": 1,
  "maxPerDomain": 500
}

2. Subdomain discovery

{
  "mode": "subdomains",
  "domain": "stripe.com",
  "maxPerDomain": 2000,
  "excludeExpired": true
}

3. All certs for a domain (batch)

{
  "mode": "domain",
  "domains": ["stripe.com", "linear.app", "vercel.com"],
  "maxPerDomain": 1000
}

4. Bulk many domains

{
  "mode": "bulk",
  "domains": ["example.com", "acme.io", "target.org"],
  "maxPerDomain": 500
}

From code (Apify SDK)

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('logiover/certificate-transparency-monitor').call({
  mode: 'subdomains',
  domain: 'stripe.com',
  excludeExpired: true,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
const subdomains = new Set();
for (const c of items) for (const n of c.nameValues) subdomains.add(n);
console.log([...subdomains]);

As an MCP tool for AI agents

Wrap this Actor in an MCP server. A recon agent calls the tool with a domain + mode and receives structured certs/subdomains — no crt.sh scraping, no SAN parsing on the agent side.

---

🔧 Input fields

Field	Type	Default	Description
mode	enum	watch	watch (new certs in N days), subdomains (wildcard), domain (all certs), bulk (many domains).
domain	string	—	Single domain for watch/subdomains.
domains	array	—	Domains for domain/bulk modes.
daysBack	int	1	Days back for watch mode (1–365). 1 = today.
maxPerDomain	int	1000	Cap on certs per domain (1–20000).
excludeExpired	bool	false	Drop certs past their notAfter.
dedupBySerial	bool	true	Collapse precert+final duplicates by serial.
useApifyProxy	bool	true	Route through Apify datacenter proxy.

---

🧩 How it works

1. Build the crt.sh query. For watch/domain/bulk: q=<domain>. For subdomains: q=%.<domain> (wildcard, matches every hostname ending in the domain). All requests use output=json.
2. Fetch with retry. crt.sh can be slow (especially for popular domains with tens of thousands of certs). The Actor uses a 60s timeout and exponential backoff retry (up to 5 attempts) for 5xx/timeouts, routed through the Apify datacenter proxy.
3. Parse + normalize. Each cert object is mapped to a flat record: commonName, nameValues (SAN hostnames split on newlines, lowercased, deduped), issuerName, serialNumber, notBefore, notAfter, entryTimestamp, loggedDate (date part), isExpired (notAfter < now).
4. Deduplicate. If dedupBySerial, certificates sharing a serial (precertificate + final cert) collapse to one row.
5. Filter. excludeExpired drops certs past their validity. watch mode filters by entryTimestamp (or notBefore fallback) within daysBack.
6. Cap. Truncate to maxPerDomain per domain.
7. Stream. Each cert is pushed to the dataset and one result event is charged.

---

💡 Tips & best practices

• Schedule watch daily. daysBack: 1 each morning catches yesterday's new certs. Big orgs log dozens/day; small sites a few/week. Consistent recurring volume.
• subdomains for asset discovery. A single wildcard query often yields hundreds of hostnames via SAN — broader than DNS brute-forcing and passively collected.
• excludeExpired for current view. CT logs retain expired certs forever; enable this to see only currently-valid infrastructure.
• Batch competitors. bulk mode over a watchlist of competitor domains builds a comparative CT corpus.
• Brand monitoring. Query your brand string (e.g. stripe) to catch lookalike certs (stripe-login.com, secure-stripe.com) the moment they're issued.
• Pair with recon. Combine with subdomain-finder (active CT + other sources), bulk-dns-records-lookup (resolve discovered hostnames), and bulk-whois-rdap-lookup (ownership).
• Be patient on big domains. google.com has tens of thousands of certs; maxPerDomain: 5000 + 60s timeout is normal. Small domains return in seconds.

---

❓ FAQ

Does this Actor need any API keys?

No. It queries crt.sh, the public Certificate Transparency log search, which is keyless. Just an Apify account.

Why crt.sh instead of direct CT-log APIs?

crt.sh aggregates all major CT logs (Google, Cloudflare, DigiCert, Let's Encrypt, …) into one searchable interface with deduplication. Querying individual logs requires parsing Merkle trees and merge-deduplicating across logs — crt.sh does that for you.

How current is the data?

CT logs are append-only and nearly real-time. A cert logged minutes ago appears in crt.sh within minutes to a few hours. watch mode with daysBack: 1 gives you certs logged in the last 24 hours.

Why are some certs duplicated (before dedup)?

CT logs record both the precertificate and the final certificate for each issuance, often with the same serial. dedupBySerial: true (default) collapses them into one row.

What's in nameValues?

The Subject Alternative Names (SAN) of the cert — every hostname the cert is valid for. For a wildcard cert that's *.stripe.com plus explicit names. This is the richest source for subdomain discovery.

Why do some certs have a null entryTimestamp?

Occasionally crt.sh has the not_before (validity start) but not the log entry timestamp. The Actor falls back to not_before for watch filtering and loggedDate.

How is this priced?

Pay-per-result: one result event per saved certificate. Runs that yield zero certs are free.

Will crt.sh be slow?

For popular domains (google, microsoft) with tens of thousands of certs, yes — the query can take 10–60s. The Actor uses a 60s timeout + retry. For most domains it's a few seconds.

Can AI agents call this directly?

Yes. Expose it through an MCP server or Apify tool integration; the agent passes a domain + mode and receives structured certs/subdomains back. This is the primary design target.

---

🔗 Related Actors

• subdomain-finder — CT-log subdomain finder (complementary, active).
• bulk-dns-records-lookup — resolve discovered hostnames.
• bulk-whois-rdap-lookup — domain ownership for cert attribution.
• bulk-ssl-certificate-checker — live cert inspection (vs CT-log history).
• cve-security-advisory-monitor — vulnerability DB (security context).
• github-activity-stream — dev-side security commits/releases.

---

📝 Changelog

2026-07-02 — v1.0

• Initial release.
• 4 modes: watch, subdomains, domain, bulk.
• crt.sh CT-log search, SAN hostname extraction, issuer/serial/validity fields.
• excludeExpired and dedupBySerial filters.
• daysBack watch window, maxPerDomain cap.
• Apify datacenter proxy default.
• Pay-per-result (result event per saved cert).

---

⚖️ Disclaimer

This Actor queries crt.sh, the public Certificate Transparency log search. CT-log data is public by design (required by the CA/Browser Forum Baseline Requirements); certificate holders consent to logging upon issuance. Respect crt.sh's usage guidelines. Use for security monitoring, asset discovery, brand protection and AI-agent grounding on data that is already public.