Subdomain Finder: certificate transparency logs

List one or more domains and get back the subdomains that have appeared on public TLS certificates, one row each. crt.sh indexes the certificates recorded in public certificate transparency logs, so hosts named on those certificates show up here, staging and test hosts included.

The search runs against crt.sh [1], a free public certificate transparency search. No scanning or guessing of names takes place. DNS lookups, on by default, go to public resolvers, not to the domain's web servers.

What you get for each subdomain

• subdomain and level (1 for blog.example.com, 2 for api.eu.example.com)
• firstSeen and lastSeen: the earliest and latest start dates of certificates naming it
• lastCertificateExpiry and hasValidCertificate
• logEntries: how many log entries name it. A certificate can be logged twice, once as a precertificate and once as the final certificate, so this can run higher than the number of certificates
• issuers, the organization names of the certificate authorities, such as Let's Encrypt
• wildcardCertificate when the name appeared as *.name, and namedInCertificate when it appeared spelled out
• resolves and ipAddresses, the current A and AAAA records, when DNS lookups are on (the default). resolves is empty when the lookup times out

A name taken from a wildcard certificate is returned without the *., so *.staging.example.com gives staging.example.com. Email addresses that some certificates carry are dropped, and so is the domain itself.

Input

• Domains (required). A URL works too; https://www.python.org/about/ is searched as python.org.
• Include names from expired certificates, on by default.
• Look up DNS addresses, on by default.
• Only subdomains that resolve, off by default. Names with no A or AAAA record are left out and aren't charged.
• Maximum subdomains per domain, 1,000 by default, shallowest names first.

Output

One row per subdomain. A domain that isn't a valid domain name, or that has no subdomains in the logs, gets one row with ok false and the reason in error, and the run moves on to the next domain. When crt.sh can't be reached after four tries, the reason goes in error too. The OUTPUT record in the key-value store gives the number of subdomains returned and domains that failed.

Speed and fair use

crt.sh limits each IP address to 60 requests a minute [2], so the actor sends one request at a time, at least 1.5 seconds apart. crt.sh is a free public service and is sometimes busy; when it answers with a server error or times out, the actor waits 5, then 10, then 20 seconds and asks again.

Pricing

Pay per event: one event per subdomain returned. There's no start fee and no charge for domains that fail or find nothing. When the run reaches the maximum charge you set, it stops, and every subdomain row in the dataset is a paid one.

Support

Questions and problems: the Issues tab on this page.

References

1. Certificate Search | crt.sh | crt.sh. https://crt.sh/
2. Rate limits for https://crt.sh/ | crt.sh discussion group | groups.google.com. https://groups.google.com/g/crtsh/c/NZJntKrBdmg