# Subdomain Finder, crt.sh Certificate Transparency Logs Search (`snow_leo_data/crtsh-subdomain-finder`) Actor

Subdomain finder past the crt.sh query limits: 7,236 subdomains of mercedes-benz.com where one crt.sh query returns 4. Certificate transparency logs search with duplicates merged, expiring and self-signed certificates flagged, and a watch mode for new certificates.

- **URL**: https://apify.com/snow\_leo\_data/crtsh-subdomain-finder.md
- **Developed by:** [Snow Leo Data](https://apify.com/snow_leo_data) (community)
- **Categories:** Developer tools, Automation, MCP servers
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.20 / 1,000 result rows

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Subdomain Finder, crt.sh Certificate Transparency Logs Search

Find every subdomain and every SSL/TLS certificate ever logged for a domain, from the public Certificate Transparency search at crt.sh. Get expiry dates, issuers, serial numbers, suspicious host names (dev, staging, vpn, admin) and a per-domain summary. Watch a list of domains and get only what is new since the last run.

What makes this actor different is not the source. It is what happens when crt.sh quietly returns less than it has.

### Why a plain crt.sh query is not enough

crt.sh is free and excellent, but it loses data in three ways, and none of them produces an error:

1. **Hyphenated domains lose their subdomains.** crt.sh's default search splits the name at the hyphen. For coca-cola.com a plain query returns **3 host names**. Asked with a pattern match instead, the same single query returns 666, and it is cut too (its newest entry is from 2021-10-08). With slicing this actor returns **1,271 host names**.
2. **Large domains are cut, and the newest certificates are the ones cut.** There is no pagination (`limit` and `offset` are ignored). When a domain has more than roughly 10000 matching names, crt.sh answers HTTP 200 with only the oldest certificates. For github.io the plain query's newest certificate is from 2021-10-16, although valid certificates from this year exist.
3. **Heavy queries time out, and the timeout looks like "not found".** When its database gives up after about 60 seconds, crt.sh answers HTTP 404, the same status a missing page gets.

This actor detects each case and works around it by **slicing**: it asks for the currently valid certificates separately, detects a cut answer, and then walks the domain zone by zone (`api.example.com`, `mail.example.com`, ...), adding zones it discovers along the way, until the answer is whole or your limit is reached.

### Measured: one crt.sh query versus this actor

Same day, same source, default limit of 40 queries per domain. Host names are unique names that belong to the domain.

| Domain | One plain query | This actor | Queries |
|---|---|---|---|
| mercedes-benz.com | 4 host names, 98 certificates | **7,236 host names**, 23,080 certificates | 40 |
| coca-cola.com | 3 host names, 203 certificates | 1,271 host names, 3,558 certificates | 40 |
| microsoft.com | 3,778 host names, 5,827 certificates | **10,804 host names**, 37,266 certificates | 40 |
| github.io | 329 host names, 2,558 certificates | 345 host names, 3,281 certificates | 40 |
| apify.com | 40 host names, 515 certificates | 40 host names, 515 certificates | 2 |

The last row matters as much as the others: a small domain needs only 2 queries. The default input (apify.com) finished on the Apify platform with 41 rows in 9.2 seconds. Slicing only starts when crt.sh actually cut the answer.

All four large domains stopped at the 40-query limit and are reported as `partial` in their summary row. Raise `maxQueriesPerDomain` to go further.

Reproduce: `python3 tools/measure.py mercedes-benz.com`.

### What you get

Choose with `output`:

- **Subdomains** (default): one row per unique host name, with first and last time it appeared in a certificate, how many certificates named it, which issuers, the latest expiry date, days left, status, and whether a label looks sensitive.
- **Certificates**: one row per unique certificate. crt.sh logs most certificates twice (a precertificate and the final certificate, with different ids). They are merged: apify.com has 980 log entries but only 515 certificates.
- **Both**.

Every domain also gets a **summary row**: counts of valid, expiring, expired and self-signed certificates, issuer shares, the list of suspicious subdomains, other domains that share certificates with this one, and how completely crt.sh answered (`complete`, `sliced`, `partial`, `failed`) with the reason.

Example subdomain row from a real run:

```json
{
  "record_type": "subdomain",
  "domain": "apify.com",
  "hostname": "*.preview.docs.apify.com",
  "is_wildcard": true,
  "subdomain_level": 2,
  "first_seen": "2025-06-24T00:00:00Z",
  "last_seen": "2026-05-25T00:00:00Z",
  "certificate_count": 2,
  "issuers": ["Amazon"],
  "latest_not_after": "2026-12-08T23:59:59Z",
  "days_to_expiry": 82,
  "status": "VALID",
  "has_valid_certificate": true,
  "latest_issuer": "Amazon",
  "latest_serial_number": "0a076daa9e1ca93acb16d1dd52e700ca",
  "latest_crtsh_url": "https://crt.sh/?id=26620732136",
  "is_suspicious": true,
  "suspicious_keywords": ["preview"],
  "suspicious_categories": ["environment"]
}
```

The dataset has 4 views in the Apify Console: Subdomains, Certificates, Domain summary and Errors.

### Useful findings, not just raw rows

- **Expired and expiring certificates.** `status` is `VALID`, `EXPIRING`, `EXPIRED` or `NOT_YET_VALID`. `EXPIRING` means within `expiringWithinDays` (default 30).
- **Suspicious host names.** 5 categories and 73 built-in keywords: environments (dev, staging, uat, preview), remote access (vpn, admin, sso, owa, citrix), DevOps (jenkins, gitlab, grafana, argocd), data (db, backup, ftp, phpmyadmin) and internal (intranet, corp, legacy). A keyword must equal a whole label part, so `devices.example.com` and `adminton.example.com` are not flagged. Add your own with `suspiciousKeywords`.
- **Issuer shares.** Which certificate authorities issue for the domain and what share each has.
- **Self-signed certificates.** Detected from the issuer. With `fetchFingerprints` on, detected exactly from the certificate bytes.
- **Related domains.** Other registrable domains named in the same certificates, often sister brands or old names.
- **Fingerprints (optional).** `fetchFingerprints` downloads certificates and adds SHA256 and SHA1 fingerprints, key algorithm and size, and signature algorithm. The final certificate is preferred over the precertificate, because only the final certificate's fingerprint matches what a server presents.

### Monitor mode: only what is new

Turn on `monitorMode` and schedule the actor, for example daily. The first run delivers everything and remembers it. Every later run delivers only certificates and host names that were not delivered before, marked `change_type: NEW`, plus the summary row with `new_certificates` and `new_hostnames`.

- Memory is kept in a named storage per `monitorName`, so several schedules do not interfere.
- Only rows that actually reached your dataset are remembered. If your spending limit stops a run halfway, the rows you did not get are still new next time.
- Certificate Transparency is append-only, so a change is always an addition: a new certificate, a new host name.

### Batch checks

Put any number of domains into `domains`. URLs, e-mail addresses and `*.` prefixes are cleaned (`https://www.example.com/path` becomes `example.com`). Duplicates are removed. Domains are processed one after another, because crt.sh blocks parallel requests.

### When crt.sh does not answer

crt.sh is a free public service and is often slow or overloaded. What we measured:

- **Parallel requests fail.** Four requests at once got HTTP 429, 502 and HTML error pages instead of JSON for every query. The actor sends one request at a time with at least 2 seconds between them.
- **Very large domains time out.** microsoft.com: the query for its currently valid certificates got no answer in 90 seconds, twice in a row (186.9 seconds in total). In another measurement crt.sh answered the same query with HTTP 404 after about 60 seconds.
- **Large domains are cut.** github.io's plain query returned 5,055 log entries, oldest first, ending at 2021-10-16. Its 49 currently valid certificates were all missing from it.
- **Load changes by the minute.** In our microsoft.com measurement, 2 of 40 queries failed and the rest answered.

What the actor does about it:

- A fast failure (429, 5xx, an HTML page, a quick 404) is retried after cooling down 6, 15 and 30 seconds.
- A slow failure (a timeout, or a 404 after 30 seconds or more) is **not** retried, because the same query dies the same way. That part of the domain is split into zones instead.
- After 4 failures in a row the domain stops, so one broken domain cannot burn an hour of your run.
- `maxQueriesPerDomain` (default 40) and `maxMinutesPerDomain` (default 15) cap the work per domain.

**You never get a silent empty result.** If nothing could be read for a domain, the dataset gets an **error row** with `error_code` and `error_message`, for example `SOURCE_GAVE_UP` or `SOURCE_TIMEOUT`. Error rows are not charged. If some queries failed but others worked, you get the data plus a summary row with `completeness: partial` and `failure_reason`. The run report (`REPORT` in the key-value store) lists every crt.sh query with its status, rows, new rows and time.

### Input

| Field | Default | What it does |
|---|---|---|
| `domains` | apify.com | Domains to search, subdomains included. |
| `output` | subdomains | `subdomains`, `certificates` or `both`. |
| `filter` | all | `all`, `valid`, `expiring`, `expired`, `suspicious`, `wildcard`. The summary row is always delivered. |
| `includeSummary` | true | One summary row per domain. |
| `expiringWithinDays` | 30 | Window for `EXPIRING`, from 1 to 825 days. |
| `suspiciousKeywords` | empty | Extra keywords. |
| `replaceDefaultKeywords` | false | Use only your keywords. |
| `monitorMode` | false | Deliver only what is new since the last run. |
| `monitorName` | default | Separate memory per watchlist. |
| `emitSeenInMonitor` | false | In monitor mode, also deliver rows seen before (`change_type: SEEN`). |
| `fetchFingerprints` | false | Download certificates for fingerprints and key details. |
| `maxFingerprintsPerDomain` | 25 | Cap for the downloads, valid certificates first. |
| `includeSubdomains` | true | Off: exact domain only. |
| `includeExpired` | true | Off: currently valid certificates only, one query per domain. |
| `sliceLargeDomains` | true | Off: no slicing. |
| `maxQueriesPerDomain` | 40 | From 2 to 400. |
| `maxMinutesPerDomain` | 15 | From 1 to 180. |
| `maxRowsPerDomain` | 0 | 0 means no limit. |
| `compactOutput` | false | Drop empty fields and long id lists, for AI agents and spreadsheets. |
| `requestTimeoutSecs` | 70 | Per request, from 15 to 180. crt.sh itself gives up after about 60 seconds. |

### Compared with other crt.sh actors in the Store

Checked on the public Store pages and input schemas:

- None of them handles the hyphen split, the silent cut or the 404-on-timeout. A single query per domain returns 4 host names for mercedes-benz.com where this actor returns 7,236.
- They return log entries, so most certificates appear twice. This actor merges them.
- Monitor mode, suspicious host names, issuer shares, related domains, fingerprints and named error rows are here in one actor.
- `ryanclinton/crt-sh-search` adds risk scores, typosquat detection and executive summaries. This actor does not score risk or look for look-alike domains. If that is what you need, use that actor.

### Honest limits

- **crt.sh is the only source.** Subdomains that never had a publicly trusted certificate (internal names, names covered only by a wildcard) are not in Certificate Transparency and will not be found. Tools that also use DNS brute force or passive DNS find more names, and fewer of them are verified by a certificate.
- **Large domains are slow.** crt.sh answers one query in seconds to a minute. 40 queries took between 449.7 and 656.6 seconds per large domain in our measurements. The number of queries, not rows, sets the run time.
- **Partial is normal for very large domains.** microsoft.com had 2 of 40 queries fail and still stopped at the limit. The summary row says so.
- **No DNS resolution or HTTP probing.** A host name in a certificate does not mean it resolves today. `has_valid_certificate` is the closest signal.
- **Self-signed detection without fingerprints compares issuer and subject common names.** Publicly logged certificates are almost never self-signed, so this is usually zero.

### FAQ

#### How is this different from opening crt.sh in a browser?

The browser shows the same cut result. This actor detects the cut, slices the search, merges duplicate log entries, and turns certificates into subdomain rows with expiry status.

#### Do I need an API key or a proxy?

No. crt.sh is public. The actor uses no proxy and no browser.

#### Why did my run take several minutes for one domain?

The domain was large enough for crt.sh to cut or time out, so the actor sliced it. The summary row shows `source_truncated`, `queries_made` and `rows_from_single_query`. Lower `maxQueriesPerDomain` for faster, less complete runs.

#### Why did a domain return an error row?

crt.sh did not answer for that domain. The row names the reason. You were not charged for it. Try again later, or lower the load by giving fewer domains per run.

#### Can I track new subdomains of my competitors or my own company?

Yes. Turn on `monitorMode`, put the domains into `domains`, and schedule the actor daily. You will get only new host names and certificates.

#### Does it find wildcard certificates?

Yes. Wildcard names keep their `*.` prefix and have `is_wildcard: true`. Use `filter: wildcard` to get only those.

#### What else is this called?

Other names for this job: phishing subdomain detection, shadow IT discovery, apex domain detection, CT log monitoring, certificate validity days. The fields behind them are `is_suspicious`, `related_domains`, `is_apex`, `log_entries` and `validity_days`.

# Actor input Schema

## `domains` (type: `array`):

One or more domains. URLs, e-mail addresses and a leading \*. are cleaned automatically. Each domain is searched together with all of its subdomains.

## `output` (type: `string`):

subdomains: one row per unique host name found in any certificate. certificates: one row per unique certificate (a precertificate and its final certificate are merged). both: both kinds of rows.

## `filter` (type: `string`):

Keep only matching rows. The per-domain summary row is always delivered, so a filter never turns a run into an empty dataset.

## `includeSummary` (type: `boolean`):

Counts of valid, expiring, expired and self-signed certificates, issuer shares, suspicious subdomains, related domains sharing certificates, and how completely crt.sh answered.

## `expiringWithinDays` (type: `integer`):

A certificate whose expiry date falls within this many days is marked EXPIRING.

## `suspiciousKeywords` (type: `array`):

Added to the built-in list (dev, staging, test, vpn, admin, jenkins, gitlab, backup, ftp, internal and others). A host name is suspicious when one of its labels, split on dots, hyphens and underscores, equals a keyword.

## `replaceDefaultKeywords` (type: `boolean`):

Ignore the built-in keyword list and match only the keywords given above.

## `monitorMode` (type: `boolean`):

Remembers every certificate and host name already delivered and returns only new ones on later runs. Schedule the actor daily to get new subdomains and new certificates as they appear. The first run delivers everything and records it.

## `monitorName` (type: `string`):

Memory is kept per watchlist name, so separate schedules do not interfere. Letters, digits and hyphens.

## `emitSeenInMonitor` (type: `boolean`):

In monitor mode, deliver previously seen rows too, each marked change\_type SEEN.

## `fetchFingerprints` (type: `boolean`):

Downloads each certificate from crt.sh to add SHA-256 and SHA-1 fingerprints, public key algorithm and size, signature algorithm, and an exact self-signed check. One extra request per certificate, currently valid certificates first.

## `maxFingerprintsPerDomain` (type: `integer`):

Caps the extra requests made by the fingerprint step.

## `includeSubdomains` (type: `boolean`):

Off: only certificates naming the exact domain.

## `includeExpired` (type: `boolean`):

Off: only currently valid certificates (one fast query per domain).

## `sliceLargeDomains` (type: `boolean`):

crt.sh silently keeps only the oldest certificates when a domain has too many, and answers HTTP 404 when a query runs too long. When that happens the actor splits the search into subdomain zones. Turn off to run a single query per domain.

## `maxQueriesPerDomain` (type: `integer`):

Upper bound for slicing. Small domains need 2 queries.

## `maxMinutesPerDomain` (type: `integer`):

Slicing stops after this time and the summary row reports the result as partial.

## `maxRowsPerDomain` (type: `integer`):

0 means no limit. The summary row is not counted.

## `compactOutput` (type: `boolean`):

Drop empty fields and long lists such as every crt.sh entry id and the full issuer distinguished name. Handy for AI agents and spreadsheets.

## `requestTimeoutSecs` (type: `integer`):

crt.sh itself gives up on a query after about 60 seconds, so values below 65 cut off answers that were about to arrive.

## Actor input object example

```json
{
  "domains": [
    "apify.com",
    "coca-cola.com"
  ],
  "output": "subdomains",
  "filter": "all",
  "includeSummary": true,
  "expiringWithinDays": 30,
  "replaceDefaultKeywords": false,
  "monitorMode": false,
  "monitorName": "default",
  "emitSeenInMonitor": false,
  "fetchFingerprints": false,
  "maxFingerprintsPerDomain": 25,
  "includeSubdomains": true,
  "includeExpired": true,
  "sliceLargeDomains": true,
  "maxQueriesPerDomain": 40,
  "maxMinutesPerDomain": 15,
  "maxRowsPerDomain": 0,
  "compactOutput": false,
  "requestTimeoutSecs": 70
}
```

# Actor output Schema

## `results` (type: `string`):

All collected rows

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "apify.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("snow_leo_data/crtsh-subdomain-finder").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": ["apify.com"] }

# Run the Actor and wait for it to finish
run = client.actor("snow_leo_data/crtsh-subdomain-finder").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "apify.com"
  ]
}' |
apify call snow_leo_data/crtsh-subdomain-finder --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,snow_leo_data/crtsh-subdomain-finder"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/j4IhOXewAVs8xfxK9/builds/quLjnk3PAseE91YfU/openapi.json