# Subdomain Monitoring (`shipsatnight/subdomain-monitoring`) Actor

Subdomain monitoring tool for domains you own: new subdomains from certificate transparency, DNS changes, SSL certificate expiry alerts, and subdomain takeover candidates. Passive only. Reports only what changed.

- **URL**: https://apify.com/shipsatnight/subdomain-monitoring.md
- **Developed by:** [Donovan](https://apify.com/shipsatnight) (community)
- **Categories:** Developer tools, Automation
- **Security:** Limited permissions
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$500.00 / 1,000 domain digests

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Subdomain Monitoring

Subdomain Monitoring is a passive subdomain monitoring tool for domains you own. Each run it finds new subdomains in public certificate transparency logs, checks DNS records, tracks SSL certificate expiry, records the HTTP status and page title of each live host, and flags dangling CNAME records that could allow a subdomain takeover. It compares the result with the last run and reports only what changed, as dataset rows and an email-ready HTML digest. Use it for lightweight attack surface monitoring without running scanners yourself.

### What does Subdomain Monitoring do?

It answers one question on a schedule: "what changed on my domain since last time?" You give it the apex domains you own. It proves ownership through a DNS TXT record on every run, discovers hosts from certificate transparency (crt.sh and Cert Spotter), resolves A, AAAA, CNAME, MX, NS and TXT records, reads each certificate's expiry date with one TLS handshake, and makes one HTTP GET per live host. The first run stores a baseline. Later runs output only new subdomains, DNS changes, certificate changes, SSL expiry alerts, HTTP status or title changes, and subdomain takeover candidates.

### Who is this subdomain monitoring tool for?

- Small IT teams and solo admins who want SSL expiry notifications and DNS change tracking without a platform contract.
- MSPs and agencies that need one weekly digest across many client domains.
- Security people who want to automate subdomain monitoring of their own external attack surface with a passive, low-noise feed.

### How to verify that you own the domain

Scanning is refused unless this record exists. It is checked again on every run, so removing it stops monitoring.

1. Pick a random token, 16 to 64 letters, digits, `-` or `_`. Example: `your-random-token-here`.
2. Add a TXT record:
   - Name: `_shipsatnight-verify.example.com`
   - Value: `domain-change-digest-verify=your-random-token-here`
3. Put the same token in the `verificationToken` input. The saved Tasks ship with a placeholder token; the run refuses it until you replace it.

If the record is missing, wrong, or the lookup fails, the domain is not scanned and not charged. When at least one other domain in the run verifies, a `verification_failed` row explains what to fix. Your token is never printed in rows, the digest, or logs. If no domain passes, the run ends with a status message, nothing is charged, and no dataset rows are written.

### What it checks (and what it never does)

| Check | How | Limit |
|---|---|---|
| New subdomains | certificate transparency logs: crt.sh and Cert Spotter public APIs | 500 hosts per domain |
| DNS change monitoring | A, AAAA, CNAME, MX, NS, TXT via public resolvers | per host |
| SSL certificate expiry monitoring and validity | one TLS handshake on port 443, fully validated against the host name (trusted chain, not expired, name match) | live hosts only |
| HTTP status and title | one GET to `https://host/` with the same certificate validation, one GET to `http://host/` only if HTTPS cannot connect at all, redirects not followed. No HTTPS page check when the certificate is invalid | 64 KB read |
| Subdomain takeover monitoring | CNAME target returns NXDOMAIN, or the response matches a known "unclaimed" page for that CNAME's service | passive only |

It never does port scans, IP or CIDR range scans, vulnerability templates, exploit attempts, or login attempts. Only hosts under a verified apex are touched. Requests identify themselves with the user agent `domain-change-digest/0.1 (+https://apify.com/shipsatnight)`.

Limits:

- At most 500 hosts per domain per run.
- At most one scan per domain per UTC day, tracked in the `domain-change-digest-state` store in your own Apify account. Extra runs are skipped and not charged. A best-effort lock stops two runs from scanning the same domain at the same moment.
- The daily limit can be reset: deleting the `domain-change-digest-state` store, or running from another Apify account, starts fresh. The lock is best effort, not a guarantee.
- Charges are per domain, not per run: each domain that delivers changes is one `domain-digest` event.
- Run timeout: at least 60 seconds plus 90 seconds per domain. A shorter timeout stops the run at the start, with nothing scanned or charged. The default of 3,600 seconds covers at most 39 domains. For 40 to 50 domains, raise it to at least 60 + 90 x (number of domains) seconds, for example 4,560 seconds for 50: in Apify Console open the Actor or Task, go to Run options (Settings on a Task), and set Timeout; through the API, pass `timeout` in seconds when you start the run. The saved MSP Multi-Domain Digest Task already uses 4,560 seconds.

Hosts that resolve only to private, loopback, link-local or cloud metadata addresses are never contacted; only their DNS records are tracked. Every TLS and HTTP connection goes to the IP address that was already resolved, so a DNS change between lookup and connection cannot redirect it.

### Input example

```json
{
  "domains": ["example.com"],
  "verificationToken": "your-random-token-here",
  "expiryWarningDays": 30,
  "maxHostsPerDomain": 500,
  "checkHttp": true
}
```

### Output example

For each domain that has changes, the dataset gets one `domain_digest` row (the paid row, which holds every change in `changes`) plus one row per change for easy filtering:

```json
{
  "domain": "example.com",
  "host": "old.example.com",
  "changeType": "takeover_candidate",
  "severity": "high",
  "field": "CNAME",
  "before": null,
  "after": "gone-app.herokuapp.com",
  "message": "Possible subdomain takeover on old.example.com: CNAME points to gone-app.herokuapp.com, which does not resolve (NXDOMAIN). Remove the DNS record or reclaim the resource.",
  "detectedAt": "2026-11-04T21:00:00.000Z"
}
```

Change types: `baseline_created`, `subdomain_new`, `host_removed`, `host_not_public`, `host_started_resolving`, `host_stopped_resolving`, `dns_changed`, `cert_changed`, `cert_invalid`, `cert_expiring`, `cert_expired`, `http_status_changed`, `http_title_changed`, `takeover_candidate`, `takeover_cleared`, `verification_failed`, `skipped_daily_limit`, `run_locked`, `host_cap_reached`, `source_error`.

Certificate problems (expired, self-signed, untrusted chain, or a certificate for another name) are reported as a high-severity `cert_invalid` row, never as healthy.

The key-value store also gets `OUTPUT_HTML` (an email-ready digest you can send with an Apify integration) and `SUMMARY`. State lives in a named key-value store `domain-change-digest-state` in your own Apify account, so the diff survives between runs.

### Pricing

Pay per event. Price at launch:

| What | Price at launch |
|---|---|
| One domain scanned, with at least one change delivered (event `domain-digest`) | $0.50 |
| Domain scanned, nothing changed since the last run | $0 |
| Domain that fails ownership verification | $0 |
| Domain skipped by the one-run-per-day limit or because another run holds it | $0 |

Example arithmetic only: if 10 domains each had changes on 4 weekly runs in a month, that would be 40 x $0.50 = $20. Quiet weeks cost nothing.

Charging and delivery happen in one step: the domain's `domain_digest` row is saved together with its `domain-digest` charge, so you never pay without getting the digest. If your maximum spend per run is reached, that domain's changes are not delivered and not charged; they are kept and reported on the next run, and remaining domains are skipped.

### How to automate subdomain monitoring

1. Add the TXT record and run once to store the baseline.
2. Create a schedule (Apify Console, Schedules) weekly or daily.
3. Add an email or Slack integration that sends `OUTPUT_HTML`, or read the dataset by API. See [Apify schedules](https://docs.apify.com/platform/schedules) and [integrations](https://docs.apify.com/platform/integrations).

Ready-made saved inputs: Weekly Subdomain Monitor, SSL Certificate Expiry Alerts, Subdomain Takeover Check, MSP Multi-Domain Digest.

### How does this compare with other approaches?

| Approach | Setup | Ongoing work | Covers forgotten subdomains | Active scanning |
|---|---|---|---|---|
| This actor | TXT record and a schedule | none | yes, from certificate logs | no |
| Calendar reminders for certificate renewals | minutes | manual, per certificate | no | no |
| Self-hosted open source recon tools on a server | hours | patching, storage, diffing, alerting | yes | often yes |
| Single-certificate uptime checks | minutes | add each host by hand | no | no |
| Full external attack surface management platforms | sales process | low | yes | usually yes, priced per asset |

### FAQ

#### What is subdomain takeover?

A subdomain takeover happens when a DNS record (usually a CNAME) still points to a third-party service you stopped using, such as a deleted app or bucket. Anyone who claims that resource on the service can then serve content on your subdomain. This actor flags CNAMEs whose target no longer exists and CNAMEs that land on a known "not claimed" page, so you can delete the record.

#### How do I monitor SSL certificate expiration for all my subdomains?

Run this actor on a schedule with your apex domain. It finds hosts through certificate transparency, reads each live certificate's expiry date, and alerts when one is within your warning window (30 days by default) and again if it expires.

#### What are certificate transparency logs?

Public, append-only logs that record every publicly trusted TLS certificate. Because almost every HTTPS subdomain gets a certificate, the logs reveal subdomains you may have forgotten. See [certificate.transparency.dev](https://certificate.transparency.dev/).

#### What is DNS monitoring?

Watching your DNS records for unexpected changes. A changed NS or MX record can mean a hijacked registrar account or a misconfiguration. This actor diffs A, AAAA, CNAME, MX, NS and TXT records per host between runs.

#### How often do SSL certificates expire?

Public certificates now last at most 398 days, and the CA/Browser Forum approved a schedule that reduces this in steps to 47 days by 2029 ([Ballot SC-081](https://cabforum.org/2025/04/11/ballot-sc081v3-introduce-schedule-of-reducing-validity-and-data-reuse-periods/)). Many free certificates last 90 days. Shorter lifetimes mean more renewals that can silently fail.

#### What does attack surface monitoring mean?

Keeping an up-to-date list of what you expose to the internet and noticing when it changes. This actor covers the passive part: hostnames, DNS, certificates and basic HTTP responses. It does not scan ports.

#### Can I monitor a domain I do not own?

No. The TXT verification record is required on every run, and IP addresses and ranges are refused.

#### Is it safe to run against production?

It is designed to be light. Per live host it makes one TLS handshake and one GET request (a second plain-HTTP GET only if HTTPS cannot connect), with a clear user agent, and it never contacts non-public addresses. A GET can still trigger application logic or show up in your logs and alerts, so turn off `checkHttp` if any host should not receive requests, and review the first run.

### Libraries and sources

- [Apify SDK for JavaScript](https://docs.apify.com/sdk/js/)
- Node.js [dns](https://nodejs.org/api/dns.html) and [tls](https://nodejs.org/api/tls.html) modules
- [crt.sh](https://crt.sh/) and [Cert Spotter API](https://sslmate.com/help/reference/ct_search_api_v1) for certificate transparency data
- Apify docs: [pay-per-event pricing](https://docs.apify.com/platform/actors/publishing/monetize), [key-value stores](https://docs.apify.com/platform/storage/key-value-store), [schedules](https://docs.apify.com/platform/schedules)

### Related actors in Apify Store

- [Real Subdomain Finder](https://apify.com/onescales/real-subdomain-finder) for one-off subdomain discovery
- [SSL Certificate Checker](https://apify.com/automation-lab/ssl-certificate-checker) for one-off certificate checks

### More from shipsatnight

Like Subdomain Monitoring? See [Bulk WHOIS, DNS, SSL & Tech Stack Lookup API](https://apify.com/shipsatnight/domain-intelligence-suite) for one-shot lookups on any list of domains.

Also by shipsatnight:

- [Username Availability Checker: GitHub, npm, X](https://apify.com/shipsatnight/username-monitor)
- [Security Jobs Monitor: Greenhouse, Lever, Ashby](https://apify.com/shipsatnight/security-hiring-signals)
- [Google Trends Scraper: Interest, Regions, Trending](https://apify.com/shipsatnight/google-trends-scraper)

### Disclaimer

Findings are hints from public data, not proof of compromise. A takeover candidate means "check this record", not "this is exploitable". Use only on domains you own or administer.

# Actor input Schema

## `domains` (type: `array`):

Apex domains to monitor, for example example.com. Each one must carry the verification TXT record described below. IP addresses and ranges are refused. Up to 50 domains per run.

## `verificationToken` (type: `string`):

A random string of 16 to 64 letters, digits, '-' or '\_' that you choose. Publish it in DNS as a TXT record named \_shipsatnight-verify.<your-domain> with the value domain-change-digest-verify=<token>. The record is checked on every run. Domains without it are not scanned and not charged.

## `expiryWarningDays` (type: `integer`):

Alert when a certificate expires within this many days.

## `maxHostsPerDomain` (type: `integer`):

Hosts checked per domain per run. Hard cap 500.

## `checkHttp` (type: `boolean`):

Send one GET request per live host to record status code and page title, and to spot unclaimed-service pages. Turn off for DNS and certificate checks only.

## Actor input object example

```json
{
  "domains": [
    "example.com"
  ],
  "expiryWarningDays": 30,
  "maxHostsPerDomain": 500,
  "checkHttp": true
}
```

# Actor output Schema

## `changes` (type: `string`):

No description

## `digest` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "example.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("shipsatnight/subdomain-monitoring").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": ["example.com"] }

# Run the Actor and wait for it to finish
run = client.actor("shipsatnight/subdomain-monitoring").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "example.com"
  ]
}' |
apify call shipsatnight/subdomain-monitoring --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,shipsatnight/subdomain-monitoring"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/stKpJCoM4XzC5YkpK/builds/pYec6ksQPPuPf2giq/openapi.json