Repository Referenced-Hostname Registration and Resolution Gate avatar

Repository Referenced-Hostname Registration and Resolution Gate

Pricing

from $50.00 / 1,000 scan_starts

Go to Apify Store
Repository Referenced-Hostname Registration and Resolution Gate

Repository Referenced-Hostname Registration and Resolution Gate

Find each hostname in a GitHub repository that does not resolve, is not registered, is near expiry, or is on hold.

Pricing

from $50.00 / 1,000 scan_starts

Rating

0.0

(0)

Developer

kingii98

kingii98

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

7 days ago

Last modified

Categories

Share

This Actor reads files in a public GitHub repository. It finds each hostname that the files reference. It then checks each unique registrable domain. Use it before a release and on a weekly schedule. Domain registration state can change without a commit.

What it finds

For each domain, the Actor reports one of these problems:

  • The domain does not resolve.
  • The domain is not registered.
  • The domain is near expiry.
  • The domain is on hold, or it is pending delete.

Input

FieldDefaultMeaning
repo_urlhttps://github.com/octocat/Hello-WorldPublic GitHub repository URL.
refmasterBranch, tag, or commit.
path_globsREADME*, *.md, *.yaml, *.yml, *.json, *.tomlFile patterns to scan.
ignore_domainsnoneDomains to skip. A domain also skips its subdomains.
expiry_warning_days45Warn when a domain expires within this number of days.
github_tokennoneOptional. It raises the GitHub rate limit.

An empty input {} is valid. It uses the defaults above.

Limits

  • Maximum 500 files for each run.
  • Maximum 1 MB for each file. The Actor skips larger files.
  • Maximum 300 unique registrable domains for each run.
  • Each request has a 15-second timeout. The Actor follows at most 3 redirects, and only for RDAP, and only to public HTTPS hosts.
  • The Actor uses HTTP only. It does not use a browser or a proxy.
  • The Actor rejects private and reserved targets. It skips localhost, IP addresses, and names that end in .local, .internal, .test, and similar. It skips example.com, example.org, and example.net.

When a limit stops the scan early, the summary shows truncated: true, and the gate result is at least warn.

How hostnames are found

The Actor reads two patterns:

  1. URLs: http, https, oci, ftp, ws, wss, git, and ssh schemes.
  2. Keys host, hostname, domain, and server with a hostname value.

The Actor does not read bare names in free text. This prevents false matches on file names such as values.yaml.

The Actor finds the registrable domain with a short built-in list of two-label suffixes (for example co.uk). The list is not the full Public Suffix List. A rare suffix can give a wrong domain.

Output

The dataset has one domain row for each unique registrable domain, then one summary row.

Domain row fields: domain, hostnames, locations (hostname, file path, line number; at most 50), dns_name, dns_result, dns_addresses, registration_state, rdap_statuses, expiry_date, days_left, flags, verdict, checked_at.

  • dns_result: resolves, nxdomain, no_records, or error. The Actor resolves the domain itself when the files name it. Otherwise it resolves the first hostname in alphabetical order.
  • registration_state: registered, not_registered, or unknown. An RDAP "not found" answer gives not_registered only when DNS also returns nxdomain. If the name resolves, the state is unknown.
  • flags: not_registered, on_hold, pending_delete, expired, expiring_soon, dns_nxdomain, dns_no_records, dns_error, rdap_unavailable.
  • verdict: fail for not_registered, on_hold, pending_delete, or expired. warn for any other flag. Else pass.

Summary row fields: gate_result (pass, warn, fail, or error), message, repository, ref, file and domain counts, verdict counts, truncated, events_charged.

gate_result is error only when the input is not valid, GitHub gives an error, or the charge limit stops the scan. A failed gate is a normal result. The run ends SUCCEEDED, and the status message gives the verdict.

Billing (pay-per-event)

EventBilling unitWhen the Actor charges
scan_startOne repository scan.One call with count 1, after the input is valid and before the first GitHub request.
domain_checkOne unique registrable domain checked.One call with count equal to the number of domains to check, after file scanning.

The Actor charges domain_check before it checks the domains. If the run budget allows fewer domains, the Actor checks only that number and sets truncated: true. Ignored domains are not charged. A run with no domains has no domain_check charge.

Local run

uv run pytest
uv run ruff check .
uv run python -m src

Local runs use the Apify storage in storage/.

Data sources

  • GitHub API and raw.githubusercontent.com for the file tree and file content.
  • dns.google DNS-over-HTTPS for A and AAAA records.
  • rdap.org for RDAP registration data. It redirects to the registry RDAP server.

Some country-code TLDs have no RDAP service. For these, registration_state is unknown and the verdict is warn.