# Domain Inspector - DNS, RDAP, HTTP, TLS and Tech (`dami_studio/domain-inspector`) Actor Inspect domains in batches. Get DNS, RDAP registration, HTTP redirects, page metadata, TLS certificate details, security headers, robots.txt, sitemaps, and technology signals in one row per domain. Private-network access is blocked; proxy use is optional and off by default. - **URL**: https://apify.com/dami\_studio/domain-inspector.md - **Developed by:** [Dami's Studio](https://apify.com/dami_studio) (community) - **Categories:** Developer tools, SEO tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing $1.50 / 1,000 domains This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Domain Inspector Inspect up to 500 domains in one run and receive one consolidated row per domain. The actor combines DNS, RDAP, HTTP, HTTPS, TLS, security, discovery, and basic technology data without requiring an API key or a proxy. ### What it checks - DNS: A, AAAA, MX, NS, TXT, and CAA records - Registration: RDAP handle, registrar, statuses, registration, update, and expiration dates - Web: HTTP and HTTPS status, redirect chain, final URL, title, description, canonical URL, language, Open Graph metadata, and response headers - TLS: subject, issuer, SANs, valid-from and valid-to dates, days until expiry, fingerprints, protocol, and trust result - Security headers: HSTS, CSP, X-Content-Type-Options, X-Frame-Options, Referrer-Policy, Permissions-Policy, COOP, and CORP - Discovery: standard robots.txt and sitemap.xml presence plus sitemap URLs declared in robots.txt - Technology signals: common CMS, frameworks, analytics, CDN, hosting, and server hints visible in public headers or HTML The result is designed for domain due diligence, lead enrichment, portfolio monitoring, security triage, SEO checks, and dataset enrichment. ### Why use this actor - One row per domain instead of separate DNS, certificate, and page datasets - Batch input with bounded concurrency and strict operation timeouts - Direct connections by default, so normal runs avoid residential proxy bandwidth - Private, loopback, link-local, reserved, test, and internal destinations are blocked - Redirect destinations are resolved and checked before every request - Partial failures keep all useful data and add precise warnings - Totally failed domains return uncharged diagnostics instead of failing the run - Empty input returns one clearly labeled, uncharged sample row for Store previews and automated checks ### Input | Field | Default | Description | |---|---:|---| | `domains` | empty | Up to 500 domains or public HTTP(S) URLs. URLs are reduced to hostnames and duplicates are removed. | | `domain` | empty | Optional single-domain shortcut, combined with `domains`. | | `maxConcurrency` | `5` | Domains inspected at once, from 1 to 20. | | `requestTimeoutSecs` | `8` | Timeout for each network operation, from 3 to 20 seconds. | | `maxRedirects` | `5` | Redirects followed for each web request, from 0 to 10. | | `checkRdap` | `true` | Include public RDAP registration data. | | `checkWeb` | `true` | Include HTTP and HTTPS checks, metadata, headers, and technology signals. | | `checkTls` | `true` | Include the port 443 certificate check. | | `checkRobotsAndSitemap` | `true` | Check standard discovery files on the best working origin. | | `proxyConfiguration` | off | Optional proxy for web requests only. DNS, RDAP, and TLS remain direct. | Example: ```json { "domains": [ "example.com", "https://apify.com/store" ], "maxConcurrency": 5, "requestTimeoutSecs": 8, "checkRdap": true, "checkWeb": true, "checkTls": true, "checkRobotsAndSitemap": true, "proxyConfiguration": { "useApifyProxy": false } } ```` ### Output Successful records have `ok: true` and contain both flat summary fields for the dataset view and detailed nested objects: ```json { "ok": true, "domain": "example.com", "inspectedAt": "2026-07-25T15:00:00.000Z", "httpsStatus": 200, "httpStatus": 301, "title": "Example Domain", "dnsA": ["93.184.216.34"], "tlsExpiresAt": "2027-01-15T23:59:59.000Z", "registrationExpiresAt": "2026-08-13T04:00:00.000Z", "technologies": ["IANA"], "warningsCount": 0, "dns": { "a": ["93.184.216.34"], "aaaa": [], "mx": [], "ns": ["a.iana-servers.net", "b.iana-servers.net"], "txt": [], "caa": [] }, "rdap": { "available": true, "handle": "2336799_DOMAIN_COM-VRSN", "statuses": ["client delete prohibited"], "registeredAt": "1995-08-14T04:00:00.000Z", "expiresAt": "2026-08-13T04:00:00.000Z" }, "http": { "status": 301, "finalUrl": "https://example.com/", "redirects": [] }, "https": { "status": 200, "finalUrl": "https://example.com/", "metadata": { "title": "Example Domain", "description": null, "canonicalUrl": null }, "securityHeaders": { "present": [], "missing": ["strict-transport-security"] } }, "tls": { "available": true, "authorized": true, "expiresAt": "2027-01-15T23:59:59.000Z", "daysUntilExpiry": 174 }, "robots": { "present": true, "status": 200 }, "sitemap": { "present": false, "status": 404 }, "warnings": [] } ``` Fields unavailable for a specific TLD, host, protocol, or certificate remain empty or `null`. A domain is still successful when at least one requested subsystem returns useful public data. Warnings explain every requested subsystem that could not be completed. Invalid or totally failed domains return `ok: false`, an `errorCode`, and warnings. These rows are diagnostics and are not charged. ### Charging and planned Store price The actor calls the pay-per-event event `domain` exactly once after each successful result row is saved. Sample rows and diagnostic rows never call the event. The intended Store price is **$0.0015 per successful domain**, below the compared marketplace price of **$0.002 per domain**. At 1,000 successful domains that is $1.50 gross customer spend. Proxy is off by default because residential bandwidth would reduce or remove this margin. Before publishing, confirm the final Apify platform fee and run-cost benchmark with representative batches, then configure the `domain` event price in the Console. ### Proxy guidance No proxy is needed for normal DNS, RDAP, TLS, or public-site checks. Enable a proxy only when a target site blocks Apify datacenter traffic and the additional result value justifies the bandwidth cost. When enabled, the proxy is used only for HTTP and HTTPS page, robots.txt, and sitemap.xml requests. The actor still resolves every target and redirect locally first and rejects non-public addresses. DNS, RDAP, and TLS never use the proxy. ### Safety and limits - Inputs must normalize to a public domain with at least two labels. - IP literals and internal suffixes such as `.local`, `.internal`, `.test`, and `.localhost` are rejected. - DNS answers containing private, loopback, link-local, reserved, documentation, or other non-global IP addresses are blocked from web and TLS connections. - Direct web connections are pinned to a checked public DNS answer to reduce DNS-rebinding risk. - HTTP is restricted to port 80 and HTTPS to port 443, including redirects. - HTML reads are capped, redirect counts are bounded, and every operation has a timeout. - This actor reports public technical metadata. It does not scan ports, bypass authentication, or access private networks. ### Run locally ```bash npm ci npm test npm run test:live npm start ``` The live smoke test uses `example.com` and requires outbound DNS and HTTPS access. `npm start` with no local input writes one uncharged sample record to the default local dataset. # Actor input Schema ## `domains` (type: `array`): Domains or public HTTP(S) URLs to inspect, one per line. URLs are reduced to their hostname. Duplicates are removed. Maximum 500. ## `domain` (type: `string`): Optional single-domain shortcut. It is combined with the domains list. ## `maxConcurrency` (type: `integer`): Number of domains inspected at once. Keep this modest to avoid DNS and target-site rate limits. ## `requestTimeoutSecs` (type: `integer`): Timeout for each DNS, RDAP, HTTP, or TLS operation. A hard per-domain deadline is also applied. ## `maxRedirects` (type: `integer`): Maximum redirects followed separately for HTTP, HTTPS, robots.txt, and sitemap.xml. Every redirect target is safety-checked. ## `checkRdap` (type: `boolean`): Look up public registration dates, status, registrar, nameservers, and handle where the registry provides RDAP data. ## `checkWeb` (type: `boolean`): Inspect both HTTP and HTTPS status, redirect chains, final URL, title, metadata, security headers, and basic technology signals. ## `checkTls` (type: `boolean`): Inspect the certificate on port 443, including validity dates, issuer, subject, SANs, fingerprint, and Node.js trust result. ## `checkRobotsAndSitemap` (type: `boolean`): Check standard robots.txt and sitemap.xml locations on the best working web origin. ## `proxyConfiguration` (type: `object`): Optional. Off by default because public domain checks normally need no proxy. When enabled, it is used only for HTTP(S), robots.txt, and sitemap.xml. DNS, RDAP, and TLS always connect directly. Proxy bandwidth can materially reduce margin. ## Actor input object example ```json { "domains": [ "example.com", "https://apify.com/" ], "domain": "example.com", "maxConcurrency": 5, "requestTimeoutSecs": 8, "maxRedirects": 5, "checkRdap": true, "checkWeb": true, "checkTls": true, "checkRobotsAndSitemap": true, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `results` (type: `string`): One consolidated dataset row per normalized domain. Successful rows include all available checks and warnings. Invalid or totally unreachable domains return uncharged diagnostic rows. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("dami_studio/domain-inspector").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("dami_studio/domain-inspector").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call dami_studio/domain-inspector --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=dami_studio/domain-inspector", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Domain Inspector - DNS, RDAP, HTTP, TLS and Tech", "description": "Inspect domains in batches. Get DNS, RDAP registration, HTTP redirects, page metadata, TLS certificate details, security headers, robots.txt, sitemaps, and technology signals in one row per domain. Private-network access is blocked; proxy use is optional and off by default.", "version": "0.1", "x-build-id": "KOJcArmtsY1ihRpFw" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/dami_studio~domain-inspector/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-dami_studio-domain-inspector", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/dami_studio~domain-inspector/runs": { "post": { "operationId": "runs-sync-dami_studio-domain-inspector", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/dami_studio~domain-inspector/run-sync": { "post": { "operationId": "run-sync-dami_studio-domain-inspector", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "domains": { "title": "Domains or URLs", "maxItems": 500, "type": "array", "description": "Domains or public HTTP(S) URLs to inspect, one per line. URLs are reduced to their hostname. Duplicates are removed. Maximum 500.", "items": { "type": "string" } }, "domain": { "title": "Single domain", "type": "string", "description": "Optional single-domain shortcut. It is combined with the domains list." }, "maxConcurrency": { "title": "Concurrent domains", "minimum": 1, "maximum": 20, "type": "integer", "description": "Number of domains inspected at once. Keep this modest to avoid DNS and target-site rate limits.", "default": 5 }, "requestTimeoutSecs": { "title": "Per-request timeout", "minimum": 3, "maximum": 20, "type": "integer", "description": "Timeout for each DNS, RDAP, HTTP, or TLS operation. A hard per-domain deadline is also applied.", "default": 8 }, "maxRedirects": { "title": "Maximum redirects", "minimum": 0, "maximum": 10, "type": "integer", "description": "Maximum redirects followed separately for HTTP, HTTPS, robots.txt, and sitemap.xml. Every redirect target is safety-checked.", "default": 5 }, "checkRdap": { "title": "Check RDAP registration", "type": "boolean", "description": "Look up public registration dates, status, registrar, nameservers, and handle where the registry provides RDAP data.", "default": true }, "checkWeb": { "title": "Check HTTP and HTTPS", "type": "boolean", "description": "Inspect both HTTP and HTTPS status, redirect chains, final URL, title, metadata, security headers, and basic technology signals.", "default": true }, "checkTls": { "title": "Check TLS certificate", "type": "boolean", "description": "Inspect the certificate on port 443, including validity dates, issuer, subject, SANs, fingerprint, and Node.js trust result.", "default": true }, "checkRobotsAndSitemap": { "title": "Check robots.txt and sitemap.xml", "type": "boolean", "description": "Check standard robots.txt and sitemap.xml locations on the best working web origin.", "default": true }, "proxyConfiguration": { "title": "Proxy for web checks only", "type": "object", "description": "Optional. Off by default because public domain checks normally need no proxy. When enabled, it is used only for HTTP(S), robots.txt, and sitemap.xml. DNS, RDAP, and TLS always connect directly. Proxy bandwidth can materially reduce margin.", "default": { "useApifyProxy": false } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```