# Domain Expiry Checker (`datapilot/domain-expiry-checker`) Actor Bulk Domain Expiry Checker scans domains using WHOIS data and optional HTTP checks. It extracts registrar, expiry date, status, name servers, and days left. Flags expiring domains (critical/warning) and outputs structured JSONβ€”ideal for domain monitoring, SEO audits, and portfolio management. πŸŒπŸ“… - **URL**: https://apify.com/datapilot/domain-expiry-checker.md - **Developed by:** [Data Pilot](https://apify.com/datapilot) (community) - **Categories:** SEO tools, Real estate, Social media - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $2.50 / 1,000 scraped results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Domain Expiry Checker ⏰ **Domain Expiry Checker** is a powerful Apify Actor designed to monitor and extract comprehensive **Domain Expiry** information using WHOIS lookups and HTTP status checks. This tool provides detailed **Domain Expiry** data including registration dates, expiration dates, registrars, and **Domain Expiry** alerts for bulk domain portfolios. Whether you're managing corporate domains, tracking web assets, or monitoring **Domain Expiry** schedules, the Domain Expiry Checker delivers critical **Domain Expiry** intelligence efficiently. With async/await architecture, intelligent expiry alert thresholds, WHOIS data extraction, HTTP status validation, and Apify Dataset integration, the Domain Expiry Checker ensures comprehensive monitoring of **Domain Expiry** status and renewal requirements. It focuses on critical **Domain Expiry** metrics including days remaining, expiry alerts, registration history, and registrar information, making it an essential tool for **Domain Expiry** management and digital asset tracking. ### πŸ”₯ Features - **Comprehensive Domain Expiry Tracking** – Extracts detailed **Domain Expiry** information including expiration dates, registration dates, and **Domain Expiry** calculations. - **WHOIS Lookup** – Performs WHOIS queries using python-whois library to retrieve authoritative domain registration data. - **Expiry Alert System** – Automatically calculates remaining days and generates **Domain Expiry** alerts (critical <30 days, warning <90 days). - **HTTP Status Check** – Optional HTTP/HTTPS status validation to verify domain accessibility. - **Registrar Information** – Extracts registrar name, WHOIS server, and registration details. - **Name Server Extraction** – Retrieves up to 6 primary name servers per domain. - **Domain Status** – Captures WHOIS status codes (active, redemption, suspended, etc.). - **Bulk Domain Processing** – Supports multiple domains (newline or list separated). - **Duplicate Removal** – Automatically deduplicates domain input. - **Date Parsing** – Intelligent date parsing from various WHOIS formats. - **Days Calculation** – Automatic calculation of days until **Domain Expiry**. - **Async/Await Architecture** – Concurrent processing with asyncio for faster execution. - **Proxy Support** – Apify residential proxy support for reliable WHOIS lookups. - **Error Handling** – Graceful error handling with detailed error logging. - **ISO 8601 Timestamps** – Records check timestamp for audit trails. - **Real-Time Dataset Push** – Automatically pushes results to Apify Dataset with alerts. --- ### βš™οΈ How It Works The Domain Expiry Checker takes domain names as input, normalizes them, and performs WHOIS lookups to extract registration and expiration data. It calculates remaining days until expiration, generates alert levels based on urgency, and optionally checks HTTP status. All results are pushed to the Apify Dataset with comprehensive metadata for tracking and analysis. **Key Processing Steps:** 1. **Input Parsing** – Accept domains (newline or list separated) 2. **Domain Normalization** – Remove protocols (http://, https://), subdomains (www.), trailing slashes 3. **Deduplication** – Remove duplicate domains 4. **WHOIS Lookup** – Query authoritative WHOIS data for each domain 5. **Date Extraction** – Parse creation, expiry, and update dates 6. **Days Calculation** – Calculate remaining days until **Domain Expiry** 7. **Alert Generation** – Generate **Domain Expiry** alerts based on days remaining 8. **HTTP Check** – Optional status check via HTTP/HTTPS requests 9. **Name Server Extraction** – Parse and normalize name servers 10. **Dataset Push** – Push results to Apify Dataset with full metadata **Key Benefits:** - Monitor **Domain Expiry** dates for entire domain portfolios - Receive alerts for domains expiring soon - Track registration history and changes - Verify domain registrar information - Monitor domain accessibility via HTTP - Automate renewal reminders - Identify domains requiring immediate attention --- ### πŸ“₯ Input The Actor accepts the following input parameters: | Field | Type | Default | Description | |-------|------|---------|-------------| | `domains` | string or array | required | Domain names to check (newline or comma separated, or JSON array) | | `check_http` | boolean | `true` | Check HTTP/HTTPS status of each domain | | `useApifyProxy` | boolean | `true` | Enable Apify residential proxies | | `apifyProxyGroups` | array | `["RESIDENTIAL"]` | Proxy group configuration | **Example Inputs:** ```json { "domains": "example.com\ngoogle.com\nfacebook.com", "check_http": true, "useApifyProxy": true } ```` ```json { "domains": ["example.com", "google.com", "facebook.com"], "check_http": true } ``` ```json { "domains": "example.com, google.com, facebook.com", "check_http": true } ``` *** ### πŸ“€ Output The Actor pushes **Domain Expiry** records with the following structure: | Field | Type | Description | |-------|------|-------------| | `domain` | string | Normalized domain name | | `registrar` | string | Domain registrar name | | `creation_date` | string | Domain creation date (YYYY-MM-DD) | | `expiry_date` | string | **Domain Expiry** date (YYYY-MM-DD) | | `updated_date` | string | Last WHOIS update date (YYYY-MM-DD) | | `http_status` | string | HTTP status (active, error, unreachable, skipped) | | `whois_status` | string | WHOIS status code | | `days_left` | integer | Days until **Domain Expiry** | | `expiry_alert` | string | Alert level (critical, warning, ok, unknown) | | `whois_server` | string | WHOIS server address | | `name_servers` | array | List of name servers (max 6) | | `error` | string | Error message if WHOIS lookup failed | | `checked_at` | string | ISO 8601 timestamp of check | **Example Output Record:** ```json { "domain": "example.com", "registrar": "Example Registrar, Inc.", "creation_date": "1995-08-14", "expiry_date": "2025-08-13", "updated_date": "2024-07-01", "http_status": "active", "whois_status": "clienttransfereprohibited", "days_left": 487, "expiry_alert": "ok", "whois_server": "whois.iana.org", "name_servers": [ "a.iana-servers.net", "b.iana-servers.net" ], "error": "", "checked_at": "2025-02-14T12:00:00Z" } ``` **Expiring Soon Example:** ```json { "domain": "mycompany.com", "registrar": "GoDaddy Inc.", "creation_date": "2010-03-15", "expiry_date": "2025-03-14", "updated_date": "2024-12-01", "http_status": "active", "whois_status": "ok", "days_left": 28, "expiry_alert": "critical", "whois_server": "whois.godaddy.com", "name_servers": [ "ns1.godaddy.com", "ns2.godaddy.com" ], "error": "", "checked_at": "2025-02-14T12:00:00Z" } ``` *** ### 🧰 Technical Stack - **Async HTTP:** aiohttp for concurrent HTTP requests - **Date Handling:** datetime with timezone support - **Regex:** Pattern matching for domain normalization - **Threading:** asyncio.to\_thread for blocking WHOIS calls - **Proxy:** Apify Proxy with RESIDENTIAL configuration - **Logging:** Apify Actor logging system - **Platform:** Apify Actor serverless environment - **Timeout:** 10 seconds for HTTP, WHOIS blocking operations *** ### πŸ“Š Data Fields Explained #### **Domain Identity** - **domain**: Normalized domain (example.com format) #### **Registration Details** - **creation\_date**: Original registration date - **updated\_date**: Last WHOIS update - **registrar**: Domain registrar company #### **Expiry Information** - **expiry\_date**: When domain registration expires - **days\_left**: Days remaining until expiry - **expiry\_alert**: Urgency level (critical, warning, ok, unknown) #### **Technical Details** - **whois\_status**: Domain status from WHOIS - **whois\_server**: WHOIS server queried - **name\_servers**: DNS nameservers for domain - **http\_status**: Domain accessibility (active, error, unreachable, skipped) #### **Metadata** - **error**: WHOIS lookup error message (if any) - **checked\_at**: UTC timestamp of check *** ### 🎯 Use Cases - **Domain Portfolio Management** – Track **Domain Expiry** for all company domains - **Renewal Alerts** – Receive alerts for domains expiring soon - **Domain Audit** – Identify domains without proper registrar information - **DNS Monitoring** – Verify name server configuration - **Bulk Domain Check** – Check hundreds of domains simultaneously - **Website Availability** – Monitor HTTP status of domains - **Registration History** – Track domain registration timelines - **Registrar Analysis** – Analyze domain registrar distribution - **Expiry Compliance** – Ensure timely domain renewals - **Asset Inventory** – Maintain accurate domain asset lists - **Risk Management** – Identify domains at risk of expiration - **Multi-Registrar Management** – Track domains across multiple registrars - **Domain Monitoring Dashboard** – Create monitoring dashboards from results - **Automated Renewals** – Trigger renewal workflows for expiring domains - **Trademark Management** – Monitor related domain expirations *** ### πŸš€ Quick Start #### **1. Prepare Input** Go to Apify Console and enter: ```json { "domains": "example.com\ngoogle.com\nfacebook.com\namazon.com", "check_http": true, "useApifyProxy": true } ``` #### **2. Run the Actor** Click **Start** button. The Actor will: - Normalize and deduplicate domains - Perform WHOIS lookups - Check HTTP status (optional) - Calculate expiry information - Push results to Dataset # *** ### ⏰ Expiry Alerts The Domain Expiry Checker uses intelligent alert thresholds: | Alert Level | Days Remaining | Action | |------------|-----------------|--------| | **critical** | < 30 days | Immediate renewal required | | **warning** | 30-89 days | Schedule renewal soon | | **ok** | 90+ days | Normal monitoring | | **unknown** | No data | Unable to determine | ### πŸ“ˆ Performance #### **Processing Speed** - \~1 domain per second (including WHOIS + HTTP check) - Async HTTP checks for parallel processing - WHOIS blocking calls handled in threads - 1-second delay between WHOIS queries (polite to servers) #### **Batch Processing** - Handles 100+ domains efficiently - Memory efficient streaming to Dataset - Real-time result push #### **Resource Usage** - Memory: ~30-50MB - CPU: ~10-15% during processing - Network: ~50KB-100KB per domain #### **Data Quality** - **WHOIS Accuracy**: Data accuracy depends on registrar updates - **Date Formats**: Various registrars use different date formats - **Completeness**: Some domains have incomplete WHOIS data - **Timing**: Checks are point-in-time snapshots - **Verification**: Always verify critical dates with registrar #### **Best Practices** - Check domains regularly (monthly recommended) - Set calendar reminders for critical expirations - Maintain backup registrar contacts - Verify registrar contact information - Consider longer renewal periods (2+ years) - Monitor for suspicious status changes - Track registrar changes ### πŸ“¦ Changelog **Initial Release:** - WHOIS lookup integration for domain data extraction - Expiry date calculation with days remaining - Smart expiry alert system (critical <30, warning <90) - HTTP/HTTPS status checking - Registrar information extraction - Name server extraction (up to 6) - Bulk domain processing - Domain normalization and deduplication - Error handling and logging - Apify Dataset integration - ISO 8601 timestamp recording - Async/await architecture with threading - Proxy configuration support - Multi-format domain input support *** ### πŸ§‘β€πŸ’» Support & Feedback - **Issues:** Submit via Apify console - **Documentation:** Check Actor details page - **Community:** Apify forum discussions - **Feature Requests:** Suggest improvements - **Bug Reports:** Include logs and domain examples *** ### πŸ“„ License & Legal **Terms of Use:** - Use for legitimate domain management purposes - Respect WHOIS rate limits - Comply with registrar ToS - Respect GDPR and privacy regulations - Use data ethically and responsibly **Disclaimer:** Domain Expiry Checker is provided as-is for domain monitoring. Users are responsible for ensuring accuracy and compliance. Always verify critical dates with registrar. *** ### πŸŽ‰ Get Started Today **Deploy now for domain monitoring!** Use for: - πŸ“Š Domain Portfolio Management - ⏰ Expiry Tracking - πŸ”” Renewal Alerts - πŸ“‹ Domain Audit - 🌐 DNS Monitoring **Perfect for:** - Domain Managers - System Administrators - Web Developers - IT Teams - Business Owners *** **Last Updated:** February 2025\ **Version:** 1.0.0\ **Status:** Production Ready\ **Platform:** Apify Actor\ **Architecture:** Async/Await\ **Reliability:** Enterprise-grade *** ### πŸ“š Related Tools - Smart Article Extractor - Fast News Content Scraper - Business Social Media Finder - Website Monitoring Tools *** ### πŸ” Domain Management Excellence This Actor is optimized for **Domain Expiry** monitoring with: - βœ… Authoritative WHOIS lookups - βœ… Intelligent expiry alerts - βœ… Bulk domain processing - βœ… HTTP status verification - βœ… Real-time Dataset integration - βœ… Error recovery # Actor input Schema ## `domains` (type: `string`): Enter domains β€” one per line. URL format also accepted. Example: google.com https://github.com www.amazon.com ## `check_http` (type: `boolean`): Also check if the domain is reachable via HTTP/HTTPS. ## `useApifyProxy` (type: `boolean`): Route HTTP checks through Apify Proxy. ## `apifyProxyGroups` (type: `array`): Select proxy type. ## Actor input object example ```json { "domains": "google.com\ngithub.com", "check_http": true, "useApifyProxy": true, "apifyProxyGroups": [ "RESIDENTIAL" ] } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "domains": `google.com github.com` }; // Run the Actor and wait for it to finish const run = await client.actor("datapilot/domain-expiry-checker").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "domains": """google.com github.com""" } # Run the Actor and wait for it to finish run = client.actor("datapilot/domain-expiry-checker").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "domains": "google.com\\ngithub.com" }' | apify call datapilot/domain-expiry-checker --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=datapilot/domain-expiry-checker", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Domain Expiry Checker", "description": "Bulk Domain Expiry Checker scans domains using WHOIS data and optional HTTP checks. It extracts registrar, expiry date, status, name servers, and days left. Flags expiring domains (critical/warning) and outputs structured JSONβ€”ideal for domain monitoring, SEO audits, and portfolio management. πŸŒπŸ“…", "version": "0.0", "x-build-id": "L9QekgjXduT3lxCqs" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/datapilot~domain-expiry-checker/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-datapilot-domain-expiry-checker", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/datapilot~domain-expiry-checker/runs": { "post": { "operationId": "runs-sync-datapilot-domain-expiry-checker", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/datapilot~domain-expiry-checker/run-sync": { "post": { "operationId": "run-sync-datapilot-domain-expiry-checker", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "domains" ], "properties": { "domains": { "title": "Domains", "type": "string", "description": "Enter domains β€” one per line. URL format also accepted.\nExample:\ngoogle.com\nhttps://github.com\nwww.amazon.com" }, "check_http": { "title": "Check HTTP Status", "type": "boolean", "description": "Also check if the domain is reachable via HTTP/HTTPS.", "default": true }, "useApifyProxy": { "title": "Use Apify Proxy", "type": "boolean", "description": "Route HTTP checks through Apify Proxy.", "default": true }, "apifyProxyGroups": { "title": "Proxy Groups", "uniqueItems": true, "type": "array", "description": "Select proxy type.", "items": { "type": "string", "enum": [ "RESIDENTIAL", "DATACENTER", "GOOGLE" ] }, "default": [ "RESIDENTIAL" ] } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```