Domain Intelligence Scraper

Extract comprehensive domain registration data using free public RDAP (Registration Data Access Protocol) APIs. Get domain age, expiration dates, registrar info, nameservers, and DNSSEC status for any domain.

Features

• Domain Age Analysis - Calculate exact domain age in days for SEO and trust scoring
• Expiration Tracking - Monitor domain expiration dates for acquisition opportunities
• Registrar Information - Identify registrars and registration details
• Nameserver Data - Extract DNS configuration and hosting providers
• DNSSEC Status - Check security configuration of domains
• Multi-TLD Support - Works with 30+ TLDs including .com, .net, .org, .io, .ai, .dev, .app, and more
• URL Parsing - Automatically extracts domains from full URLs

Use Cases

• SEO Analysis - Domain age is a ranking factor; analyze competitor domain ages
• Lead Generation - Find newly registered domains in your niche
• Security Research - Identify suspicious domains, check DNSSEC status
• Domain Investing - Track expiring domains for acquisition
• Competitor Intelligence - Analyze when competitors registered their domains
• Due Diligence - Verify domain ownership and registration history

Input

Parameter	Type	Description
domains	array	List of domains to analyze (e.g., google.com, stripe.com)
urls	array	List of URLs to extract domains from (alternative to domains)
delaySeconds	number	Delay between requests (default: 0.3, min: 0.1, max: 5)

Example Input

{
  "domains": ["google.com", "stripe.com", "github.com"]
}

Or with URLs:

{
  "urls": ["https://www.example.com/page", "https://shop.mystore.com/products"]
}

Output

Each domain returns a comprehensive data object:

{
  "domain": "google.com",
  "tld": "com",
  "registrar": "MarkMonitor Inc.",
  "registration_date": "1997-09-15T04:00:00Z",
  "expiration_date": "2028-09-14T04:00:00Z",
  "domain_age_days": 9937,
  "status": ["clientDeleteProhibited", "clientTransferProhibited", "clientUpdateProhibited", "serverDeleteProhibited", "serverTransferProhibited", "serverUpdateProhibited"],
  "nameservers": ["NS1.GOOGLE.COM", "NS2.GOOGLE.COM", "NS3.GOOGLE.COM", "NS4.GOOGLE.COM"],
  "dnssec": false,
  "rdap_url": "https://rdap.verisign.com/com/v1/domain/google.com",
  "scraped_at": "2024-01-15T10:30:00.000000",
  "scrape_success": true
}

Supported TLDs

The scraper has optimized RDAP endpoints for these TLDs:

• Generic: .com, .net, .org, .info, .biz, .xyz, .online, .tech, .site, .shop, .club
• Tech: .io, .ai, .dev, .app, .co
• Country: .us, .uk, .de, .fr, .nl, .eu, .ca, .au, .in, .jp, .kr, .cn, .ru, .br, .mx
• Media: .tv, .cc, .me

For unlisted TLDs, the scraper automatically uses the RDAP bootstrap service.

How It Works

This scraper uses RDAP (Registration Data Access Protocol), the modern successor to WHOIS. RDAP provides structured JSON data through free public APIs maintained by domain registries. Unlike traditional WHOIS:

• Returns machine-readable JSON data
• No parsing required
• More reliable and standardized
• Free and public - no API keys needed!

Pricing

Pay per result: $0.001 per domain (that's $1 per 1,000 domains)

Rate Limits

The scraper includes built-in rate limiting to respect RDAP server policies. Default delay is 0.3 seconds between requests. If you receive rate limit errors, increase the delaySeconds parameter.

Legal Notice

This scraper accesses publicly available RDAP data, which is provided by domain registries for legitimate lookup purposes. Please use responsibly and in accordance with applicable terms of service.