BGP.HE.NET Recon Scraper

What does BGP.HE.NET Recon Scraper do?

BGP.HE.NET Recon Scraper extracts BGP and network recon data from bgp.he.net — Hurricane Electric's free BGP Toolkit — for any mix of ASNs, IP prefixes, IP addresses, or domain names in a single run. It auto-detects the target type per input string and returns whois/RDAP records, BGP peer and prefix tables, live routing visibility, reverse-DNS/PTR data, and DNS records, all as clean structured JSON. Just enter your targets and let the Actor handle proxy rotation, retries, and pagination on Apify's infrastructure — no need to write your own scraper or babysit browser sessions.

Why use BGP.HE.NET Recon Scraper?

• Network reconnaissance — map an organization's ASN, announced/originated prefixes, upstream and peer ASNs, and RPKI/IRR validation status in one pass.
• Threat intel & attack surface mapping — pull reverse-DNS hostnames sharing an IP, prefix delegations, and BGP routing visibility for security research and OSINT.
• Infrastructure audits — verify which AS announces your prefixes, whether routes are RPKI-valid, and how widely a prefix is visible across BGP collectors.
• Bulk lookups on autopilot — schedule recurring runs, feed the dataset into other tools via the Apify API, or trigger it from Zapier/Make integrations.

How to use BGP.HE.NET Recon Scraper

1. Open the Actor's Input tab.
2. Add one or more Targets — any mix of ASNs (AS15169 or 15169), IPv4/IPv6 prefixes (8.8.8.0/24), IP addresses (8.8.8.8), or domain names (google.com).
3. Optionally toggle whois/RDAP lookups, BGP routing visibility, and adjust the record caps.
4. Click Start and download the results from the Dataset tab in JSON, CSV, Excel, or HTML.

Input

Field	Type	Description
targets	array of strings	ASN, prefix, IP, or domain. Type is auto-detected per entry.
includeWhois	boolean	Fetch raw whois text and RDAP JSON (default true).
includeRouting	boolean	For prefix targets, fetch live BGP visibility and observed AS-paths (default true).
maxRoutingRecords	integer	Cap on visibility/AS-path entries kept per prefix (default 300).
maxHostnamesPerIp	integer	Cap on reverse-DNS hostnames kept per IP — bgp.he.net often lists hundreds of unrelated domains sharing an IP (default 50).
proxyConfiguration	object	Optional Apify Proxy configuration.

See the Input tab for the full schema.

Output

Each target produces one dataset item shaped by its detected type. Example for an ASN target:

{
  "targetType": "asn",
  "asn": "15169",
  "orgName": "Google LLC",
  "stats": {
    "prefixes_originated_all": 1417,
    "bgp_peers_observed_all": 701,
    "average_as_path_length_all": 3.652
  },
  "links": {
    "Company Website": { "text": "https://about.google/intl/en/", "href": "https://about.google/intl/en/" },
    "Country of Origin": { "text": "United States", "href": "/country/US" }
  },
  "peersV4": [{ "rank": "1", "name": "TATA COMMUNICATIONS (AMERICA) INC", "asn": "AS6453" }],
  "prefixesV4": [{ "prefix": "8.8.4.0/24", "org": "Google LLC", "irrValid": true, "rpkiValid": true }],
  "inputTarget": "AS15169"
}

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

Data table

Target type	Key fields
ASN	orgName, stats (prefix/peer/RPKI counters), links, peersV4/peersV6, prefixesV4/prefixesV6, whois, rdap
Prefix	announcedBy, matchingDelegations, ptrRecords, rirInfo, bgpVisibility, bgpAsPaths, whois, rdap
IP	announcedBy, ptr, hostnames, whois
Domain	dnsRecords (NS/A/AAAA/MX/TXT/SOA), prefixMappings

Pricing / Cost estimation

This Actor makes lightweight HTTP requests (no browser) — each target typically costs a handful of requests. It runs comfortably on the free Apify plan for moderate target lists; larger batches (hundreds of ASNs with routing data enabled) will use proportionally more compute units.

Tips or Advanced options

• Turn off includeRouting if you only need static prefix/whois data — it skips the live BGP visibility and AS-path lookups, which are the slowest calls.
• Lower maxHostnamesPerIp for busy shared-hosting IPs to keep dataset items small; bgp.he.net caps this list at 1000 entries per IP regardless.
• Mix target types freely in one targets list — an ASN, its prefixes, and specific IPs can all be scraped in the same run.

FAQ, disclaimers, and support

This Actor only reads publicly available pages on bgp.he.net and does not require authentication. Some tabs on bgp.he.net (whois text, RADB/IRR lookups, cert transparency search, traceroute) are loaded client-side via separate AJAX calls; this Actor reproduces the ones relevant to recon (whois, RDAP, BGP visibility, AS-paths) but does not cover cert search, RBL, or traceroute tooling. Please respect bgp.he.net's terms of use and avoid excessive request volumes. Found a bug or need a custom variant? Open an issue on the Actor's Issues tab.

Cyber recon toolkit

Part of the Cyber recon toolkit — OSINT, attack-surface recon, and vulnerability-intel data collection for security research:

• Bugcrowd Scraper — Scrapes public Bugcrowd data: disclosed submissions, program directory, leaderboard.
• HackerOne Scraper — Scrapes public HackerOne data: Hacktivity disclosed reports, program directory, researcher leaderboard.
• Intigriti Scraper — Scrapes public Intigriti data: bug bounty/VDP program directory, researcher leaderboard and profiles.
• CVE Scraper — CVE.org vulnerability info scraper.
• TLSX - TLS Data Gathering and Analysis — Fast, configurable TLS grabber for TLS-based data collection and analysis.
• gau - Get All URLs — Fetch known URLs from Wayback Machine, Common Crawl, AlienVault OTX, and URLScan.
• Git Email Extractor — Shallow-clones git repos and extracts contributor emails from commit history.
• Wappalyzer Technology Lookup — Detect the technology stack behind any website using Wappalyzer.
• Meta Tags Scraper — Web page metadata scraper (meta tags, Open Graph, Twitter Card).
• Universal Contact Extractor — Extract phone numbers, emails and social media URLs from web pages.
• Web Text Extractor — Extract clean text or Markdown content from web pages.

Did you find this useful?

⭐ Rate this actor on Apify! Your feedback helps other users find it and helps us keep improving it.