urlscan.io Threat Intelligence Scraper

urlscan.io Threat Intelligence Scraper

Scrape urlscan.io scan results for any domain, IP, ASN, or hash, up to a million per run. Every scan comes with its final URL, page title, IP, ASN, country, server, and threat tags. No API key. Export to CSV, JSON, Excel, or XML.

urlscan.io's own search API is rate-limited and its UI makes you click through scans one by one. This reads the public search index directly, filtered by domain, IP, ASN, hash, filename, or threat tag, and returns each matching scan in one fixed schema. It is the fastest way to turn urlscan.io into a dataset for threat hunting, brand protection, or attack surface mapping.

Who uses it	What they scrape urlscan.io for
SOC analysts	Pulling every scan tagged phishing for a brand's domains into a SIEM
Threat hunters	Finding infrastructure that shares an IP, ASN, or JS hash with a known bad actor
Brand protection teams	Monitoring newly registered lookalike domains and phishing kits
OSINT researchers	Building a timeline of scans for a domain or IP across time
Security engineers	Feeding urlscan.io results into an automated triage or enrichment pipeline

What it does

This Actor collects urlscan.io scan results matching a Lucene query and returns each scan as a flat row with its URL, page title, IP, ASN, country, server, and threat tags.

• 🔍 Lucene query input: use the full urlscan.io search syntax, including AND, OR, NOT, wildcards, and date ranges.
• 📄 Flat scan rows: each result is one row with the final URL, page title, IP, ASN, country, server, and threat tags.
• 🏷️ Threat tags included: phishing, malware, and other urlscan.io tags come through on every scan.
• 🌍 Network context: IP, ASN, and country are returned for every scan so you can pivot on infrastructure.
• 📦 Bulk export: collect up to 1,000,000 scans per run and export to CSV, JSON, Excel, or XML.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with urlscan.io data

🎣 Monitor phishing campaigns.

A SOC analyst runs a query like domain:yourbrand.com AND task.tags:phishing every hour and feeds new scans into a SIEM alert.

🕵️ Hunt shared infrastructure.

A threat hunter takes a malicious IP or ASN and pulls every scan on that infrastructure to find other malicious domains.

🛡️ Protect your brand.

A brand protection team searches for newly registered lookalike domains and phishing kits targeting their trademarks.

📊 Map an attack surface.

An OSINT researcher collects all scans for a domain over the last 30 days to build a timeline of changes and spot suspicious activity.

🔗 Enrich alerts automatically.

A security engineer pipes urlscan.io results into an enrichment pipeline to add scan context to internal alerts.

Why choose this scraper

	What you get
No API key	urlscan.io's public search index is read directly, so you do not need to register an API key or manage rate limits.
Full query syntax	Use the same Lucene syntax as the urlscan.io search page, including AND, OR, NOT, wildcards, and date ranges.
Threat tags	Every scan includes its urlscan.io tags, such as phishing or malware, so you can filter and alert on them.
Network pivots	IP, ASN, and country are returned for every scan, making it easy to find related infrastructure.
Bulk collection	Collect up to 1,000,000 scans per run, far beyond what the urlscan.io UI allows.

How it compares

This Actor focuses on urlscan.io scan results with full metadata, while gau - Get All URLs aggregates URLs from multiple sources including urlscan.io but does not return scan metadata.

Feature	ParseForge	gau - Get All URLs
urlscan.io scan metadata (IP, ASN, country, server)	Yes	Not listed
Threat tags (phishing, malware)	Yes	Not listed
Full Lucene query syntax	Yes	Not listed
Bulk export up to 1,000,000 scans	Yes	Not listed
Multiple source aggregation (Wayback, Common Crawl, OTX)	Not listed	Yes

Configure the run

Drive the Actor with a single urlscan.io Lucene query, and set the maximum number of scans and the page size to stay within rate limits. The Input tab lists every parameter.

A first run with the defaults:

{
  "query": "domain:apify.com",
  "maxItems": 10,
  "pageSize": 100
}

A larger pull:

{
  "query": "domain:apify.com",
  "maxItems": 200,
  "pageSize": 100
}

Pricing

Pay-per-result: $0.0355 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$3.55
1,000 results	$35.50
10,000 results	$355.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the urlscan.io Threat Intelligence Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to urlscan.io through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/urlscan-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check your query syntax. urlscan.io uses Lucene syntax, so make sure you are using the correct field names and operators. Try a broader query like domain:example.com first.

Why is the run slow?

Lower the page size. The default is 100, but you can set it as low as 1 to be friendlier to rate limits, especially on the free tier.

Why did the run stop before reaching my maximum scans?

urlscan.io may rate-limit requests. Try lowering the page size or increasing the delay between requests. The Actor will return the scans it collected before hitting the limit.

Can I search for multiple domains at once?

Yes. Use OR in your query, for example domain:example.com OR domain:example.org.

Why are some fields empty?

Not every scan has all fields populated. For example, a scan may not have a page title or server header. Empty fields are returned as null.

FAQ

Question	Answer
Do I need a urlscan.io API key?	No. This Actor reads the public search index directly, so you do not need to register for an API key or manage rate limits.
What query syntax can I use?	You can use the full urlscan.io Lucene search syntax, including AND, OR, NOT, wildcards, and date ranges. See the urlscan.io search documentation for details.
How many scans can I collect?	You can collect up to 1,000,000 scans per run. Set the maximum scans input to control how many results are returned.
What data does each scan include?	Each scan row includes the final URL, page title, IP address, ASN, country, server, and threat tags such as phishing or malware.
Can I search by IP or ASN?	Yes. Use queries like page.ip:8.8.8.8 or page.asn:AS15169 to find scans on a specific IP or ASN.
Can I search by file hash?	Yes. Use hash:sha256... to find scans that contain a known JavaScript or file hash.
Can I search by filename?	Yes. Use filename:wp-login.php to find scans that reference a specific filename.
Can I filter by country?	Yes. Use page.country:RU to find scans hosted in a specific country, and combine with date ranges like date:>now-7d.
What export formats are supported?	You can export the results to CSV, JSON, Excel, or XML.
Is this legal?	This Actor only reads publicly available scan results from urlscan.io. You are responsible for complying with urlscan.io's terms of service and applicable laws.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by urlscan.io GmbH. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.