NVD CVE Scraper

An Apify actor that fetches recent CVE (Common Vulnerabilities and Exposures) data from the National Vulnerability Database (NVD) using their official API.

Features

• Fetches CVEs published in the last 7 days
• Uses the official NVD API 2.0 for reliable data access
• Extracts CVE ID, description, publication date, and metadata
• Outputs structured data to Apify dataset

Why Use the API Instead of Scraping?

The NVD website loads CVE data dynamically via JavaScript, making traditional HTML scraping unreliable. The official API provides:

• Structured, reliable data
• Better performance
• No risk of breaking when the website UI changes
• Official support from NIST

Input Configuration

You can customize the actor by modifying these parameters in the code:

// Adjust the date range (currently set to last 7 days)
const sevenDaysAgo = new Date(now.getTime() - 7 * 24 * 60 * 60 * 1000);

// Adjust results per page (max 2000)
resultsPerPage: '100'

Output

Each scraped CVE includes:

• cve: CVE identifier (e.g., CVE-2024-12345)
• description: English description of the vulnerability
• published: Publication date
• lastModified: Last modification date
• sourceIdentifier: Source that reported the CVE

Example Output

{
  "cve": "CVE-2024-12345",
  "description": "A buffer overflow vulnerability in XYZ software allows remote attackers to execute arbitrary code...",
  "published": "2024-11-15T10:15:00.000",
  "lastModified": "2024-11-15T10:15:00.000",
  "sourceIdentifier": "security@example.com"
}

Rate Limits

The NVD API has rate limits:

• Without API key: ~5 requests per 30 seconds
• With API key: 50 requests per 30 seconds

For this actor (single request), rate limits shouldn't be an issue.

Getting an API Key (Optional)

For higher rate limits or frequent runs:

1. Request a free API key at: https://nvd.nist.gov/developers/request-an-api-key
2. Add it to your code:

const response = await fetch(apiUrl, {
    headers: {
        'User-Agent': 'Apify-CVE-Scraper/1.0',
        'apiKey': 'YOUR_API_KEY_HERE'
    }
});

Installation

1. Create a new Apify actor
2. Copy the code into your actor's main file
3. Deploy and run

Usage Examples

Run as Scheduled Actor

Set up a schedule to run daily and monitor new CVEs:

1. Go to your actor in Apify Console
2. Click "Schedule"
3. Set to run daily at your preferred time

Export Data

The scraped CVEs are stored in the actor's dataset and can be:

• Downloaded as JSON, CSV, or Excel
• Accessed via Apify API
• Pushed to external services using integrations

Advanced Customization

Filter by Severity

Add CVSS score filtering to the API request:

const params = new URLSearchParams({
    pubStartDate: sevenDaysAgo.toISOString(),
    pubEndDate: now.toISOString(),
    cvssV3Severity: 'CRITICAL', // or HIGH, MEDIUM, LOW
    resultsPerPage: '100'
});

Search by Keyword

Filter CVEs containing specific keywords:

const params = new URLSearchParams({
    keywordSearch: 'buffer overflow',
    resultsPerPage: '100'
});

API Documentation

Full NVD API documentation: https://nvd.nist.gov/developers/vulnerabilities

Troubleshooting

No CVEs returned: Check that there were CVEs published in your date range. Try expanding the date range.

API errors: Ensure you're not hitting rate limits. Add delays between requests if making multiple API calls.

Empty descriptions: Some CVEs may not have English descriptions immediately upon publication.

License

This actor uses public data from the National Vulnerability Database. Please review NVD's terms of use.

Support

For issues or questions:

• Check NVD API status: https://nvd.nist.gov/general/news
• Review API documentation
• Contact Apify support for actor-specific issues