# CDC NNDSS Public Health Surveillance Harvester (`datapilot/cdc-nndss-public-health-surveillance-harvester`) Actor

state, and year. Extract disease indicators, locations, weekly reported case counts, dates, and source links. Supports disease-name discovery mode and exports results to Apify Dataset in JSON format.

- **URL**: https://apify.com/datapilot/cdc-nndss-public-health-surveillance-harvester.md
- **Developed by:** [Data Pilot](https://apify.com/datapilot) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 scraped results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

🩺 **CDC NNDSS Public Health Surveillance Harvester** is a powerful Apify Actor designed to collect disease surveillance records from the CDC's National Notifiable Diseases Surveillance System (NNDSS) open data. This tool provides **Public Health Surveillance Harvester** intelligence including disease indicator, reporting area, MMWR week-ending date, reported value, and a source link for every record. Whether you're tracking a disease over time, building a public health dashboard, or researching outbreak trends, the Public Health Surveillance Harvester delivers structured results efficiently.

With keyword search, reporting-area and year filters, a disease-name listing mode, automatic retries, and reliable Apify Dataset delivery, this **Public Health Surveillance Harvester** actor turns the CDC's open data into clean, ready-to-use records. It focuses on key signals including indicator name, location, week date, and reported value, making **Public Health Surveillance Harvester** a practical tool for epidemiology research and public health monitoring.

***

### 📋 Table of Contents

- [Features](#-features)
- [Data Source](#-data-source)
- [How It Works](#-how-it-works)
- [Input](#-input)
- [Output](#-output)
- [Technical Stack](#-technical-stack)
- [Data Fields](#-data-fields-explained)
- [Use Cases](#-use-cases)
- [Quick Start](#-quick-start)
- [Configuration](#-configuration)
- [Performance](#-performance)
- [Important Notes](#-important-notes)
- [License & Legal](#-license--legal)

***

### 🔥 Features

- **CDC Surveillance Data** – Pulls records from the CDC's official open data portal (data.cdc.gov). This is the core of **Public Health Surveillance Harvester**.
- **Keyword Search** – Case-insensitive search across disease and indicator names (default: `measles`).
- **Reporting Area Filter** – Narrow results to a state or reporting area with the `state` input.
- **Year Filter** – Limit results to a single year with the `year` input.
- **Disease Name Listing Mode** – Set `listOnly` to get a list of the disease names that match your keyword, so you can pick the right search term.
- **MMWR Week-Ending Dates** – Converts each record's MMWR year and week into the Saturday that ends that week (`YYYY-MM-DD`).
- **Newest First** – Records are ordered by year and week, most recent first.
- **Empty Weeks Skipped** – Records without a reported numeric value are excluded.
- **Automatic Retries** – Retries failed requests up to 3 times with increasing wait between attempts.
- **Apify Proxy Support** – Optional Apify Proxy for reliable **Public Health Surveillance Harvester** access.
- **Proxy Fallback Handling** – Retries without a proxy if a proxied request fails with an error.
- **Single-Batch Dataset Push** – Pushes all records to the Apify Dataset together.
- **Detailed Logging** – Logs proxy status, search settings, and results for every run.

***

### 📊 Data Source

#### **CDC Open Data — NNDSS (data.cdc.gov)**

- **Authority**: US Centers for Disease Control and Prevention (CDC), National Notifiable Diseases Surveillance System
- **Endpoint**: `https://data.cdc.gov/resource/x9gk-5huc.json` (Socrata Open Data API, queried with SoQL)
- **Access Method**: Browser-impersonated HTTP requests via `curl_cffi` (Chrome 124 fingerprint), parsing the JSON response
- **Coverage**: Disease indicators and reporting areas available in this CDC dataset
- **Data**: Indicator label, reporting area, year, MMWR week, and reported value
- **Access**: Public API, no API key required
- **Update Frequency**: Reflects the dataset's contents at time of run

***

### ⚙️ How It Works

The Public Health Surveillance Harvester accepts a keyword and optional area and year filters. It builds a SoQL query, requests matching records from the CDC API, converts each record's MMWR year and week into a calendar date, and pushes the cleaned records to the Apify Dataset as one batch. In `listOnly` mode it instead returns the distinct disease names that match your keyword.

**Key Processing Steps:**

1. **Input Parsing** – Read `keyword`, `state`, `year`, `maxResults`, and `listOnly`
2. **Proxy Setup** – Configure Apify Proxy if enabled, with automatic direct-connection fallback
3. **Query Building** – Build the filter: records with a reported value, keyword match, optional area match, optional year
4. **API Request** – Fetch records ordered by year and week (newest first), up to `maxResults`
5. **Date Conversion** – Convert MMWR year and week into the week-ending (Saturday) date
6. **Normalization** – Keep records with a numeric value and build one clean record each
7. **Dataset Push** – Push all **Public Health Surveillance Harvester** records at once
8. **Summary Logging** – Report how many records were saved

**Key Benefits:**

- Search CDC surveillance data without writing SoQL queries
- Get consistent records with real calendar dates instead of raw MMWR week numbers
- Find the exact disease names available with `listOnly` before running a full search
- Automate recurring **Public Health Surveillance Harvester** runs to track new weekly data

***

### 📥 Input

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `keyword` | string | `"measles"` | Text to match (case-insensitive, partial match) against the indicator/disease name. |
| `state` | string | *(none)* | Optional reporting area filter (case-insensitive, partial match). |
| `year` | integer | *(none)* | Optional year filter. |
| `maxResults` | integer | `100` | Maximum records to fetch (capped at 50,000). |
| `listOnly` | boolean | `false` | If `true`, returns only the matching disease names instead of data records. |
| `proxyConfiguration` | object | `{"useApifyProxy": true}` | Apify Proxy configuration. |

**Example Input:**

```json
{
  "keyword": "measles",
  "year": 2025,
  "maxResults": 100
}
```

**Filter by Reporting Area:**

```json
{
  "keyword": "pertussis",
  "state": "California",
  "maxResults": 200
}
```

**List Matching Disease Names:**

```json
{
  "keyword": "hepatitis",
  "listOnly": true
}
```

***

### 📤 Output

#### **Data records (default mode)**

| Field | Type | Description |
|-------|------|-------------|
| `indicator` | string | Disease or indicator name from the CDC dataset |
| `location` | string | Reporting area |
| `date` | string | MMWR week-ending date (`YYYY-MM-DD`), or `YYYY-W<week>` if the date cannot be calculated |
| `value` | number | Reported value for that indicator, area, and week |
| `link` | string | API link that returns the source record from the CDC dataset |

**Example Public Health Surveillance Harvester Record:**

```json
{
  "indicator": "Measles, Total",
  "location": "TEXAS",
  "date": "2025-03-29",
  "value": 12,
  "link": "https://data.cdc.gov/resource/x9gk-5huc.json?label=Measles%2C+Total&states=TEXAS&year=2025&week=13"
}
```

#### **Disease name records (`listOnly` mode)**

| Field | Type | Description |
|-------|------|-------------|
| `disease_name` | string | A distinct indicator name matching your keyword (up to 500) |

***

### 🧰 Technical Stack

- **HTTP Client:** `curl_cffi` with Chrome 124 TLS fingerprint impersonation
- **Source:** CDC Socrata Open Data API (SoQL queries)
- **Date Handling:** MMWR week calculation (Sunday–Saturday weeks, week 1 is the first week with at least 4 days in the year), with timezone-aware UTC
- **Async:** `asyncio` with executor-run blocking requests
- **Proxy:** Apify Proxy, optional via `useApifyProxy`, with automatic no-proxy fallback on failure
- **Logging:** Apify Actor logging system
- **Platform:** Apify Actor serverless environment

***

### 📊 Data Fields Explained

#### **What**

- **indicator**: The disease or condition the value refers to

#### **Where**

- **location**: The reporting area for the value

#### **When**

- **date**: The Saturday that ends the MMWR week of the record

#### **How Much**

- **value**: The reported numeric value from the dataset

#### **Source**

- **link**: API link to the source record in the CDC dataset

***

### 🎯 Use Cases

- **Disease Trend Tracking** – Follow a disease over weeks and years
- **Public Health Dashboards** – Feed charts and dashboards with clean time-series data
- **Epidemiology Research** – Collect surveillance data for analysis
- **Regional Comparison** – Compare reporting areas for the same indicator
- **Journalism & Reporting** – Gather public figures for data-driven stories
- **Alerting & Monitoring** – Run on a schedule to track newly published weeks

***

### 🚀 Quick Start

#### **1. Prepare Input**

Go to Apify Console and enter:

```json
{
  "keyword": "measles",
  "year": 2025,
  "maxResults": 100
}
```

#### **2. Run the Actor**

Click **Start**. The Actor will:

- Connect through Apify Proxy if enabled
- Query the CDC API with your filters
- Convert MMWR weeks into dates
- Push all results to the Dataset as one batch

#### **3. Monitor Progress**

Console shows:

```
🛡️ Apify Proxy connected.
🔎 Fetching CDC NNDSS data for 'measles' (year: 2025) (up to 100)...
✅ Successfully harvested 100 CDC surveillance records to dataset!
🎉 Complete!
```

#### **4. View & Download Results**

- **Results Tab**: All **Public Health Surveillance Harvester** records
- **Export**: JSON, CSV, Excel, or HTML
- **API Access**: Available via the Apify API

***

### ⚙️ Configuration

Default run:

```json
{
  "keyword": "measles",
  "maxResults": 100
}
```

Find the right disease name first:

```json
{
  "keyword": "influenza",
  "listOnly": true
}
```

Direct connection without proxy:

```json
{
  "keyword": "mumps",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

***

### 📈 Performance

#### **Processing Speed**

- A single API request per run (plus retries if needed)
- Up to 50,000 records can be requested in one call

#### **Resource Usage**

- Memory: Low, since no browser instance is launched
- Network: One lightweight request per run
- Proxy: Optional Apify Proxy with automatic no-proxy fallback on failure

***

### ⚠️ Important Notes

#### **Legal & Compliance**

- **Public Data Only**: Retrieves only publicly available CDC open data
- **Fair Use**: Uses a single request per run to keep load on the source minimal
- **Not Medical Advice**: This data is for research and information purposes, not for diagnosis or treatment decisions
- **Legal**: Not legal advice

#### **Data Quality**

- **Source Figures**: Values are passed through exactly as published in the CDC dataset. Surveillance counts can be provisional and may be revised by the CDC, so confirm figures with the CDC before official or published use.
- **Keyword Matching**: Matching is a partial, case-insensitive text match, so broad keywords can return several related indicators. Use `listOnly` to see the exact names.
- **Reporting Area Matching**: The `state` filter is also a partial match against the dataset's reporting area names.
- **Skipped Records**: Records without a numeric value are left out.
- **Date Conversion**: `date` is calculated from the MMWR year and week. If that fails, the value is shown as `YYYY-W<week>`.
- **Link Field**: `link` is a JSON API link to the matching record, not a human-readable web page.
- **Empty Results**: If nothing matches, the dataset is empty and a warning is logged. In `listOnly` mode, nothing is pushed when no names match.
- **Failures**: If the CDC API cannot be reached after 3 attempts, the run ends with an error.

#### **Best Practices**

- Run with `listOnly` first to find the exact disease name you want
- Combine `keyword`, `state`, and `year` to keep results focused
- Keep `maxResults` close to what you need
- Run on a schedule to pick up newly published weeks
- Verify important figures on the CDC website before publishing

***

### 📄 License & Legal

**Terms of Use:**

- Use for legitimate research, reporting, and public health analysis
- Respect the terms of use of the CDC open data portal
- Use data responsibly and in compliance with applicable laws

**Disclaimer:**
Public Health Surveillance Harvester is provided as-is for research and informational purposes. It is not affiliated with or endorsed by the CDC. It does not provide medical advice. Users are responsible for compliance with all applicable laws and the data source's terms. This is not legal advice.

***

### ⚖️ Public Health Surveillance Harvester Excellence

This Actor is optimized for **Public Health Surveillance Harvester** research with:

- ✅ Official CDC open data
- ✅ Keyword, reporting-area, and year filters
- ✅ Disease name listing mode
- ✅ MMWR week-to-date conversion
- ✅ Automatic retries and proxy fallback
- ✅ Production-ready code

# Actor input Schema

## `keyword` (type: `string`):

Disease or condition name (partial matching works), e.g., 'measles', 'pertussis', 'hepatitis'.

## `state` (type: `string`):

Optional reporting area (e.g., 'California', 'Florida', or 'U.S. Residents'). Leave blank for all.

## `year` (type: `integer`):

Filter by MMWR Year (e.g., 2024, 2025). Leave blank for all years (newest first).

## `maxResults` (type: `integer`):

Maximum number of disease records to fetch.

## `listOnly` (type: `boolean`):

Set to true to only output available matching disease labels/indicators without harvesting records.

## `proxyConfiguration` (type: `object`):

Apify Proxy configuration.

## Actor input object example

```json
{
  "keyword": "measles",
  "maxResults": 100,
  "listOnly": false,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `crawlResults` (type: `string`):

No description

## `surveillanceOverview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "keyword": "measles",
    "maxResults": 100
};

// Run the Actor and wait for it to finish
const run = await client.actor("datapilot/cdc-nndss-public-health-surveillance-harvester").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "keyword": "measles",
    "maxResults": 100,
}

# Run the Actor and wait for it to finish
run = client.actor("datapilot/cdc-nndss-public-health-surveillance-harvester").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "keyword": "measles",
  "maxResults": 100
}' |
apify call datapilot/cdc-nndss-public-health-surveillance-harvester --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datapilot/cdc-nndss-public-health-surveillance-harvester"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Pt8g4cre82TymsjhD/builds/DLWfoCr8JYlZYbBA4/openapi.json