# CFPB Consumer Complaint Alert & Harvester (`datapilot/cfpb-consumer-complaint-alert-harvester`) Actor

Extract complaint IDs, company names, products, issues, response statuses, dates, and complaint links. Track previously seen complaints and identify new alerts using Apify Key-Value Store. Export results to Apify Dataset in JSON format.

- **URL**: https://apify.com/datapilot/cfpb-consumer-complaint-alert-harvester.md
- **Developed by:** [Data Pilot](https://apify.com/datapilot) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 scraped results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

🚨 **CFPB Consumer Complaint Alert & Harvester** is a powerful Apify Actor designed to collect consumer financial complaints from the US Consumer Financial Protection Bureau (CFPB) and flag the ones that are new since your last run. This tool provides **Complaint Alert** intelligence including company, product, issue, company response, complaint date, and a direct link to each complaint. Whether you're monitoring a company's reputation, tracking financial product issues, or building a compliance watchlist, the Complaint Alert Harvester delivers actionable results efficiently.

With keyword search, a configurable date window, automatic pagination, new-complaint detection, and reliable Apify Dataset delivery, this **Complaint Alert** actor remembers which complaints it has already seen and marks the rest as new. It focuses on key signals including company, product, issue, and complaint date, making **Complaint Alert** monitoring fast and structured.

***

### 📋 Table of Contents

- [Features](#-features)
- [Data Source](#-data-source)
- [How It Works](#-how-it-works)
- [Input](#-input)
- [Output](#-output)
- [Technical Stack](#-technical-stack)
- [Data Fields](#-data-fields-explained)
- [Use Cases](#-use-cases)
- [Quick Start](#-quick-start)
- [Configuration](#-configuration)
- [Performance](#-performance)
- [Important Notes](#-important-notes)
- [License & Legal](#-license--legal)

***

### 🔥 Features

- **CFPB Complaint Search** – Pulls complaints from the CFPB's official Consumer Complaint Database. This is the core of **Complaint Alert** monitoring.
- **Keyword Search** – Search by company name or any term (default: `Equifax`), or leave it empty to search all complaints.
- **Configurable Date Window** – Look back any number of days with `daysBack`.
- **New Complaint Detection** – Compares complaint IDs against those saved from previous runs and sets an `is_new` flag on each record.
- **Alerts-Only Mode** – Turn on `onlyNewAlerts` to receive only complaints not seen before.
- **Newest First** – Complaints are fetched in order of creation date, most recent first.
- **Automatic Pagination** – Fetches complaints 100 at a time, up to 1,000 per run.
- **Direct Complaint Links** – Each record includes a link to the complaint's detail page.
- **Narrative Flag** – Shows whether a complaint includes a consumer narrative (`has_narrative`).
- **Retry Handling** – Retries failed requests up to 3 times with increasing wait between attempts.
- **Apify Proxy Support** – Optional Apify Proxy, with a direct-connection retry if the proxy receives HTTP 403.
- **Single-Batch Dataset Push** – Pushes all records to the Apify Dataset together.
- **Detailed Logging** – Logs baseline creation, new-complaint alerts, and completion for every **Complaint Alert** run.

***

### 📊 Data Source

#### **CFPB Consumer Complaint Database**

- **Authority**: US Consumer Financial Protection Bureau (consumerfinance.gov)
- **Endpoint**: `https://www.consumerfinance.gov/data-research/consumer-complaints/search/api/v1/`
- **Access Method**: Browser-impersonated HTTP requests via `curl_cffi` (Chrome 124 fingerprint), after a warm-up visit to the search page to obtain cookies
- **Coverage**: Complaints published in the CFPB's public database
- **Data**: Complaint ID, company, product, sub-product, issue, sub-issue, company response, consumer consent status, narrative presence, and date received
- **Access**: Public API, no API key required
- **Update Frequency**: Reflects the database at time of run

***

### ⚙️ How It Works

The CFPB Consumer Complaint Alert actor accepts a search term and a number of days to look back. It requests complaints from the CFPB database newest first, builds one record per complaint, and compares each complaint ID with the list of IDs saved in the Apify Key-Value Store. Complaints not in that list are marked `is_new`. After the run, all IDs seen are saved for next time, and the records are pushed to the Apify Dataset as one batch.

**Key Processing Steps:**

1. **Input Parsing** – Read `searchTerm`, `daysBack`, and `onlyNewAlerts`
2. **Proxy Setup** – Configure Apify Proxy if enabled, with automatic direct-connection fallback
3. **Session Warm-Up** – Visit the CFPB search page to obtain cookies
4. **Load State** – Read previously seen complaint IDs for this search term from the Key-Value Store
5. **Fetch Complaints** – Request complaints newest first, 100 per page, up to 1,000
6. **New Detection** – Mark complaints not seen before as `is_new`
7. **Filtering** – If `onlyNewAlerts` is on, skip complaints seen in earlier runs
8. **Save State** – Store the updated list of seen IDs
9. **Dataset Push** – Push all **Complaint Alert** records at once
10. **Summary Logging** – Report baseline creation, new alerts, or no new complaints

**Key Benefits:**

- Monitor complaints about a company or topic without manual searching
- See at a glance which complaints are new
- Run on a schedule to build an automated **Complaint Alert** feed
- Get clean, structured records ready for dashboards or notifications

***

### 📥 Input

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `searchTerm` | string | `"Equifax"` | Term to search for. Leave empty to search all complaints. |
| `daysBack` | integer | `30` | How many days back to search. |
| `onlyNewAlerts` | boolean | `false` | If `true`, returns only complaints not seen in earlier runs. |
| `proxyConfiguration` | object | `{"useApifyProxy": true}` | Apify Proxy configuration. |

Each run returns at most 1,000 complaints.

**Example Input:**

```json
{
  "searchTerm": "Equifax",
  "daysBack": 30,
  "onlyNewAlerts": false
}
```

**Alerts-Only Monitoring:**

```json
{
  "searchTerm": "Wells Fargo",
  "daysBack": 7,
  "onlyNewAlerts": true
}
```

**Run Without Proxy:**

```json
{
  "searchTerm": "student loan",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

***

### 📤 Output

The Actor pushes **Complaint Alert** records with the following structure:

| Field | Type | Description |
|-------|------|-------------|
| `complaint_id` | string | CFPB complaint ID |
| `company` | string | Company the complaint is about |
| `product` | string | Financial product category |
| `sub_product` | string | Product sub-category |
| `issue` | string | Main issue reported |
| `sub_issue` | string | Issue sub-category |
| `status` | string | Company response status |
| `consumer_consent` | string | Consumer consent status for publishing the narrative |
| `has_narrative` | boolean | Whether the complaint includes a consumer narrative |
| `date` | string | Date the complaint was received (`YYYY-MM-DD`) |
| `is_new` | boolean | `true` if the complaint was not seen in earlier runs |
| `link` | string | Link to the complaint's detail page |

The narrative text itself is not included.

**Example Complaint Alert Record:**

```json
{
  "complaint_id": "1234567",
  "company": "EQUIFAX, INC.",
  "product": "Credit reporting or other personal consumer reports",
  "sub_product": "Credit reporting",
  "issue": "Incorrect information on your report",
  "sub_issue": "Information belongs to someone else",
  "status": "Closed with explanation",
  "consumer_consent": "Consent provided",
  "has_narrative": true,
  "date": "2026-10-28",
  "is_new": true,
  "link": "https://www.consumerfinance.gov/data-research/consumer-complaints/search/detail/1234567"
}
```

***

### 🧰 Technical Stack

- **HTTP Client:** `curl_cffi` with Chrome 124 TLS fingerprint impersonation
- **Source:** CFPB Consumer Complaint Database search API
- **State Storage:** Apify Key-Value Store, one key per search term (`SEEN_IDS_<TERM>`)
- **Async:** `asyncio` with executor-run blocking requests and a short delay between pages
- **Proxy:** Apify Proxy, optional via `useApifyProxy`, with direct fallback on HTTP 403 or errors
- **Logging:** Apify Actor logging system
- **Platform:** Apify Actor serverless environment

***

### 📊 Data Fields Explained

#### **Identification**

- **complaint_id / link**: The complaint's ID and its detail page

#### **Company & Product**

- **company / product / sub_product**: Who the complaint is about and the product involved

#### **Issue & Outcome**

- **issue / sub_issue / status**: What went wrong and how the company responded

#### **Consumer Details**

- **consumer_consent / has_narrative**: Consent status and whether a narrative exists

#### **Monitoring**

- **date**: When the complaint was received
- **is_new**: Whether this complaint is new compared with earlier runs

***

### 🎯 Use Cases

- **Reputation Monitoring** – Track new complaints about your company or a competitor
- **Compliance Watchlists** – Follow complaint volumes for regulated products
- **Financial Journalism** – Spot emerging consumer issues
- **Market Research** – Study which products and issues draw complaints
- **Alert Pipelines** – Send notifications when new complaints appear
- **Trend Analysis** – Build datasets of complaint dates, issues, and company responses

***

### 🚀 Quick Start

#### **1. Prepare Input**

Go to Apify Console and enter:

```json
{
  "searchTerm": "Equifax",
  "daysBack": 30
}
```

#### **2. Run the Actor**

Click **Start**. The Actor will:

- Connect through Apify Proxy if enabled
- Load the saved list of seen complaint IDs
- Fetch complaints newest first
- Flag new complaints and push all results to the Dataset

#### **3. Monitor Progress**

Console shows (first run):

```
🛡️ Apify Proxy connected.
🔎 Searching CFPB complaints for 'Equifax' from last 30 days...
Fetched 100 complaints.
🏁 Baseline initialized: 100 complaints recorded.
✅ Successfully pushed 100 items to dataset!
🎉 Complete!
```

Later runs log `🚨 ALERT: N NEW complaint(s) identified since last run!` or `ℹ️ No new complaints since last run.`

#### **4. View & Download Results**

- **Results Tab**: All **Complaint Alert** records
- **Filter**: By `is_new`, company, product, or date
- **Export**: JSON, CSV, Excel, or HTML
- **API Access**: Available via the Apify API

***

### ⚙️ Configuration

Broad scan of all recent complaints:

```json
{
  "searchTerm": "",
  "daysBack": 7
}
```

Alerts only, for a daily schedule:

```json
{
  "searchTerm": "Equifax",
  "daysBack": 3,
  "onlyNewAlerts": true
}
```

Direct connection without proxy:

```json
{
  "searchTerm": "mortgage",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

***

### 📈 Performance

#### **Processing Speed**

- 100 complaints per request, with a 0.3 second pause between pages
- Up to 1,000 complaints per run
- Paging stops when a page returns no new items or fewer items than requested

#### **Resource Usage**

- Memory: Low, since no browser instance is launched
- Network: One warm-up request plus one request per 100 complaints
- Proxy: Optional Apify Proxy with fallback on failure

***

### ⚠️ Important Notes

#### **How New Complaints Are Detected**

- **First run**: When no saved list exists for a search term, every complaint is marked `is_new: true`, the run is logged as a baseline, and `onlyNewAlerts` does not filter anything.
- **Later runs**: Complaints whose IDs were saved earlier are marked `is_new: false`. With `onlyNewAlerts` on, they are left out of the output.
- **Separate state per search term**: Each search term has its own saved list.
- **State is saved every run**: All IDs seen in a run are saved, including those left out by `onlyNewAlerts`.
- **Storage persistence**: The saved list lives in the Actor's Key-Value Store. Apify's default key-value store is created per run, so for "new since last run" to work across scheduled runs the state must be kept in a store that persists between runs (for example a named store). Check this setup before relying on `is_new`.

#### **Legal & Compliance**

- **Public Data Only**: Retrieves only complaint data the CFPB publishes publicly
- **Fair Use**: Uses sequential requests with a short delay to keep load on the source minimal
- **Privacy**: The Actor does not collect complaint narrative text
- **Legal**: Not legal advice

#### **Data Quality**

- **Unverified Complaints**: Complaints are consumer submissions and are not verified by the CFPB as facts. A complaint does not prove wrongdoing by the company named.
- **Run Limit**: A single run returns at most 1,000 complaints. Use a shorter `daysBack` if a search returns more.
- **Failures**: If a page cannot be fetched after retries, the Actor logs an error, stops paging, and saves what it has collected so far.
- **Freshness**: Reflects the CFPB database at time of run

#### **Best Practices**

- Keep `daysBack` short on scheduled runs so each run stays under the 1,000 complaint limit
- Use `onlyNewAlerts` for scheduled **Complaint Alert** monitoring
- Use a separate scheduled run for each search term you want to track
- Verify important complaints on the CFPB website before acting on them

***

### 📄 License & Legal

**Terms of Use:**

- Use for legitimate monitoring, research, and journalism
- Respect the terms of use of the CFPB website
- Do not present unverified complaints as established facts
- Use data responsibly and in compliance with applicable laws

**Disclaimer:**
CFPB Consumer Complaint Alert & Harvester is provided as-is for research and monitoring purposes. It is not affiliated with or endorsed by the CFPB. Users are responsible for compliance with all applicable laws and the data source's terms. This is not legal advice.

***

### ⚖️ Complaint Alert Excellence

This Actor is optimized for **Complaint Alert** monitoring with:

- ✅ Official CFPB complaint data
- ✅ New-complaint detection with `is_new` flags
- ✅ Alerts-only mode
- ✅ Keyword and date-window filtering
- ✅ Automatic pagination, retries, and proxy fallback
- ✅ Production-ready code

# Actor input Schema

## `searchTerm` (type: `string`):

Full-text search across all fields (e.g. 'Equifax', 'Wells Fargo', 'identity theft', 'credit card').

## `daysBack` (type: `integer`):

How many days back to look for complaints.

## `onlyNewAlerts` (type: `boolean`):

If enabled, only newly arrived complaints since the last run will be pushed to the dataset.

## `proxyConfiguration` (type: `object`):

Apify Proxy configuration (US-based residential/datacenter proxy recommended).

## Actor input object example

```json
{
  "searchTerm": "Equifax",
  "daysBack": 30,
  "onlyNewAlerts": false,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `crawlResults` (type: `string`):

No description

## `complaintOverview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchTerm": "Equifax",
    "daysBack": 30
};

// Run the Actor and wait for it to finish
const run = await client.actor("datapilot/cfpb-consumer-complaint-alert-harvester").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchTerm": "Equifax",
    "daysBack": 30,
}

# Run the Actor and wait for it to finish
run = client.actor("datapilot/cfpb-consumer-complaint-alert-harvester").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchTerm": "Equifax",
  "daysBack": 30
}' |
apify call datapilot/cfpb-consumer-complaint-alert-harvester --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datapilot/cfpb-consumer-complaint-alert-harvester"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/QDZVpOtUVKDhukqS9/builds/ktmgsPzgofaaIZrDg/openapi.json