CFPB Consumer Complaint Alert & Harvester

🚨 CFPB Consumer Complaint Alert & Harvester is a powerful Apify Actor designed to collect consumer financial complaints from the US Consumer Financial Protection Bureau (CFPB) and flag the ones that are new since your last run. This tool provides Complaint Alert intelligence including company, product, issue, company response, complaint date, and a direct link to each complaint. Whether you're monitoring a company's reputation, tracking financial product issues, or building a compliance watchlist, the Complaint Alert Harvester delivers actionable results efficiently.

With keyword search, a configurable date window, automatic pagination, new-complaint detection, and reliable Apify Dataset delivery, this Complaint Alert actor remembers which complaints it has already seen and marks the rest as new. It focuses on key signals including company, product, issue, and complaint date, making Complaint Alert monitoring fast and structured.

---

📋 Table of Contents

• Features
• Data Source
• How It Works
• Input
• Output
• Technical Stack
• Data Fields
• Use Cases
• Quick Start
• Configuration
• Performance
• Important Notes
• License & Legal

---

🔥 Features

• CFPB Complaint Search – Pulls complaints from the CFPB's official Consumer Complaint Database. This is the core of Complaint Alert monitoring.
• Keyword Search – Search by company name or any term (default: Equifax), or leave it empty to search all complaints.
• Configurable Date Window – Look back any number of days with daysBack.
• New Complaint Detection – Compares complaint IDs against those saved from previous runs and sets an is_new flag on each record.
• Alerts-Only Mode – Turn on onlyNewAlerts to receive only complaints not seen before.
• Newest First – Complaints are fetched in order of creation date, most recent first.
• Automatic Pagination – Fetches complaints 100 at a time, up to 1,000 per run.
• Direct Complaint Links – Each record includes a link to the complaint's detail page.
• Narrative Flag – Shows whether a complaint includes a consumer narrative (has_narrative).
• Retry Handling – Retries failed requests up to 3 times with increasing wait between attempts.
• Apify Proxy Support – Optional Apify Proxy, with a direct-connection retry if the proxy receives HTTP 403.
• Single-Batch Dataset Push – Pushes all records to the Apify Dataset together.
• Detailed Logging – Logs baseline creation, new-complaint alerts, and completion for every Complaint Alert run.

---

📊 Data Source

CFPB Consumer Complaint Database

• Authority: US Consumer Financial Protection Bureau (consumerfinance.gov)
• Endpoint: https://www.consumerfinance.gov/data-research/consumer-complaints/search/api/v1/
• Access Method: Browser-impersonated HTTP requests via curl_cffi (Chrome 124 fingerprint), after a warm-up visit to the search page to obtain cookies
• Coverage: Complaints published in the CFPB's public database
• Data: Complaint ID, company, product, sub-product, issue, sub-issue, company response, consumer consent status, narrative presence, and date received
• Access: Public API, no API key required
• Update Frequency: Reflects the database at time of run

---

⚙️ How It Works

The CFPB Consumer Complaint Alert actor accepts a search term and a number of days to look back. It requests complaints from the CFPB database newest first, builds one record per complaint, and compares each complaint ID with the list of IDs saved in the Apify Key-Value Store. Complaints not in that list are marked is_new. After the run, all IDs seen are saved for next time, and the records are pushed to the Apify Dataset as one batch.

Key Processing Steps:

1. Input Parsing – Read searchTerm, daysBack, and onlyNewAlerts
2. Proxy Setup – Configure Apify Proxy if enabled, with automatic direct-connection fallback
3. Session Warm-Up – Visit the CFPB search page to obtain cookies
4. Load State – Read previously seen complaint IDs for this search term from the Key-Value Store
5. Fetch Complaints – Request complaints newest first, 100 per page, up to 1,000
6. New Detection – Mark complaints not seen before as is_new
7. Filtering – If onlyNewAlerts is on, skip complaints seen in earlier runs
8. Save State – Store the updated list of seen IDs
9. Dataset Push – Push all Complaint Alert records at once
10. Summary Logging – Report baseline creation, new alerts, or no new complaints

Key Benefits:

• Monitor complaints about a company or topic without manual searching
• See at a glance which complaints are new
• Run on a schedule to build an automated Complaint Alert feed
• Get clean, structured records ready for dashboards or notifications

---

📥 Input

Field	Type	Default	Description
searchTerm	string	"Equifax"	Term to search for. Leave empty to search all complaints.
daysBack	integer	30	How many days back to search.
onlyNewAlerts	boolean	false	If true, returns only complaints not seen in earlier runs.
proxyConfiguration	object	{"useApifyProxy": true}	Apify Proxy configuration.

Each run returns at most 1,000 complaints.

Example Input:

{
  "searchTerm": "Equifax",
  "daysBack": 30,
  "onlyNewAlerts": false
}

Alerts-Only Monitoring:

{
  "searchTerm": "Wells Fargo",
  "daysBack": 7,
  "onlyNewAlerts": true
}

Run Without Proxy:

{
  "searchTerm": "student loan",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

---

📤 Output

The Actor pushes Complaint Alert records with the following structure:

Field	Type	Description
complaint_id	string	CFPB complaint ID
company	string	Company the complaint is about
product	string	Financial product category
sub_product	string	Product sub-category
issue	string	Main issue reported
sub_issue	string	Issue sub-category
status	string	Company response status
consumer_consent	string	Consumer consent status for publishing the narrative
has_narrative	boolean	Whether the complaint includes a consumer narrative
date	string	Date the complaint was received (YYYY-MM-DD)
is_new	boolean	true if the complaint was not seen in earlier runs
link	string	Link to the complaint's detail page

The narrative text itself is not included.

Example Complaint Alert Record:

{
  "complaint_id": "1234567",
  "company": "EQUIFAX, INC.",
  "product": "Credit reporting or other personal consumer reports",
  "sub_product": "Credit reporting",
  "issue": "Incorrect information on your report",
  "sub_issue": "Information belongs to someone else",
  "status": "Closed with explanation",
  "consumer_consent": "Consent provided",
  "has_narrative": true,
  "date": "2026-10-28",
  "is_new": true,
  "link": "https://www.consumerfinance.gov/data-research/consumer-complaints/search/detail/1234567"
}

---

🧰 Technical Stack

• HTTP Client: curl_cffi with Chrome 124 TLS fingerprint impersonation
• Source: CFPB Consumer Complaint Database search API
• State Storage: Apify Key-Value Store, one key per search term (SEEN_IDS_<TERM>)
• Async: asyncio with executor-run blocking requests and a short delay between pages
• Proxy: Apify Proxy, optional via useApifyProxy, with direct fallback on HTTP 403 or errors
• Logging: Apify Actor logging system
• Platform: Apify Actor serverless environment

---

📊 Data Fields Explained

Identification

• complaint_id / link: The complaint's ID and its detail page

Company & Product

• company / product / sub_product: Who the complaint is about and the product involved

Issue & Outcome

• issue / sub_issue / status: What went wrong and how the company responded

Consumer Details

• consumer_consent / has_narrative: Consent status and whether a narrative exists

Monitoring

• date: When the complaint was received
• is_new: Whether this complaint is new compared with earlier runs

---

🎯 Use Cases

• Reputation Monitoring – Track new complaints about your company or a competitor
• Compliance Watchlists – Follow complaint volumes for regulated products
• Financial Journalism – Spot emerging consumer issues
• Market Research – Study which products and issues draw complaints
• Alert Pipelines – Send notifications when new complaints appear
• Trend Analysis – Build datasets of complaint dates, issues, and company responses

---

🚀 Quick Start

1. Prepare Input

Go to Apify Console and enter:

{
  "searchTerm": "Equifax",
  "daysBack": 30
}

2. Run the Actor

Click Start. The Actor will:

• Connect through Apify Proxy if enabled
• Load the saved list of seen complaint IDs
• Fetch complaints newest first
• Flag new complaints and push all results to the Dataset

3. Monitor Progress

Console shows (first run):

🛡️ Apify Proxy connected.
🔎 Searching CFPB complaints for 'Equifax' from last 30 days...
Fetched 100 complaints.
🏁 Baseline initialized: 100 complaints recorded.
✅ Successfully pushed 100 items to dataset!
🎉 Complete!

Later runs log 🚨 ALERT: N NEW complaint(s) identified since last run! or ℹ️ No new complaints since last run.

4. View & Download Results

• Results Tab: All Complaint Alert records
• Filter: By is_new, company, product, or date
• Export: JSON, CSV, Excel, or HTML
• API Access: Available via the Apify API

---

⚙️ Configuration

Broad scan of all recent complaints:

{
  "searchTerm": "",
  "daysBack": 7
}

Alerts only, for a daily schedule:

{
  "searchTerm": "Equifax",
  "daysBack": 3,
  "onlyNewAlerts": true
}

Direct connection without proxy:

{
  "searchTerm": "mortgage",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

---

📈 Performance

Processing Speed

• 100 complaints per request, with a 0.3 second pause between pages
• Up to 1,000 complaints per run
• Paging stops when a page returns no new items or fewer items than requested

Resource Usage

• Memory: Low, since no browser instance is launched
• Network: One warm-up request plus one request per 100 complaints
• Proxy: Optional Apify Proxy with fallback on failure

---

⚠️ Important Notes

How New Complaints Are Detected

• First run: When no saved list exists for a search term, every complaint is marked is_new: true, the run is logged as a baseline, and onlyNewAlerts does not filter anything.
• Later runs: Complaints whose IDs were saved earlier are marked is_new: false. With onlyNewAlerts on, they are left out of the output.
• Separate state per search term: Each search term has its own saved list.
• State is saved every run: All IDs seen in a run are saved, including those left out by onlyNewAlerts.
• Storage persistence: The saved list lives in the Actor's Key-Value Store. Apify's default key-value store is created per run, so for "new since last run" to work across scheduled runs the state must be kept in a store that persists between runs (for example a named store). Check this setup before relying on is_new.

Legal & Compliance

• Public Data Only: Retrieves only complaint data the CFPB publishes publicly
• Fair Use: Uses sequential requests with a short delay to keep load on the source minimal
• Privacy: The Actor does not collect complaint narrative text
• Legal: Not legal advice

Data Quality

• Unverified Complaints: Complaints are consumer submissions and are not verified by the CFPB as facts. A complaint does not prove wrongdoing by the company named.
• Run Limit: A single run returns at most 1,000 complaints. Use a shorter daysBack if a search returns more.
• Failures: If a page cannot be fetched after retries, the Actor logs an error, stops paging, and saves what it has collected so far.
• Freshness: Reflects the CFPB database at time of run

Best Practices

• Keep daysBack short on scheduled runs so each run stays under the 1,000 complaint limit
• Use onlyNewAlerts for scheduled Complaint Alert monitoring
• Use a separate scheduled run for each search term you want to track
• Verify important complaints on the CFPB website before acting on them

---

📄 License & Legal

Terms of Use:

• Use for legitimate monitoring, research, and journalism
• Respect the terms of use of the CFPB website
• Do not present unverified complaints as established facts
• Use data responsibly and in compliance with applicable laws

Disclaimer: CFPB Consumer Complaint Alert & Harvester is provided as-is for research and monitoring purposes. It is not affiliated with or endorsed by the CFPB. Users are responsible for compliance with all applicable laws and the data source's terms. This is not legal advice.

---

⚖️ Complaint Alert Excellence

This Actor is optimized for Complaint Alert monitoring with:

• ✅ Official CFPB complaint data
• ✅ New-complaint detection with is_new flags
• ✅ Alerts-only mode
• ✅ Keyword and date-window filtering
• ✅ Automatic pagination, retries, and proxy fallback
• ✅ Production-ready code