ConsumerAffairs Review Scraper

Scrape customer reviews and warranty service ratings from ConsumerAffairs.com with automatic pagination handling and structured data output.

---

🚀 Features

• Comprehensive Review Extraction: Collects reviewer names, locations, dates, ratings, and full text
• Automatic Pagination: Handles multi-page reviews using response_url for continuous scraping
• Proxy Ready: Built-in Apify Proxy support for reliable data collection
• Rate Limit Control: item_limit parameter to manage request volume

---

⚙️ Input Configuration

{
  "proxyConfiguration": {
    "useApifyProxy": true
  },
  "item_limit": 40,
  "urls": [
    "https://www.consumeraffairs.com/auto_warranty/endurance-warranty.html"
  ]
}

Input Parameters

Parameter	Type	Description
proxyConfiguration	Object	Standard Apify proxy configuration
item_limit	Number	Max reviews to collect per URL (1-100)
urls	Array	Service review page URLs

---

📊 Output Sample

{
  "response_url": "https://www.consumeraffairs.com/...page=2",
  "name": "Keith",
  "location": "Hutchinson, MN",
  "date": "Reviewed April 30, 2025",
  "review_text": "The purchase process was very straightforward...",
  "rating": "4"
}

Output Fields

Field	Description
response_url	Next pagination URL (null on last page)
name	Reviewer first name (anonymized)
location	City/State location of reviewer
date	Original review timestamp format
review_text	Full review content (250+ chars)
rating	1-5 star rating (string format)

---

🐍 Python Usage Example

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_KEY")

run_input = {
    "item_limit": 50,
    "urls": ["https://www.consumeraffairs.com/finance/company.html"],
    "proxyConfiguration": {"useApifyProxy": True}
}

actor_call = client.actor("your-username/my-consumeraffairs-actor").call(run_input=run_input)

# Process results with pagination support
for item in client.dataset(actor_call["defaultDatasetId"]).iterate_items():
    if item['response_url']:
        print(f"Next page: {item['response_url']}")
    print(f"Review from {item['name']} ({item['rating']}/5)")

---

🛠️ Support & Customization

Need custom fields or higher limits?
✉️ Email: support@getdataforme.com
📬 Contact Form: https://getdataforme.com/contact/

---

🔍 Technical Details

• Pagination Handling: Automatic detection of "Next Page" links
• Data Sanitization: Clean HTML-to-text conversion
• Error Handling: 3 retries for failed requests
• Rate Limits: 5 requests/second default
• Legal Compliance: Adheres to ConsumerAffairs robots.txt

Last Updated: October 2023 | Actor Version: 1.0.0