ConsumerAffairs Reviews Scraper

Scrape company reviews and ratings from ConsumerAffairs.com by company URL. Extract review text, star ratings, dates, reviewer names, verified buyer badges, company responses, and location data — all without an API key.

⚠️ Proxy Note: ConsumerAffairs uses anti-bot protection that may block datacenter IPs. If you get 0 results, try running with residential proxies or locally with a VPN.

Features

• 🔍 Search by URL — pass any ConsumerAffairs company page URL
• ⭐ Full review data — rating, title, body, date, reviewer, location
• ✅ Verified buyer detection — identifies verified purchaser reviews
• 💬 Company responses — captures official company replies to reviews
• 🏢 Company metadata — overall rating, total review count, category
• 📄 Pagination — automatically follows pagination across all review pages
• 🔄 Multi-layout parsing — handles HTML, JSON-LD schema, and API-intercepted data
• 🛡️ Anti-bot bypass — uses Apify residential proxy + Cloudflare challenge detection
• 🚫 No API key required — fully client-side scraping

Input

Field	Type	Required	Default	Description
companyUrls	array<string>	✅	—	ConsumerAffairs company URLs (e.g., https://www.consumeraffairs.com/travel/united.html)
maxReviews	integer	❌	200	Maximum reviews to scrape per company
includeCompanyMetadata	boolean	❌	true	Extract overall rating, total reviews, category
useProxy	boolean	❌	true	Use Apify residential proxy for anti-bot bypass

Example Input

{
    "companyUrls": [
        "https://www.consumeraffairs.com/travel/united.html",
        "https://www.consumeraffairs.com/finance/wells-fargo.html"
    ],
    "maxReviews": 500,
    "includeCompanyMetadata": true,
    "useProxy": true
}

Output

Each review is pushed to the Apify dataset as a JSON object:

Field	Type	Description
reviewId	string	Unique review identifier (from CA or content hash)
sourceUrl	string	URL the review was scraped from
rating	integer/float	Star rating (1–5)
reviewer	string	Reviewer display name
date	string	Review date (as shown on page)
reviewTitle	string	Review headline/summary
reviewText	string	Full review body (max 5,000 chars)
verifiedBuyer	boolean	Whether the reviewer is a verified buyer
companyResponse	string	Official company response if present
location	string	Reviewer location if available
companyName	string	Company name (if metadata enabled)
overallRating	float	Company's overall rating (if metadata enabled)
totalReviews	integer	Company's total review count (if metadata enabled)
category	string	Company category (if metadata enabled)

Example Output

{
    "reviewId": "abc123def456",
    "sourceUrl": "https://www.consumeraffairs.com/travel/united.html?page=1",
    "rating": 2,
    "reviewer": "John D.",
    "date": "August 1, 2024",
    "reviewTitle": "Cancelled flight, no refund",
    "reviewText": "My flight was cancelled and I still haven't received a refund after 3 months...",
    "verifiedBuyer": true,
    "companyResponse": "We apologize for the inconvenience. Our team is looking into this...",
    "location": "Chicago, IL",
    "companyName": "United Airlines",
    "overallRating": 1.4,
    "totalReviews": 12847,
    "category": "travel"
}

Use Cases

🏢 Brand Reputation Monitoring

Track what consumers are saying about your company across ConsumerAffairs. Monitor review volume, average ratings, and sentiment trends over time.

📊 Competitor Analysis

Scrape competitor review pages to benchmark customer satisfaction, identify common complaints, and find gaps in the market.

📈 Market Research

Aggregate consumer feedback across an entire industry category (insurance, finance, telecom, travel). Identify recurring issues and pain points.

💬 Customer Experience Analytics

Extract review text for NLP sentiment analysis, topic modeling, or feed into LLM pipelines for automated insight generation.

⚖️ Compliance & Dispute Tracking

Monitor verified buyer complaints and company responses for regulatory compliance or legal evidence gathering.

How It Works

1. Playwright renders each company page with full JavaScript execution
2. Lazy-loading triggered via scroll simulation to load all reviews
3. API interception captures any XHR/fetch JSON responses containing review data
4. Multi-strategy parsing extracts reviews from HTML containers, JSON-LD schema, or intercepted API payloads
5. Cloudflare detection identifies challenge pages and retries automatically
6. Pagination follows "Next" links, numeric page links, or constructs ?page=N URLs
7. Deduplication ensures the same review never appears twice across pages or sources

Technical Details

• Runtime: Python 3.12 + Playwright (Chromium)
• Proxy: Apify residential proxy (configurable)
• Parsing: BeautifulSoup + lxml for HTML; native JSON for API/JSON-LD
• Anti-bot: Cloudflare challenge detection with retry logic
• Dedup: SHA-256 content hashing for reviews without explicit IDs

Related Actors

• Google Maps Reviews Scraper — scrape Google Maps reviews
• Trustpilot Reviews Scraper — extract Trustpilot company reviews
• Yelp Reviews Scraper — scrape Yelp business reviews
• Amazon Reviews Scraper — get Amazon product reviews
• Glassdoor Reviews Scraper — scrape employer reviews

Limitations

• ConsumerAffairs may rate-limit or block IPs; using Apify proxy is strongly recommended
• Some pages may trigger Cloudflare challenges — the actor retries but cannot guarantee bypass
• Review text is capped at 5,000 characters per review
• Maximum 10,000 reviews per company (configurable via input)

Support

If you encounter issues or have feature requests, please open an issue on the actor's Apify store page.

---

No API key required. No login required. Just provide company URLs and start scraping.