Changelog
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog ,
and this project adheres to Semantic Versioning .
[2.1.0] - 2025-01-14
🆕 Business Intelligence & Pricing Data
New Company Fields
Min project size : Minimum project budget requirement ($USD)Hourly rate range : From/to hourly rates ($USD/hr)Employees count : Team size range (e.g., "50-249", "2-9")Year founded : Company establishment yearClient testimonials : "What Clients Have Said" summary with pricing insights
Enhanced Review Analysis
Project Summary : Detailed project description extracted from reviewsFeedback Summary : Results and outcomes summaryDetailed Ratings : Quality, Schedule, Cost ratings with individual commentsFull Review : Complete review text with improved text cleaning
Technical Improvements
Advanced text cleaning : Removes newlines, tabs, excessive whitespace from all fieldsPosition-based parsing : More reliable extraction using HTML structure analysisRobust fallback logic : Multiple extraction strategies for maximum data coverageUniversal parsing : Works across different company types and profiles
🔧 Output Schema Updates
Added new fields to CSV/JSON export
Enhanced Apify Dataset Schema with new pricing and review fields
Updated overview and reviews views in Apify Console UI
Improved field labels and formatting
[2.0.0] - 2025-01-10
🚀 Major Enhancements
New Data Fields
Domain extraction : Real company domains with intelligent parsingNumeric fields : minProjectSize, hourlyRateFrom/To, employeesFrom/To as numbersIndustries breakdown : Complete industry focus with percentagesClient focus : Client size/type distribution with percentagesMost common project size : Extracted as numeric valueTimezone : Company timezone informationMultiple addresses : Support for multiple office locationsEnhanced verification : Detailed business entity and legal filing status
Intelligent deduplication : Global tracking prevents duplicate companies across pagesFollow redirects : Option to follow redirects for accurate domain extractionRobust error handling : Graceful handling of frame detachment and timeout errorsIncreased timeouts : Protocol timeout increased to 3 minutes for stabilitySmart retry logic : Failed requests retry up to 3 times
Export Improvements
CSV Excellence : Switched to csv-writer library for perfect CSV generationArray formatting : Services, industries as separate columns with percentagesHTML entity decoding : All special characters properly decodedIndexed columns : Reviews and portfolio items in numbered columnsClean data : Aggressive whitespace and quote cleaning
New Features
LIST_WEBSITES mode : Optimized scraping mode for faster resultsReview sorting : Sort reviews by relevance, date, or ratingDetailed statistics : Comprehensive runtime statistics with:
Duplicate count tracking
Error categorization
Performance metrics
Beautiful console output
📊 Statistics Dashboard
📊 SUMMARY STATISTICS :
• Total unique companies collected
• Duplicates removed
• Runtime and speed metrics
• Error breakdown by type
• Performance analysis
🔧 Technical Improvements
Enhanced TypeScript types
Better memory management
Optimized concurrency settings
Improved logging with progress tracking
Clean code architecture
🐛 Bug Fixes
Fixed double-saving of companies in LIST_DETAIL mode
Resolved CSV export issues with nested objects
Fixed review parsing selectors
Corrected domain extraction logic
Fixed race conditions in parallel processing
[1.0.0] - 2024-01-09
Added
Initial release of Clutch.co Scraper
Support for scraping company listings with filters
Detailed company profile extraction including:
Basic information (name, website, tagline, rating)
Verification status and details
Business entity information
Portfolio items
Client reviews with ratings
Service focus areas with percentages
Founded date and full description
Multiple scraping modes: LIST, LIST_DETAIL
Pagination support with customizable limits
Review scraping with configurable max reviews per company
Portfolio scraping (optional)
Custom data extension via extendOutputFunction
Dataset clearing option for development
Support for both headless and headful browser modes
Comprehensive error handling and retry logic
TypeScript support with full type definitions
Optimized Docker image with Puppeteer pre-installed
Features
Flexible URL support : Direct profile URLs or filtered list pagesSmart pagination : Handles Clutch's pagination with configurable limitsData enrichment : Merges data from list and detail pagesPerformance optimized : Configurable concurrency and request limitsDeveloper friendly : TypeScript, detailed logging, clear error messages
Technical Details
Built with Crawlee (formerly Apify SDK v3)
Uses PuppeteerCrawler for JavaScript rendering
Implements request routing for different page types
Includes anti-blocking measures (proxy support, fingerprinting)
[0.1.0] - 2024-01-08
Added
Initial development version
Basic scraping functionality
Testing and debugging features
