We are pushing daily updates and improvements.
Notable changes to this actor and our webscraping framework will be documented here.
2026-01-27
🚀 Features
Added graph mode to all crawler. This allows combining structured extraction for item pages as well as capturing the relashionship between all pages of the website
2026-01-20
🚀 Features
Improved extracted item metadata to always include the page title, page kind (home/list/item/other) and the list of outbound links, in addition to the url and extraction timestamp
2026-01-17
🚀 Features
Implemented direct API scraping for item pages to replace JS rendering. This combines the speed of simple http requests with the richness of a fully rendered page
2026-01-10
🚀 Performance
Reduced cold start time for new jobs
Increased extraction speed by ~10%
2025-12-30
🚀 Improvements
Improved crawler stealth
Improved chrome fingerprint spoofing
Improved http client impersonation
2025-12-17
🚀 Features
Added direct API scraping for paginated list pages (alternative to chrome rendering with infinite scroll), resulting in 10x speedup for crawling catalog pages
2025-12-10
🚀 Improvements
Improved URL detection logic for different page types
2025-12-08
🚀 Features
Added universal extraction modes, in addition to structured (schema-driven) extraction
2025-12-07
🚀 Improvements
Improved crawler stealth and robustness to Anti-Bot scripts