Map any website's structure. Our AI analyzes the site and builds a crawler that extracts page relationships, perfect for understanding architecture before running extraction jobs.
We are pushing daily updates and improvements.
Notable changes to this actor and our webscraping framework will be documented here.
2026-01-27
🚀 Features
Added graph mode to all crawler. This allows combining structured extraction for item pages as well as capturing the relashionship between all pages of the website
2026-01-20
🚀 Features
Improved extracted item metadata to always include the page title, page kind (home/list/item/other) and the list of outbound links, in addition to the url and extraction timestamp
2026-01-17
🚀 Features
Implemented direct API scraping for item pages to replace JS rendering. This combines the speed of simple http requests with the richness of a fully rendered page
2026-01-10
🚀 Performance
Reduced cold start time for new jobs
Increased extraction speed by ~10%
2025-12-30
🚀 Improvements
Improved crawler stealth
Improved chrome fingerprint spoofing
Improved http client impersonation
2025-12-17
🚀 Features
Added direct API scraping for paginated list pages (alternative to chrome rendering with infinite scroll), resulting in 10x speedup for crawling catalog pages
2025-12-10
🚀 Improvements
Improved URL detection logic for different page types
2025-12-08
🚀 Features
Added universal extraction modes, in addition to structured (schema-driven) extraction
2025-12-07
🚀 Improvements
Improved crawler stealth and robustness to Anti-Bot scripts
