Dec 9, 2022
Improving observability of actor runs and proxy usage
New
Console
Proxy
Actor
One of the main challenges when scraping a website is monitoring the data quality across the actor runs. So we're working on a monitoring toolset to simplify this for users of our platform.
As a first step, we've just extended the list of runs with a couple of useful charts visualizing the main metrics, such as the number of dataset items, different fields, and the amount of data in the key-value store.
To see these charts, open a list of runs on any of your actors or tasks, and click "Show statistics."
Is there anything you'd like us to add? Let us know!
In addition, we've largely improved the proxy page that now includes a complete breakdown of usage per domain and proxy type.