Actor Benchmark
Pricing
Pay per usage
Compares various builds of the same actor to measure how they perform on the same input
Pricing
Pay per usage
Rating
5.0
(1)
Developer
Apify
Maintained by ApifyActor stats
2
Bookmarked
5
Total users
0
Monthly active users
15 days ago
Last modified
Categories
Share
List of build numbers or tags to benchmark. Each entry will receive an identical run in parallel. You can use exact version numbers (e.g. "0.3.67") or tags (e.g. "canary"), which will use the latest build associated with that tag.
Number of times each build is run. Results are averaged to reduce the effect of outliers. All runs are started in parallel on your account, so make sure you have enough memory quota available to allow them all to start simultaneously.
The ID or name of the Apify actor to benchmark (e.g. "apify/website-content-crawler").
The JSON input passed to every run of the benchmarked actor.
The ID or name of an Apify task to benchmark (e.g. "my-username/my-task").
Timeout in seconds for each actor run. Runs exceeding this limit will be aborted.
Memory in megabytes allocated to each actor run.
A JavaScript function expression called once for every finished run, to pick what you want to measure. It receives one object with: run (the run object from the Apify API, refreshed after the run finished), client (an authenticated Apify API client, e.g. to read the run's dataset or key-value store), build (the build number or tag), runNumber (1-based index of the run within its build) and utils (sum, mean, median and round helpers). It may be async and must return a plain object, e.g. { durationSecs: 12.3, emptyItems: 4 }. The object is stored as is in the "allRuns" dataset and passed to "Aggregate function". The code runs inside this Actor with your API token; it is not sandboxed. Must be provided together with "Aggregate function".
A JavaScript function expression called once for every build, to turn the objects extracted from that build's runs into the statistics to report (averages, percentiles, …). It receives one object with: build (the build number or tag), runs (array of the objects returned by "Extract function" for the build's succeeded or timed-out runs whose extraction worked) and utils (sum, mean, median and round helpers). It may be async and must return a plain object, e.g. { meanDurationSecs: 12.3 }. The object, plus build, becomes one item of the default dataset, and every field that is a number in all builds is plotted in its own chart. Each build is aggregated on its own, so the function cannot compare builds with each other (e.g. against a baseline build).
Save one bar chart per numeric field returned by the aggregate function to the key-value store, with the builds on the X axis.
Your request to an AI model that reads the statistics returned by the aggregate function and writes a report, e.g. "Which build is fastest, and is the cost difference worth it?". Leave empty to not generate a report. Only the statistics of each build, plus which actor or task was benchmarked and how many runs each build had, are sent to the model, not the raw run data. The request goes through the Apify OpenRouter Actor, which starts an additional run billed to your account. The report is saved as "report.md" in the key-value store and written to the log. A failed request only logs a warning.
