# Swe Bench Leaderboard Scraper (`bigibyte/swe-bench-leaderboard-scraper`) Actor

Collect AI coding agent benchmark scores from SWE-bench leaderboards. Extract performance metrics, agent details, and resolve rates into clean structured datasets.

- **URL**: https://apify.com/bigibyte/swe-bench-leaderboard-scraper.md
- **Developed by:** [BigiByte](https://apify.com/bigibyte) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SWE-bench Leaderboard Scraper

![SWE-bench Leaderboard Scraper cover](https://images.bigibyte.com/cdn-cgi/image/width=1260,height=540,fit=cover,format=webp/apify-actors-covers/swe-bench-leaderboard-scraper-cover.png)

Collect AI coding agent benchmark results from SWE-bench leaderboards. Pick a mode, set item limits, and export clean JSON, CSV, or Excel data seamlessly.

### What it does

This scraper processes SWE-bench leaderboard metrics and flattens each board's results into structured dataset rows. You can extract data across all boards or focus on specific ones like Verified, Lite, Multilingual, and others. Each record captures key performance indicators such as agent details, evaluation model, cost, resolved score percentage, and related benchmark parameters.

### Modes

- **All leaderboards**: Extracts and flattens benchmark metrics across all boards (or a selected list of target boards) up to your configured maximum item limit.

### Input example

```json
{
  "mode": "all",
  "leaderboards": ["Verified", "Lite"],
  "maxItems": 50,
  "requestDelaySecs": 1,
  "requestTimeoutSecs": 30
}

```

### Output example

```json
{
  "leaderboard": "Multilingual",
  "agent": "mini-SWE-agent",
  "agent_org": "SWE-agent",
  "checked": true,
  "cost": 105.55,
  "date": "2026-02-13",
  "folder": "20260213_mini-v2.0.0a0_gemini-3-flash",
  "model_display": "Gemini 3 Flash",
  "model_org": "Google DeepMind",
  "name": "Gemini 3 Flash",
  "resolved": 72.7,
  "scraped_at": "2026-10-26T00:00:00+00:00"
}

```

The overview output table organizes fields including leaderboard, agent, agent_org, checked status, cost, evaluation date, folder identifier, and execution timestamps.

### Pricing and usage limits

- **Pay-as-you-go / Compute units**: Compute usage depends on the number of leaderboards selected, your `maxItems` cap, and network response speed.
- **Custom limits**: Set `maxItems` in the input configuration to strictly control dataset size and resource consumption per run.
- **Scheduled runs**: Running small batch updates periodically consumes minimal compute resources while maintaining up-to-date benchmark tracking.

### Use cases

- **Benchmark tracking**: Gather updated SWE-bench scores for AI coding agents and models directly into structured spreadsheets.
- **Competitive analysis**: Side-by-side evaluation of Lite and Verified benchmark performances across leading LLMs.
- **Model monitoring**: Track resolve rate improvements and compute cost efficiency trends over time.
- **Data analysis**: Export datasets directly to CSV or Excel formats for python analysis notebooks and visual dashboards.
- **Research reporting**: Keep track of structured evaluation records across various agent architectures.

### FAQ

**Who is this for?**

Researchers, developers, and AI analysts looking to aggregate and compare software engineering agent benchmarks without manual record-keeping.

**What export formats are supported?**

JSON, CSV, Excel, XML, and other standard Apify export formats.

**Can I run it on a schedule?**

Yes. Use Apify schedules to automatically refresh your dataset on an automated hourly, daily, or weekly interval.

**What happens when a data request encounters an error?**

The execution handles errors gracefully by recording an error status row and continuing the rest of the extraction job.

**How do I filter specific boards?**

Specify desired names such as Verified or Lite in the leaderboards input list. Leaving the array empty processes all available leaderboards automatically.

### Related tools & custom pipelines

If you plan to map, analyze, or join these benchmark datasets, BigiByte provides full data scraping solutions tailored to your technical requirements.

Need extra open-data extractions, automated benchmark monitors, or specialized data pipelines? BigiByte builds custom scrapers and APIs: <https://bigibyte.com>

### Trademark Notice

SWE-bench and related names, logos, and trademarks belong to their respective owners. Their use in this documentation is solely to identify the subject of evaluation and does not imply affiliation, sponsorship, endorsement, or partnership.

### Disclaimer

This scraper is an independent automation tool developed by BigiByte and is not affiliated with, endorsed by, sponsored by, or officially associated with SWE-bench or its maintainers. Users are responsible for ensuring their usage complies with relevant terms, guidelines, and policies.

### Is web scraping legal?

This actor extracts publicly accessible data without bypassing paywalls, or technical security layers. While collecting public web data is widely permitted, compliance depends on how data is gathered and used:

- **Personal data**: When collecting identifiers like emails, phone numbers, or social handles, ensure compliance with applicable data protection laws such as GDPR or CCPA.
- **Copyrighted content**: Public text, images, and visual assets are often protected by intellectual property rights. Avoid directly duplicating or re-distributing full creative works.
- **Platform terms & rate limits**: Practice responsible data extraction by avoiding aggressive traffic spikes, adhering to rate limits, and respecting published platform policies.
- **Disclaimer**: This summary serves informational purposes only and does not constitute formal legal counsel. Seek professional legal advice for specific compliance queries.

For a detailed guide, see Apify's legal overview: [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/)

# Actor input Schema

## `mode` (type: `string`):

What to scrape. All mode flattens each leaderboard's results.

## `leaderboards` (type: `array`):

Optional leaderboard names to keep (for example Verified or Lite). Leave empty to include all.

## `maxItems` (type: `integer`):

Stop after this many successful rows. Hard max is 1000.

## `requestDelaySecs` (type: `number`):

Seconds to wait between requests.

## `userAgent` (type: `string`):

User-Agent header sent with each request.

## `timeoutSecs` (type: `integer`):

Overall run budget in seconds.

## `requestTimeoutSecs` (type: `integer`):

Per-request timeout in seconds.

## `proxyConfiguration` (type: `object`):

Optional proxy. Leave off unless you need it.

## Actor input object example

```json
{
  "mode": "all",
  "leaderboards": [],
  "maxItems": 100,
  "requestDelaySecs": 1,
  "userAgent": "ActorResearch/1.0",
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `overview` (type: `string`):

Dataset rows in the overview table.

## `dataset` (type: `string`):

Full dataset for this run.

## `items` (type: `string`):

Dataset items as JSON.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "leaderboards": []
};

// Run the Actor and wait for it to finish
const run = await client.actor("bigibyte/swe-bench-leaderboard-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "leaderboards": [] }

# Run the Actor and wait for it to finish
run = client.actor("bigibyte/swe-bench-leaderboard-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "leaderboards": []
}' |
apify call bigibyte/swe-bench-leaderboard-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,bigibyte/swe-bench-leaderboard-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/YTJZZ5kB8oHx5nMhl/builds/DfLiwjFHYeqhsd2Rq/openapi.json