# crates.io Scraper: Crates, Versions, Downloads & Dependents (`winterlock/crates-io-scraper`) Actor

Export Rust crate metadata, version history, daily download series, owners and reverse dependencies from the official crates.io API. No login, no HTML parsing.

- **URL**: https://apify.com/winterlock/crates-io-scraper.md
- **Developed by:** [Alex Burton](https://apify.com/winterlock) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 crate (details or search hit)s

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## crates.io Scraper: Crates, Versions, Downloads & Dependents

Export Rust package data from crates.io to JSON, CSV or Excel. Everything comes from the official crates.io API, so there is no HTML parsing and no login.

One Actor, six jobs:

| Mode | What you get |
|---|---|
| **Crate details** | Description, newest and max-stable version, all-time and recent downloads, keywords, categories, licence, minimum Rust version, edition, crate size, repository and dates |
| **Version history** | Every version with its publish date, download count, yanked flag, licence, size, minimum Rust version, edition and publisher |
| **Daily downloads** | One row per day, optionally broken down by version |
| **Owners** | Every user and team that can publish the crate |
| **Reverse dependencies** | The crates that depend on yours, with their version, download count and the version requirement they declare |
| **Registry search** | crates.io search results sorted by relevance, downloads, recency or name |

### Why use this one

- **Reverse dependencies are actually usable.** The crates.io API returns dependency edges and crate versions as two separate arrays that mean nothing until you join them on an internal ID. This Actor does the join, so you get "crate X version Y depends on you, requiring ^1.0" in one flat row.
- **Minimum Rust version and edition as columns.** `rustVersion` and `edition` are pulled from the newest version of every crate, which is what you need when checking whether a dependency tree will build on your toolchain.
- **Yanked versions kept by default.** Auditing a lockfile means seeing what was withdrawn, not hiding it.
- **Polite by design.** crates.io asks clients to identify themselves and throttles hard. This Actor sends a descriptive User-Agent and defaults to two parallel requests, so runs finish instead of getting rate-limited.

### How to use it

1. Choose **What to scrape**.
2. Add **Crates** by name or crates.io link, or write **Search queries**.
3. Click **Start**, then download the results or pull them through the Apify API.

#### Example input: who depends on my crate

```json
{
  "mode": "reverseDependencies",
  "crates": ["serde"],
  "maxReverseDependencies": 500
}
```

#### Example input: audit a dependency list

```json
{
  "mode": "crateDetails",
  "crates": ["serde", "tokio", "reqwest", "clap"]
}
```

#### Example input: the most downloaded HTTP crates

```json
{
  "mode": "search",
  "searchQueries": ["http client"],
  "searchSort": "downloads",
  "maxResultsPerSearch": 100
}
```

### Output

Each item carries a `type` field: `crate`, `version`, `downloads`, `owner` or `reverseDependency`.

**Crate**

```json
{
  "type": "crate",
  "registry": "crates.io",
  "name": "serde",
  "description": "A generic serialization/deserialization framework",
  "newestVersion": "1.0.229",
  "downloads": 512345678,
  "recentDownloads": 41234567,
  "versionCount": 291,
  "license": "MIT OR Apache-2.0",
  "rustVersion": "1.61",
  "edition": "2018",
  "repository": "https://github.com/serde-rs/serde",
  "url": "https://crates.io/crates/serde"
}
```

**Reverse dependency**

```json
{
  "type": "reverseDependency",
  "name": "serde",
  "dependentCrate": "hashbrown",
  "dependentVersion": "0.17.1",
  "dependentDownloads": 98123456,
  "requirement": "^1.0",
  "dependencyKind": "normal",
  "isOptional": true,
  "url": "https://crates.io/crates/hashbrown"
}
```

### How daily downloads are counted

crates.io splits a day's downloads between the few versions it itemises and a
remainder covering every other version. A day's total here is the sum of both, so
it matches the figure on the crate's page. Turn on **Break downloads down by
version** to get a row per version instead; the remainder arrives as one extra row
per day labelled `other versions`, flagged with `isRemainder`.

### Pricing

Pay only for results:

| Result | Price |
|---|---|
| Crate (details or search hit) | $5.00 per 1,000 |
| Version, owner, download day or reverse dependency | $1.00 per 1,000 |

Set a maximum spend per run in the run options and the Actor stops cleanly when it is reached.

### Limits to know about

These come from crates.io, not from this Actor:

- **crates.io rate-limits hard.** Keep *Max parallel requests* at 2 unless you see no throttling in the log. A throttled run retries with backoff rather than failing, but it takes longer.
- **Daily downloads cover roughly the last 90 days.** crates.io does not serve a longer public series, and per-version rows exist only for versions still being downloaded. Older traffic is folded into an "extra downloads" total with no version attached.
- **Reverse dependencies are paged 100 at a time** and widely used crates have tens of thousands, so set *Max reverse dependencies* to what you actually need.
- **`license`, `rustVersion`, `edition` and `crateSizeBytes` describe the newest version**, not the crate as a whole. They can differ between versions; use Version history mode for the full picture.
- Only published crates are visible. There is no private registry access here.

### FAQ

**Do I need a crates.io account or token?** No. Everything here is public.

**Why is my search returning fewer results than crates.io shows?** Set *Max results per search query* higher; it defaults to 50.

**Can I track a crate daily?** Yes. Create a schedule in Apify Console and connect the dataset to Slack, Google Sheets, Zapier or Make.

**What does `dependencyKind` mean?** `normal`, `dev` or `build`, matching Cargo's dependency sections.

### Feedback

Found a bug or need a field that is missing? Open an issue on the Actor's **Issues** tab.

# Actor input Schema

## `mode` (type: `string`):

Crate metadata, version history, a daily download series, crate owners, the crates that depend on it, or a registry search.

## `crates` (type: `array`):

Crate names (serde, tokio) or crates.io and docs.rs links.

## `searchQueries` (type: `array`):

For Registry search, for example: http client

## `searchSort` (type: `string`):

How crates.io orders the search results. Relevance is the site's default; the others are useful for finding what is popular or freshly released.

## `maxResultsPerSearch` (type: `integer`):

For Search. How many crates to return per query. Each 100 is one request.

## `maxVersionsPerCrate` (type: `integer`):

For Versions. How many versions to return per crate, newest first. A crate's whole history comes in one request, so this only caps the rows saved.

## `includeYankedVersions` (type: `boolean`):

Yanked versions are kept by default so you can audit them.

## `downloadsPerVersion` (type: `boolean`):

Off by default, which gives one total per day. On, you get a row per version per day and one extra request per crate.

## `maxReverseDependencies` (type: `integer`):

Widely used crates have tens of thousands of dependents, so this is capped.

## `maxItems` (type: `integer`):

Stop the run after saving this many items. Leave empty for no limit.

## `maxConcurrency` (type: `integer`):

crates.io rate-limits aggressively. Two is a safe default; raising it risks throttling.

## `proxyConfiguration` (type: `object`):

Only needed for very large runs that trip crates.io rate limits.

## Actor input object example

```json
{
  "mode": "crateDetails",
  "crates": [
    "serde"
  ],
  "searchQueries": [
    "http client"
  ],
  "searchSort": "relevance",
  "maxResultsPerSearch": 50,
  "maxVersionsPerCrate": 100,
  "includeYankedVersions": true,
  "downloadsPerVersion": false,
  "maxReverseDependencies": 100,
  "maxConcurrency": 2,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "crates": [
        "serde"
    ],
    "searchQueries": [
        "http client"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("winterlock/crates-io-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "crates": ["serde"],
    "searchQueries": ["http client"],
}

# Run the Actor and wait for it to finish
run = client.actor("winterlock/crates-io-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "crates": [
    "serde"
  ],
  "searchQueries": [
    "http client"
  ]
}' |
apify call winterlock/crates-io-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,winterlock/crates-io-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ufgC98f2nMxexIqPs/builds/PG45zm38loiXoIZOG/openapi.json