# Crates.io Scraper — Rust Packages, Downloads & Metadata (`logiover/crates-io-scraper`) Actor

Scrape crates.io by top downloads, search query, or crate name. Extract name, version, downloads, categories, repository, homepage for Rust ecosystem research, dependency auditing, and competitive analysis. No API key, no login.

- **URL**: https://apify.com/logiover/crates-io-scraper.md
- **Developed by:** [Logiover](https://apify.com/logiover) (community)
- **Categories:** Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.50 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Crates.io Scraper — Rust Package Registry Data, Downloads & Metadata

Extract structured data from crates.io, the official Rust package registry, with no API key and no login required.

### What does Crates.io Scraper do?

Crates.io Scraper connects directly to the official `crates.io` public REST API (`https://crates.io/api/v1/`) to extract package metadata at scale. The actor operates in three modes: **list** (paginate the full registry ranked by total downloads, recent downloads, newest, or alphabetically), **search** (full-text query against crate names and descriptions), and **crateDetail** (fetch precise metadata for a list of known crate names).

Each page request returns up to 100 crates; the actor paginates automatically until your `maxResults` limit is reached. Because crates.io's API is fully public and keyless, no credentials are needed. A descriptive `User-Agent` header is sent on every request to comply with crates.io's published API guidelines. Rate limiting is handled gracefully with polite inter-request delays and automatic retry on transient server errors. A single run can collect thousands of crate records in minutes.

### Who is it for?

- **Rust developers** who want to benchmark their crate's download rank against similar packages.
- **Open source analysts** mapping the Rust ecosystem's growth, popular categories, and dependency trends.
- **Security researchers** building Software Composition Analysis (SCA) databases of known Rust dependencies.
- **Data engineers** building internal package registries or governance dashboards that require structured crate metadata.
- **Competitive intelligence teams** tracking which Rust libraries dominate specific domains (async, web, CLI, crypto, etc.).

### Use cases

- Compile a ranked list of the top 1,000 Rust crates by total downloads for ecosystem research.
- Search for all crates matching "tokio" to find async runtime alternatives and related packages.
- Pull version history and repository links for a curated list of internal dependencies for auditing.
- Track weekly download velocity of competitor crates using `sort=recent-downloads` in a scheduled run.
- Feed crate metadata into an AI agent to generate dependency health reports or README summaries.

### Why use Crates.io Scraper?

- **Completely keyless** — no API key, no account, no OAuth. Just run and get data.
- **13 fields per crate** — name, description, downloads, recent downloads, versions, homepage, repository, docs URL, categories, keywords, timestamps.
- **Bulk pagination** — collect hundreds to thousands of crates in a single run.
- **Three modes** — list ranked crates, keyword search, or targeted detail fetch by crate name.
- **Export to CSV, JSON, Excel** — all standard Apify dataset formats out of the box.
- **Pay-per-result** — the actor's monetization is based on output volume, so you only pay for what you extract.

### What data can you extract?

The actor extracts all publicly available metadata from the crates.io API for each crate.

| Field | Type | Description |
|---|---|---|
| `name` | string | The crate's unique registry name (e.g. `serde`) |
| `description` | string | Author-provided short description |
| `downloads` | number | All-time total download count |
| `recentDownloads` | number | Downloads in the last 90 days |
| `maxVersion` | string | Highest published semver version |
| `newestVersion` | string | Most recently published version |
| `homepage` | string | Project homepage URL (may be null) |
| `repository` | string | Source code repository URL (GitHub, GitLab, etc.) |
| `documentation` | string | Docs.rs or custom documentation URL |
| `categories` | string | Comma-separated category slugs (e.g. `web-programming, http-client`) |
| `keywords` | string | Comma-separated author-defined keywords |
| `createdAt` | string | ISO 8601 timestamp of first publish |
| `updatedAt` | string | ISO 8601 timestamp of most recent update |

#### Example output record

```json
{
  "name": "serde",
  "description": "A generic serialization/deserialization framework",
  "downloads": 980000000,
  "recentDownloads": 42000000,
  "maxVersion": "1.0.197",
  "newestVersion": "1.0.197",
  "homepage": "https://serde.rs",
  "repository": "https://github.com/serde-rs/serde",
  "documentation": "https://docs.rs/serde",
  "categories": "encoding, no-std",
  "keywords": "serde, serialization, no_std",
  "createdAt": "2015-01-28T00:00:00Z",
  "updatedAt": "2024-03-25T15:00:00Z"
}
```

### How to use

#### Option A — List top crates by downloads

Use `mode=list` to paginate the entire crates.io registry ranked by your chosen sort.

1. Open the actor's input tab.
2. Set **Mode** to `list`.
3. Set **Sort** to `downloads` (or `recent-downloads`, `alpha`, `new-crates`, `updated`).
4. Set **Max Results** to the number of crates you want (e.g. 1000).
5. Click **Start**.

```json
{
  "mode": "list",
  "sort": "downloads",
  "maxResults": 1000
}
```

#### Option B — Search crates by keyword

Use `mode=search` to find crates matching a search query, ranked by relevance or downloads.

1. Set **Mode** to `search`.
2. Set **Query** to your search term (e.g. `async http client`).
3. Set **Max Results** to cap the number of results.

```json
{
  "mode": "search",
  "query": "async http client",
  "sort": "downloads",
  "maxResults": 200
}
```

#### Option C — Fetch specific crates by name

Use `mode=crateDetail` when you have a known list of crate names and want precise metadata including resolved category and keyword labels.

```json
{
  "mode": "crateDetail",
  "crateNames": ["serde", "tokio", "reqwest", "axum", "clap"]
}
```

### Input parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `mode` | string | `list` | Operation mode: `list`, `search`, or `crateDetail` |
| `sort` | string | `downloads` | Sort order for list/search: `downloads`, `recent-downloads`, `new-crates`, `updated`, `alpha` |
| `query` | string | — | Search query (required when `mode=search`) |
| `crateNames` | array | — | List of crate names to fetch (required when `mode=crateDetail`) |
| `maxResults` | integer | `500` | Maximum records to return (1–10000) |
| `proxyConfiguration` | object | no proxy | Apify proxy settings (datacenter proxy is sufficient) |

#### Full input JSON

```json
{
  "mode": "list",
  "sort": "downloads",
  "query": "",
  "crateNames": [],
  "maxResults": 500,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

### Output example

```json
{
  "name": "tokio",
  "description": "An event-driven, non-blocking I/O platform for writing asynchronous I/O backed applications.",
  "downloads": 750000000,
  "recentDownloads": 38000000,
  "maxVersion": "1.37.0",
  "newestVersion": "1.37.0",
  "homepage": null,
  "repository": "https://github.com/tokio-rs/tokio",
  "documentation": "https://docs.rs/tokio",
  "categories": "asynchronous, network-programming",
  "keywords": "io, async, non-blocking, futures",
  "createdAt": "2016-08-04T00:00:00Z",
  "updatedAt": "2024-03-01T12:00:00Z"
}
```

### Tips for best results

- **Use `sort=recent-downloads`** when you want the currently trending crates rather than historical cumulative totals.
- **Set `maxResults` to 100–300** for quick test runs before scaling to thousands.
- **Use `mode=crateDetail`** for detailed category and keyword arrays on a specific known list — the detail endpoint returns richer taxonomy data.
- **Combine search + list**: run a search to discover relevant crates, then run crateDetail on the found names for full metadata.
- **Schedule weekly runs** with `sort=recent-downloads` to track download velocity trends over time.
- **Export to Google Sheets** via Apify integration to share data with non-technical teammates.
- **Filter by category in post-processing** — categories are returned as comma-separated strings and easy to filter in spreadsheets or pandas.
- **Watch for null fields** — `homepage` and `documentation` are often null for newer or unmaintained crates; filter these out if you need complete records only.
- **Paginate in chunks** — if you want all 100,000+ crates on the registry, set `maxResults=10000` and run multiple actors with page offset logic, or use the `alpha` sort to cover the full namespace.
- **Use proxy for high-volume runs** — crates.io uses Fastly CDN and rarely blocks, but enabling Apify datacenter proxy improves reliability for very large runs.

### Integrations

**Google Sheets** — Connect the dataset to Google Sheets via the Apify integration to share crate rankings with your team in real time.

**Slack** — Use Apify webhooks to send a Slack message when your scheduled run completes, with a link to the new dataset.

**Zapier / Make** — Trigger downstream workflows when new crates matching your search appear, or when a crate's download count crosses a threshold.

**Webhooks** — Configure an Apify webhook to POST the dataset URL to your internal API on run completion.

**Schedule** — Use Apify Scheduler to run this actor weekly or daily to maintain a fresh snapshot of crates.io rankings for your dashboard.

### API usage

#### cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/logiover~crates-io-scraper/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode":"list","sort":"downloads","maxResults":300}'
```

#### Node.js

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });

const run = await client.actor('logiover/crates-io-scraper').call({
  mode: 'list',
  sort: 'downloads',
  maxResults: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

#### Python

```python
from apify_client import ApifyClient

client = ApifyClient('YOUR_TOKEN')

run = client.actor('logiover/crates-io-scraper').call(run_input={
    'mode': 'list',
    'sort': 'downloads',
    'maxResults': 500,
})

for item in client.dataset(run['defaultDatasetId']).iterate_items():
    print(item['name'], item['downloads'])
```

### Use with AI agents (MCP)

This actor is available as a tool in Apify's Model Context Protocol (MCP) server, letting AI agents call it directly. For example, an agent can be prompted: *"Use the crates-io-scraper to find the top 50 Rust HTTP client crates and summarize their download trends."* The agent will invoke the actor, wait for results, and synthesize the dataset into a report — no human in the loop required.

### FAQ

#### Does this actor require an API key or login?

No. Crates.io's API is fully public and keyless. The actor sends a descriptive `User-Agent` header as required by crates.io's usage guidelines, but no authentication is needed.

#### How many crates does the registry have?

As of 2026, crates.io has over 145,000 published crates. This actor can paginate the full registry by setting `maxResults` to 10,000 or more (run multiple times with different sort orders to cover the full set).

#### What is the difference between `maxVersion` and `newestVersion`?

`maxVersion` is the highest semver version (not counting pre-releases), while `newestVersion` is the most recently published version. For stable crates they are usually identical; for crates with active pre-release cycles (e.g. `1.0.0-beta.1`), they may differ.

#### Why do some fields come back as null?

Fields like `homepage` and `documentation` are optional in crates.io's publishing spec. Many crate authors omit them, especially for small utility crates. `categories` and `keywords` may also be empty if the author skipped that step.

#### What does `recentDownloads` represent?

It is the download count for the last 90 days as reported by crates.io's API. This is a useful proxy for current adoption and trending status.

#### How do I export the data to CSV or Excel?

In the Apify dataset view, click **Export** and choose CSV, XLSX, or JSON. You can also use the Apify API to download the dataset programmatically.

#### How fast is the actor?

The actor fetches 100 crates per API request with a 500 ms delay between requests (to be polite to crates.io). A run of 1,000 crates takes approximately 10 requests × 500 ms = ~5 seconds of wait time plus network latency, typically completing in under 30 seconds.

#### Is there a rate limit on the crates.io API?

Crates.io does not publish a hard rate limit, but their guidelines ask bots to use a descriptive User-Agent and avoid hammering the API. This actor enforces a 500 ms delay between requests and retries on 429 (Too Many Requests) responses automatically.

#### How do I track a specific crate's growth over time?

Use `mode=crateDetail` with the crate name in a scheduled run (e.g. daily). Each run saves the current download count to a dataset, giving you a time series you can chart in Google Sheets or a BI tool.

#### Is this actor legal to use?

Yes. Crates.io is a public registry operated by the Rust Foundation. All data returned by this actor is publicly accessible via crates.io's documented public API. No authentication bypass, scraping of private data, or violation of crates.io's terms of service is involved.

#### What Rust categories can I filter by?

Crates.io uses a curated category taxonomy. Common ones include: `web-programming`, `network-programming`, `asynchronous`, `cryptography`, `data-structures`, `command-line-utilities`, `database`, `encoding`, `algorithms`, `development-tools`. Filter by category string in post-processing after extracting data.

#### Are download counts real-time?

Download counts are updated by crates.io periodically (typically daily). They reflect cumulative downloads served by the crates.io CDN and may lag by up to 24 hours.

### Is it legal?

This actor accesses only publicly available data from crates.io's documented REST API. All requests are made without authentication and do not access any private or user-specific data. The actor complies with crates.io's usage guidelines by identifying itself with a descriptive `User-Agent` string. Use of public package registry data for research, analytics, and tooling is a standard practice in the software industry.

Always review crates.io's [Terms of Service](https://foundation.rust-lang.org/policies/crates-io-crawling/) before large-scale data collection and ensure your use case is consistent with their crawler/bot policies.

### Related scrapers

- **[npm Package Scraper](https://apify.com/logiover)** — Extract package metadata from the npm registry for JavaScript ecosystem research.
- **[GitHub Activity Scraper](https://apify.com/logiover/github-activity-scraper)** — Scrape GitHub repository stats, commits, issues, and contributor data.
- **[CVE Advisory Scraper](https://apify.com/logiover/cve-advisory-scraper)** — Pull CVE security advisories from NVD for dependency vulnerability tracking.
- **[Tech Stack Detector](https://apify.com/logiover/tech-stack-detector)** — Identify which technologies (including Rust crates) power any website.

### 📝 Changelog

#### 2026-08-01

- Completed the August 2026 full health check: verified empty/programmatic default, Console UI default, and two source-informed alternative inputs on Apify.
- Confirmed successful live execution, non-empty structured output, dataset-field/type integrity, and logical sample quality within the 5-minute quality window.

# Actor input Schema

## `mode` (type: `string`):

What to scrape. Leave as default (List) to instantly get the most-downloaded Rust crates — no other input needed. search = query-based; crateDetail = specific crate names.

## `sort` (type: `string`):

Ordering for List and Search modes.

## `query` (type: `string`):

Search text (used when mode = Search). Leave blank in other modes; Search with no query falls back to the top crates.

## `crateNames` (type: `array`):

List of specific crate names to fetch detail for (used when mode = Crate Detail). Leave empty in other modes; Crate Detail with no names falls back to the top crates.

## `maxResults` (type: `integer`):

Maximum number of crates to return. Default 500, max ~10000 for List/Search.

## `proxyConfiguration` (type: `object`):

Apify proxy settings. crates.io works best over a direct connection (default). Enable Apify proxy only if you need it; automatic proxy selection is used when enabled.

## Actor input object example

```json
{
  "mode": "list",
  "sort": "downloads",
  "maxResults": 500,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

All records extracted by this run. Open the Dataset tab to browse, filter, and export as CSV, JSON, or Excel.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("logiover/crates-io-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "proxyConfiguration": { "useApifyProxy": False } }

# Run the Actor and wait for it to finish
run = client.actor("logiover/crates-io-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call logiover/crates-io-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=logiover/crates-io-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/zCNyh4VdbnOxrYQoP/builds/fTAGboex7c3kQIsTk/openapi.json