Crates.io Scraper

Scrape Rust crates from crates.io, the Rust package registry. Search by keyword and get names, download counts, versions, descriptions, and repository links.

What does Crates.io Scraper do?

Crates.io Scraper uses the crates.io public API to search for Rust packages and extract metadata. It collects crate names, descriptions, current versions, total and recent download counts, repository URLs, and documentation links.

Sort results by relevance, downloads, recent activity, or newest crates.

Why scrape Crates.io?

Crates.io is the official package registry for Rust, hosting over 150,000 crates. It's the definitive source for understanding the Rust ecosystem.

Key reasons to scrape it:

• Ecosystem analysis — Map the Rust package landscape for any domain
• Technology research — Find the most popular libraries for specific use cases
• Competitive intelligence — Track download trends for competing crates
• Developer tools — Build dashboards or recommendation engines for Rust developers
• Market research — Understand Rust adoption patterns across industries

Use cases

• Rust developers finding the best libraries for their projects
• Engineering managers evaluating Rust ecosystem maturity
• Technical writers researching popular Rust packages for tutorials
• Open-source maintainers tracking competitor crate adoption
• Researchers studying open-source package ecosystems
• Investors analyzing Rust ecosystem growth trends

How to scrape Crates.io

1. Go to Crates.io Scraper on Apify Store
2. Enter one or more search keywords
3. Choose sort order (relevance, downloads, etc.)
4. Set result limits
5. Click Start and wait for results
6. Download data as JSON, CSV, or Excel

Input parameters

Parameter	Type	Default	Description
searchQueries	string[]	(required)	Keywords to search for
sortBy	string	"relevance"	Sort: relevance, downloads, recent-downloads, recent-updates, new
maxResultsPerSearch	integer	100	Max crates per keyword
maxPages	integer	5	Max pages (50 crates/page)

Input example

{
    "searchQueries": ["web framework", "async runtime"],
    "sortBy": "downloads",
    "maxResultsPerSearch": 50
}

Output

Each crate in the dataset contains:

Field	Type	Description
name	string	Crate name
description	string	Crate description
version	string	Latest stable version
downloads	number	Total all-time downloads
recentDownloads	number	Downloads in the last 90 days
createdAt	string	First published date
updatedAt	string	Last updated date
repository	string	Source code repository URL
homepage	string	Homepage URL
documentation	string	Documentation URL
cratesUrl	string	Crates.io page URL
scrapedAt	string	ISO timestamp of extraction

Output example

{
    "name": "serde",
    "description": "A generic serialization/deserialization framework",
    "version": "1.0.217",
    "downloads": 403158920,
    "recentDownloads": 27365812,
    "createdAt": "2015-01-28T22:03:53.242031Z",
    "updatedAt": "2025-01-07T20:45:16.399655Z",
    "repository": "https://github.com/serde-rs/serde",
    "homepage": "https://serde.rs",
    "documentation": "https://docs.rs/serde",
    "cratesUrl": "https://crates.io/crates/serde",
    "scrapedAt": "2026-03-03T03:09:32.123Z"
}

How much does it cost to scrape Crates.io?

Crates.io Scraper uses pay-per-event pricing:

Event	Price
Run started	$0.001
Crate extracted	$0.001 per crate

Cost examples

Scenario	Crates	Cost
Quick search	50	$0.051
Ecosystem survey	200	$0.201
Large analysis	500	$0.501

Platform costs are negligible — typically under $0.001 per run.

Using Crates.io Scraper with the Apify API

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('automation-lab/crates-scraper').call({
    searchQueries: ['web framework'],
    sortBy: 'downloads',
    maxResultsPerSearch: 50,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Found ${items.length} crates`);
items.forEach(crate => {
    console.log(`${crate.name} v${crate.version} (${crate.downloads.toLocaleString()} downloads)`);
});

Python

from apify_client import ApifyClient

client = ApifyClient('YOUR_API_TOKEN')

run = client.actor('automation-lab/crates-scraper').call(run_input={
    'searchQueries': ['web framework'],
    'sortBy': 'downloads',
    'maxResultsPerSearch': 50,
})

dataset = client.dataset(run['defaultDatasetId']).list_items().items
print(f'Found {len(dataset)} crates')
for crate in dataset:
    print(f"{crate['name']} v{crate['version']} ({crate['downloads']:,} downloads)")

Integrations

Crates.io Scraper works with all Apify integrations:

• Scheduled runs — Track crate popularity trends over time
• Webhooks — Get notified when a scrape completes
• API — Trigger runs and fetch results programmatically
• Google Sheets — Export crate data to a spreadsheet
• Slack — Share popular crate updates with your team

Connect to Zapier, Make, or Google Sheets for automated workflows.

Tips

• Sort by downloads to find the most established crates in a category
• Sort by recent-downloads to find crates gaining momentum right now
• Sort by new to discover recently published crates
• Compare downloads vs recentDownloads to assess growth trends
• Use repository links to check activity, issues, and community engagement
• Multiple keywords let you survey different aspects of the ecosystem in one run

FAQ

How many crates can I get? Each page returns up to 50 crates. With pagination, you can fetch hundreds per keyword.

Does it include dependency information? No — the search API returns metadata only. For dependency trees, you'd need to query individual crate version endpoints.

Can I filter by category? The API search is keyword-based. Use specific terms like "web framework" or "database driver" to find crates in a category.

How often is crates.io data updated? Download counts and version info are updated in real time as crates are published and downloaded.

Use Crates.io Scraper with Claude AI (MCP)

You can integrate Crates.io Scraper as a tool in Claude AI or any MCP-compatible client. This lets you ask Claude to fetch crates.io data in natural language.

Setup

CLI:

$claude mcp add crates-scraper -- npx -y @anthropic-ai/apify-mcp-server@latest --actors=automation-lab/crates-scraper

JSON config (Claude Desktop, Cline, etc.):

{
  "mcpServers": {
    "crates-scraper": {
      "command": "npx",
      "args": ["-y", "@anthropic-ai/apify-mcp-server@latest", "--actors=automation-lab/crates-scraper"]
    }
  }
}

Set your APIFY_TOKEN as an environment variable or pass it via --token.

Example prompts

• "Search crates.io for async HTTP libraries"
• "Get stats for these Rust crates"
• "Find the most downloaded Rust web framework crates"

cURL

curl "https://api.apify.com/v2/acts/automation-lab~crates-scraper/run-sync-get-dataset-items?token=YOUR_API_TOKEN" \
  -X POST -H "Content-Type: application/json" \
  -d '{"searchQueries": ["web framework"], "sortBy": "downloads", "maxResultsPerSearch": 50}'

The scraper returns fewer results than maxResultsPerSearch. Some search keywords return a limited number of matching crates. The crates.io search API returns only relevant matches, not padded results.

Why are recentDownloads much lower than downloads? The downloads field is all-time total, while recentDownloads covers the last 90 days. A large gap is normal for well-established crates with years of download history.

Other package registry scrapers

• npm Scraper -- Scrape package data from the npm registry.
• PyPI Scraper -- Scrape Python package data from PyPI.
• Homebrew Scraper -- Scrape Homebrew formula data.
• Pub.dev Scraper -- Scrape Dart and Flutter packages from pub.dev.
• Docker Hub Scraper -- Scrape Docker image data from Docker Hub.