Crates.io Rust Crates Scraper

Crates.io Rust Crates Scraper

Scrape Rust crate metadata from crates.io by keyword search, direct name lookup, or sorted lists, up to a million per run. Every crate comes with its name, description, version, downloads, and repository link. No API key required. Export to CSV, JSON, Excel, or XML.

Crates.io is the central registry for Rust packages, but browsing it manually or hitting its API for bulk data means writing code and handling pagination. This Actor reads the public web interface directly, so you can pull crate lists by search term, a list of exact names, or pre-sorted feeds like newest or most downloaded. You get a flat dataset of crate metadata ready for analysis, dependency tracking, or market research.

Who uses it	What they scrape Crates.io for
Rust developers	Audit dependencies across a list of crates for licensing or maintenance status.
Security researchers	Monitor newly published or updated crates for supply-chain risks.
Market analysts	Track download trends and category growth in the Rust ecosystem.
Dev tool builders	Populate an internal registry or recommendation engine with fresh crate data.

What it does

This Actor collects Rust crate metadata from crates.io by search, name lookup, or sorted lists, and returns each crate as a flat row.

• 🔎 Search by keyword: free-text query returns ranked results, like the crates.io search box.
• 📋 Direct lookup by name: paste a list of crate names, one per line, and get each one's current metadata.
• 🆕 Newest crates: pull the most recently published crates in order of arrival.
• 📈 Most downloaded: get crates sorted by all-time download count.
• 🔄 Recently updated: collect crates that got a new version release.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with Crates.io data

🔍 Audit a dependency list.

A Rust team pastes their Cargo.toml crate names into lookup mode and gets current version, license, and repository links for every direct dependency before a release.

📈 Track ecosystem trends.

A developer advocate runs the most downloaded mode weekly to see which crates are gaining traction and writes a community update based on the top movers.

🆕 Monitor new crate publications.

A security researcher schedules the newest mode daily and flags any crate with a suspicious name or description for manual review.

🔄 Watch for recent updates.

A platform team runs recently updated mode against their internal allowlist to catch new versions of critical libraries and trigger CI tests.

Why choose this scraper

	What you get
No API key	Reads the public crates.io pages, no registration or token needed.
Flat row output	Every crate is one row with name, description, version, downloads, and repo link.
Five collection modes	Search, lookup, newest, most downloaded, or recently updated.
Up to 1M crates	Set maxItems as high as you need for bulk collection.

How it compares

No other Store actor targets Crates.io the same way, so the honest comparison is with the alternatives teams actually weigh.

	Crates.io Rust Crates Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When Crates.io changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the Actor from a search term, a list of crate names, or a pre-sorted feed, and set a maximum item count so only the top N results reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

{
 "maxItems": 10,
 "mode": "search",
 "query": "tokio",
 "names": "tokio\nserde\nclap\nreqwest\naxum"
}

A larger pull:

{
 "maxItems": 200,
 "mode": "search",
 "query": "tokio",
 "names": "tokio\nserde\nclap\nreqwest\naxum"
}

Pricing

Pay-per-result: $0.016 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$1.60
1,000 results	$16.00
10,000 results	$160.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the Crates.io Rust Crates Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to Crates.io through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/crates-io-rust-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results from search mode?

Check that your search term is not too restrictive. Try a broader keyword that you know returns results on the crates.io website. Also verify that maxItems is set to at least 1.

Why does lookup mode return fewer crates than I pasted?

The Actor only returns crates that exist on crates.io. Check for typos in your crate names. Names are case-sensitive and must match the exact crate ID.

The run is taking a long time with a high maxItems value.

Large collections require many page requests. Reduce maxItems or use a more targeted mode like search with a specific keyword to speed up the run.

Some fields are empty in my output.

Not every crate has a description, repository link, or other optional metadata. Empty fields mean crates.io does not list that information for that crate.

I get an error when using special characters in search.

URL-encode your query or stick to alphanumeric terms. Extremely long or symbol-heavy queries may not parse correctly on the crates.io search endpoint.

FAQ

Question	Answer
Do I need a crates.io API key or token?	No. This Actor reads the public web pages, so no registration or authentication is required.
What data fields does each crate row include?	Each row includes the crate name, description, latest version, total downloads, repository URL, and other metadata visible on the crates.io listing page.
Can I scrape all crates on crates.io?	You can collect up to one million crates per run. Use the newest or most downloaded modes with a high maxItems value to pull a large sample of the registry.
How does the search mode work?	It sends your free-text query to the crates.io search and returns the ranked results, matching the same relevance order you see on the website.
What format does the lookup mode expect for crate names?	Paste names one per line or separated by semicolons. The Actor trims whitespace and looks up each exact name.
Can I filter by category or keyword in search?	The search mode accepts the same free-text queries as the crates.io search box, which indexes names, descriptions, and keywords. Use terms like 'parser' or 'crypto' to narrow results.
Is there a rate limit?	The Actor respects the server response times. For very large runs, it spaces requests to avoid overwhelming crates.io.
Can I export the results?	Yes. You can download your dataset in CSV, JSON, Excel, or XML format from the Apify run page.
Does this Actor get crate source code or README files?	No. It collects the metadata shown on the crates.io listing and search pages. For source code, pair it with a GitHub scraper using the repository URL field.
How do I schedule regular runs?	Use Apify's scheduler to run the Actor daily or weekly with your chosen mode and settings.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Rust Foundation. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.