# Crates.io Scraper - Rust Package Metadata, Versions & Downloads (`klondikeking/cratesio-scraper`) Actor Extract Rust crate metadata from crates.io: versions, downloads, keywords, categories, and repository links. Ideal for dependency auditing and Rust ecosystem research. - **URL**: https://apify.com/klondikeking/cratesio-scraper.md - **Developed by:** [Pierrick McD0nald](https://apify.com/klondikeking) (community) - **Categories:** Developer tools, AI, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Crates.io Scraper — Rust Package Metadata, Versions & Downloads Extract comprehensive metadata from [crates.io](https://crates.io), the official Rust package registry. This Actor returns crate descriptions, license identifiers, keywords, categories, total downloads, recent download counts, version lists, and repository links. Whether you are auditing dependencies, tracking popular crates, or building a Rust ecosystem dataset, this tool delivers structured data without writing a single line of Rust. ### Use Cases - **Dependency auditing** — Pull metadata, license, and version history for crates in your supply chain. - **Market research** — Compare download counts and recent activity across popular Rust packages. - **Catalog generation** — Build searchable indexes of Rust crates with keywords, categories, and documentation links. - **Competitive analysis** — Track release cadence and adoption trends for competing libraries. - **Open-source intelligence** — Map repository links, documentation URLs, and maintainer ecosystems. ### Input | Field | Type | Required | Description | |-------|------|----------|-------------| | `crateNames` | Array | No | List of crate names to scrape (e.g., `serde`, `tokio`, `axum`). Leave empty to search by query or use defaults. | | `searchQuery` | String | No | Search term for discovering crates. Used when `crateNames` is empty. | | `maxItems` | Number | No | Maximum number of crates to extract (default: 10). | | `includeDownloads` | Boolean | No | Fetch per-version download statistics for each crate (default: true). | | `proxyConfiguration` | Object | No | Proxy configuration. Apify proxy is included by default. | ### Output The Actor outputs a dataset with one row per crate: ```json { "crateName": "serde", "description": "A generic serialization/deserialization framework", "homepage": "https://serde.rs", "documentation": "https://docs.rs/serde", "repository": "https://github.com/serde-rs/serde", "license": "MIT OR Apache-2.0", "keywords": "serialization, deserialization, json", "categories": "encoding, data-structures", "downloads": 125000000, "recentDownloads": 45000000, "versionsCount": 180, "latestVersion": "1.0.228", "latestVersionDownloads": 120000000, "createdAt": "2015-04-23T15:23:49Z", "updatedAt": "2025-09-27T16:51:35Z", "sourceUrl": "https://crates.io/crates/serde", "error": "" } ```` ### Pricing Pay per event: **$0.001 per crate extracted**. Charges are applied after each crate is successfully processed. If your spending limit is reached, the Actor stops gracefully without overcharging. ### Limitations - The Actor uses the public crates.io API, which may rate-limit aggressive usage. - Download statistics are aggregated per-version and may not reflect real-time counters. - Private crates or yanked-only crates may return limited metadata. - Search results are capped by the crates.io pagination limit (100 per request). ### FAQ **Q: Can I scrape a specific list of crates?** A: Yes. Pass the crate names in `crateNames` and set `maxItems` to the desired count. **Q: Do I need a crates.io account or API token?** A: No. The Actor uses the public crates.io API with a standard User-Agent header. **Q: What happens if a crate is not found?** A: The Actor logs the error and continues processing the remaining crates. The output row will contain the crate name and an `error` field. ### Changelog - **v1.0.0** — Initial release with crate search, metadata extraction, and download statistics. *** Built for developers who need reliable Rust ecosystem data. Open an issue on this Actor's Apify page if you need support. # Actor input Schema ## `crateNames` (type: `array`): List of Rust crate names to scrape (e.g., serde, tokio, axum). Leave empty to search by query. ## `searchQuery` (type: `string`): Search crates.io for crates matching this query. Used when crateNames is empty. ## `maxItems` (type: `integer`): Maximum number of crates to extract when searching or using defaults. ## `includeDownloads` (type: `boolean`): If enabled, fetches per-version download statistics for each crate. ## `proxyConfiguration` (type: `object`): Configure proxy for scraping. Apify proxy is included by default. ## Actor input object example ```json { "crateNames": [ "serde", "tokio", "axum" ], "searchQuery": "web framework", "maxItems": 3, "includeDownloads": true, "proxyConfiguration": { "useApifyProxy": true } } ``` # Actor output Schema ## `results` (type: `string`): No description ## `stats` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "crateNames": [ "serde", "tokio", "axum" ], "searchQuery": "web framework", "maxItems": 3, "includeDownloads": true, "proxyConfiguration": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("klondikeking/cratesio-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "crateNames": [ "serde", "tokio", "axum", ], "searchQuery": "web framework", "maxItems": 3, "includeDownloads": True, "proxyConfiguration": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("klondikeking/cratesio-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "crateNames": [ "serde", "tokio", "axum" ], "searchQuery": "web framework", "maxItems": 3, "includeDownloads": true, "proxyConfiguration": { "useApifyProxy": true } }' | apify call klondikeking/cratesio-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=klondikeking/cratesio-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Crates.io Scraper - Rust Package Metadata, Versions & Downloads", "description": "Extract Rust crate metadata from crates.io: versions, downloads, keywords, categories, and repository links. Ideal for dependency auditing and Rust ecosystem research.", "version": "1.0", "x-build-id": "bja2VxOjfvedlu1My" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/klondikeking~cratesio-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-klondikeking-cratesio-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/klondikeking~cratesio-scraper/runs": { "post": { "operationId": "runs-sync-klondikeking-cratesio-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/klondikeking~cratesio-scraper/run-sync": { "post": { "operationId": "run-sync-klondikeking-cratesio-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "crateNames", "searchQuery", "maxItems", "includeDownloads" ], "properties": { "crateNames": { "title": "Crate Names", "type": "array", "description": "List of Rust crate names to scrape (e.g., serde, tokio, axum). Leave empty to search by query.", "default": [], "items": { "type": "string" } }, "searchQuery": { "title": "Search Query", "type": "string", "description": "Search crates.io for crates matching this query. Used when crateNames is empty.", "default": "" }, "maxItems": { "title": "Max Items", "type": "integer", "description": "Maximum number of crates to extract when searching or using defaults.", "default": 10 }, "includeDownloads": { "title": "Include Download Stats", "type": "boolean", "description": "If enabled, fetches per-version download statistics for each crate.", "default": true }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "Configure proxy for scraping. Apify proxy is included by default." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```