# NuGet Scraper (`automation-lab/nuget-scraper`) Actor Scrape NuGet packages — search by keyword or extract metadata for specific package IDs including version, downloads, authors, tags, and license. - **URL**: https://apify.com/automation-lab/nuget-scraper.md - **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community) - **Categories:** Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## NuGet Scraper Scrape packages from the [NuGet Gallery](https://www.nuget.org), Microsoft's official .NET package registry. Search by keyword or look up specific packages by ID to extract version info, download counts, authors, tags, license URLs, and project links — all via the public NuGet REST API, no browser needed. ### What does NuGet Scraper do? NuGet Scraper uses the NuGet v3 public API to extract package metadata from the world's official .NET package registry. Two modes are supported: - **Search mode** — enter a keyword and get ranked results with full metadata, including the most popular matching packages - **Package lookup mode** — provide a list of exact package IDs and get the latest version details for each All data is fetched from NuGet's official public API. No authentication required, no browser overhead. ### Who is it for? - 🔵 **.NET developers** — discover the best NuGet packages for their projects - 📊 **Technology analysts** — research .NET ecosystem trends and library adoption - 🏢 **Engineering managers** — evaluate package health, maintenance status, and download momentum - 🔒 **Security teams** — audit package authorship, verification status, and license types - 📝 **Technical writers** — build comparison guides and package surveys - 🤖 **Developers building tools** — power package recommendation engines or dependency dashboards ### Why scrape NuGet? NuGet hosts over 350,000 unique packages and serves billions of downloads per month. It is the canonical source for .NET library ecosystem data. Key reasons to use this scraper: - **Ecosystem analysis** — understand which packages dominate any .NET technology area - **Technology benchmarking** — compare download counts and maintenance activity across competing libraries - **Competitive intelligence** — monitor download growth for packages competing in the same space - **Dependency auditing** — bulk-check license types, verification status, and project URLs - **Developer tooling** — feed package data into recommendation systems, dashboards, or IDE plugins ### What data does it extract? Each output item contains: | Field | Description | |-------|-------------| | `id` | Package ID (e.g., `Newtonsoft.Json`) | | `version` | Latest published version | | `description` | Full package description | | `summary` | Short summary (if provided by author) | | `authors` | List of author names | | `owners` | List of NuGet gallery owners | | `totalDownloads` | All-time total download count | | `verified` | Whether the package is NuGet-verified | | `tags` | List of keyword tags | | `licenseUrl` | URL to license page | | `projectUrl` | Link to project or GitHub repository | | `iconUrl` | URL to package icon image | | `packageUrl` | Direct link to the package on nuget.org | | `registrationUrl` | NuGet registration API URL for this package | ### How much does it cost to scrape NuGet packages? NuGet Scraper pricing uses Pay-Per-Event billing — you pay only for what you extract: | Event | Price | |-------|-------| | Run started | $0.005 (one-time per run) | | Per package extracted | $0.001 | **Example costs:** - 50 packages → ~$0.055 - 200 packages → ~$0.205 - 1,000 packages → ~$1.005 No proxy costs — the NuGet public API is freely accessible over plain HTTPS. ### How to use NuGet Scraper #### Search mode (default) 1. Open [NuGet Scraper](https://apify.com/automation-lab/nuget-scraper) on Apify Store 2. Set **Mode** to `Search by keyword` 3. Enter a **Search query** (e.g., `dependency injection`) 4. Set **Max results** (up to 1,000) 5. Optionally enable **Include prerelease** to include alpha/beta versions 6. Click **Start** and wait for results 7. Download as JSON, CSV, or Excel #### Package lookup mode 1. Set **Mode** to `Lookup specific packages` 2. Add exact package IDs to the **Package IDs** list (e.g., `Newtonsoft.Json`, `Serilog`) 3. Click **Start** — each package is fetched individually for the latest version data ### Input parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `mode` | string | `search` | `search` to search by keyword, `packages` to look up specific IDs | | `searchQuery` | string | — | Keyword for search mode | | `packageIds` | string[] | — | Exact package IDs for packages mode | | `maxResults` | integer | 50 | Max packages to return (search mode, up to 1,000) | | `includePrerelease` | boolean | false | Include prerelease versions in search results | | `maxRequestRetries` | integer | 3 | Retry attempts per failed request | ### Output format Each dataset item is a JSON object: ```json { "id": "Newtonsoft.Json", "version": "13.0.4", "description": "Json.NET is a popular high-performance JSON framework for .NET", "summary": "", "authors": ["James Newton-King"], "owners": ["dotnetfoundation", "jamesnk", "newtonsoft"], "totalDownloads": 8156351910, "verified": true, "tags": ["json"], "licenseUrl": "https://www.nuget.org/packages/Newtonsoft.Json/13.0.4/license", "projectUrl": "https://www.newtonsoft.com/json", "iconUrl": "https://api.nuget.org/v3-flatcontainer/newtonsoft.json/13.0.4/icon", "packageUrl": "https://www.nuget.org/packages/Newtonsoft.Json", "registrationUrl": "https://api.nuget.org/v3/registration5-semver1/newtonsoft.json/index.json" } ```` ### Tips and best practices - **For broad market research**, use search mode with keywords like `logging`, `dependency injection`, or `serialization` - **For specific package audits**, use packages mode with a curated list of IDs — ideal for scanning your own dependency tree - **Sort by downloads** in your downstream analysis to find the most battle-tested packages - **Check `verified`** — NuGet-verified packages have confirmed ownership and are less likely to be typosquatted - **Use `projectUrl`** to find GitHub repositories for deeper analysis (stars, issues, activity) - **Pagination is automatic** — the scraper handles multi-page results transparently, up to your `maxResults` limit ### Integrations NuGet Scraper integrates with the full Apify ecosystem: - **Apify datasets** — results stored automatically, exportable as JSON, CSV, Excel, or XML - **Webhooks** — trigger downstream workflows when a run completes - **Apify Scheduler** — run weekly to track NuGet ecosystem trends over time - **Google Sheets** — push results via Apify's Google Sheets integration - **Make / Zapier** — connect to thousands of apps for no-code automation ### API Usage You can trigger NuGet Scraper programmatically using the Apify API or SDK. **Node.js (apify-client)** ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: 'YOUR_API_TOKEN' }); const run = await client.actor('automation-lab/nuget-scraper').call({ mode: 'search', searchQuery: 'logging', maxResults: 100, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` **Python** ```python from apify_client import ApifyClient client = ApifyClient('YOUR_API_TOKEN') run = client.actor('automation-lab/nuget-scraper').call(run_input={ 'mode': 'search', 'searchQuery': 'logging', 'maxResults': 100, }) for item in client.dataset(run['defaultDatasetId']).iterate_items(): print(item['id'], item['totalDownloads']) ``` **cURL** ```bash curl -X POST \ "https://api.apify.com/v2/acts/automation-lab~nuget-scraper/runs?token=YOUR_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{"mode":"search","searchQuery":"logging","maxResults":20}' ``` See the [Apify API docs](https://docs.apify.com/api/v2) for full reference. ### Use with AI agents (MCP) NuGet Scraper is available as a tool for AI agents via the [Apify MCP server](https://mcp.apify.com). #### Quick setup **Claude Code / CLI:** ```bash claude mcp add --transport http "https://mcp.apify.com?token=YOUR_APIFY_TOKEN&tools=automation-lab/nuget-scraper" ``` **Claude Desktop / Cursor / VS Code** — add to your MCP config JSON: ```json { "mcpServers": { "apify": { "url": "https://mcp.apify.com?token=YOUR_APIFY_TOKEN&tools=automation-lab/nuget-scraper", "transport": "http" } } } ``` Replace `YOUR_APIFY_TOKEN` with your token from [console.apify.com/account/integrations](https://console.apify.com/account/integrations). #### Example prompts - "Find the top 20 .NET JSON serialization packages by download count" - "Look up the latest stable version of Newtonsoft.Json and Serilog and tell me which has more downloads" - "Search for dependency injection packages and export the top 50 results to CSV" ### Legality and terms of use NuGet Scraper uses the official [NuGet public API](https://api.nuget.org/v3/index.json) which is freely available and designed for programmatic access. All data returned is publicly visible on the NuGet Gallery. - No authentication or login bypass is performed - Rate limits are respected via retry-with-backoff - This scraper complies with NuGet's [terms of use](https://www.nuget.org/policies/Terms) - Users are responsible for ensuring their use case complies with applicable laws ### FAQ **Can I scrape all packages on NuGet?** Yes, the NuGet search API supports pagination up to 1,000 results per search. For full catalog access, combine multiple keyword searches. **Does it get prerelease packages?** Optionally. Enable `includePrerelease` to include alpha, beta, and RC versions in search results. **What if a package ID doesn't exist?** In packages mode, missing packages are skipped with a warning. Other packages in the list continue to be processed. **Is this official NuGet data?** Yes — the scraper uses the official `api.nuget.org` REST API endpoints, not HTML scraping. **How current is the data?** The NuGet API reflects real-time registry state. Download counts may lag slightly (updated hourly by Microsoft). **Why is package lookup mode returning a prerelease version?** In packages mode, the scraper walks backward through NuGet's registration index to find the latest stable version when `includePrerelease` is disabled. If the latest version contains a prerelease marker (e.g., `-dev`, `-beta`, `-rc`), the most recent stable release is returned instead. If all versions in the registry are prerelease, the latest version is returned regardless of the setting. **What happens if the NuGet API is temporarily unavailable?** Each request retries automatically up to `maxRequestRetries` times (default 3) with exponential backoff. If all retries fail for a specific package, that package is skipped with a warning in the run log and processing continues. The run still succeeds with partial results. ### Related actors - [npm Scraper](https://apify.com/automation-lab/npm-scraper) — scrape Node.js packages from the npm registry - [PyPI Scraper](https://apify.com/automation-lab/pypi-scraper) — scrape Python packages from PyPI - [Crates.io Scraper](https://apify.com/automation-lab/crates-scraper) — scrape Rust packages from crates.io - [pub.dev Scraper](https://apify.com/automation-lab/pubdev-scraper) — scrape Dart/Flutter packages from pub.dev # Actor input Schema ## `mode` (type: `string`): Choose 'search' to search for packages by keyword, or 'packages' to look up specific package IDs. ## `searchQuery` (type: `string`): Keyword to search NuGet packages for. Used only when mode is 'search'. Example: 'json serializer' ## `packageIds` (type: `array`): List of exact NuGet package IDs to look up. Used only when mode is 'packages'. Example: \['Newtonsoft.Json', 'Serilog'] ## `maxResults` (type: `integer`): Maximum number of packages to return. Applies only to search mode. ## `includePrerelease` (type: `boolean`): When enabled, search results will include prerelease (alpha/beta/rc) package versions. ## `maxRequestRetries` (type: `integer`): Number of retry attempts for failed requests. ## Actor input object example ```json { "mode": "search", "searchQuery": "json serializer", "packageIds": [ "Newtonsoft.Json", "Serilog", "AutoMapper" ], "maxResults": 20, "includePrerelease": false, "maxRequestRetries": 3 } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "search", "searchQuery": "json serializer", "packageIds": [ "Newtonsoft.Json", "Serilog", "AutoMapper" ], "maxResults": 20, "maxRequestRetries": 3 }; // Run the Actor and wait for it to finish const run = await client.actor("automation-lab/nuget-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "search", "searchQuery": "json serializer", "packageIds": [ "Newtonsoft.Json", "Serilog", "AutoMapper", ], "maxResults": 20, "maxRequestRetries": 3, } # Run the Actor and wait for it to finish run = client.actor("automation-lab/nuget-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "search", "searchQuery": "json serializer", "packageIds": [ "Newtonsoft.Json", "Serilog", "AutoMapper" ], "maxResults": 20, "maxRequestRetries": 3 }' | apify call automation-lab/nuget-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=automation-lab/nuget-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "NuGet Scraper", "description": "Scrape NuGet packages — search by keyword or extract metadata for specific package IDs including version, downloads, authors, tags, and license.", "version": "0.1", "x-build-id": "djhkKdKZfoTlxMmpb" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/automation-lab~nuget-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-automation-lab-nuget-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/automation-lab~nuget-scraper/runs": { "post": { "operationId": "runs-sync-automation-lab-nuget-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/automation-lab~nuget-scraper/run-sync": { "post": { "operationId": "run-sync-automation-lab-nuget-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "mode": { "title": "Mode", "enum": [ "search", "packages" ], "type": "string", "description": "Choose 'search' to search for packages by keyword, or 'packages' to look up specific package IDs.", "default": "search" }, "searchQuery": { "title": "Search query", "type": "string", "description": "Keyword to search NuGet packages for. Used only when mode is 'search'. Example: 'json serializer'" }, "packageIds": { "title": "Package IDs", "type": "array", "description": "List of exact NuGet package IDs to look up. Used only when mode is 'packages'. Example: ['Newtonsoft.Json', 'Serilog']", "items": { "type": "string" } }, "maxResults": { "title": "Max results", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Maximum number of packages to return. Applies only to search mode.", "default": 50 }, "includePrerelease": { "title": "Include prerelease", "type": "boolean", "description": "When enabled, search results will include prerelease (alpha/beta/rc) package versions.", "default": false }, "maxRequestRetries": { "title": "Max request retries", "minimum": 1, "maximum": 10, "type": "integer", "description": "Number of retry attempts for failed requests.", "default": 3 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```