# NPM Registry Scraper (`crawlerbros/npm-registry-scraper`) Actor Scrape NPM package metadata, version history, maintainers, dependents, and download stats from the public NPM registry. Search packages or pull a specific list of package names. - **URL**: https://apify.com/crawlerbros/npm-registry-scraper.md - **Developed by:** [Crawler Bros](https://apify.com/crawlerbros) (community) - **Categories:** Developer tools, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 10 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing from $3.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## NPM Registry Scraper Scrape the public NPM package registry — search packages or look up by exact name. Extract package metadata (name, latest version, description, license, keywords, author, maintainers, repository), score signals (quality / popularity / maintenance), and weekly download counts. HTTP-only via `registry.npmjs.org` and `api.npmjs.org`. No auth, no proxy, no rate-limit drama. ### What this actor does - **Two modes:** `search` (free-text package search) and `lookup` (exact package names) - **Rich metadata:** title, latest version, dist-tags, description, keywords, license, homepage, repository, bugs, maintainers, author, created/modified timestamps - **NPM search score signals:** final score, quality, popularity, maintenance, plus the per-query searchScore - **Download stats:** weekly / monthly / daily download counts via `api.npmjs.org/downloads` - **Filters:** min weekly downloads, keyword tagged, SPDX license - **Optional version history:** pass `includeVersions: true` to emit the full sorted list of published versions - **Empty fields are omitted** — no nulls or blank strings reach the dataset ### Output per package - `name`, `latestVersion`, `distTags` (when more than just `latest` exists) - `description`, `keywords[]` - `license` (SPDX where available; falls back to license.type / license.name dict shape) - `homepageUrl`, `repositoryUrl`, `bugsUrl`, `npmUrl` - `author` — `{name, email, url}` (handles both string and dict shapes) - `maintainers[]` — array of `{name, email, url}` - `createdAt`, `modifiedAt` — ISO 8601 timestamps - `downloadsLastWeek` (or `downloads_day` / `downloads_month` per `downloadsPeriod`) - `scoreFinal`, `score_quality`, `score_popularity`, `score_maintenance`, `searchScore` (search mode only) - `versions[]` — when `includeVersions=true` - `recordType: "package"`, `scrapedAt` ### Input | Field | Type | Default | Description | |---|---|---|---| | `mode` | string | `search` | `search` or `lookup` | | `searchQuery` | string | `react` | Free-text NPM search (mode=search) | | `packageNames` | array | – | Exact package names (mode=lookup); supports scoped like `@types/node` | | `downloadsPeriod` | string | `last-week` | `last-day` / `last-week` / `last-month` | | `minDownloadsLastWeek` | int | – | Drop packages below this weekly download count | | `keywordAnyOf` | array | `[]` | Only emit packages tagged with at least one of these keywords | | `licenseAnyOf` | array | `[]` | Only emit packages whose SPDX license matches | | `includeVersions` | bool | `false` | Emit full version history | | `includeMaintainers` | bool | `true` | Include maintainer list | | `includeDownloads` | bool | `true` | Fetch download counts | | `maxItems` | int | `50` | Hard cap (1–5000) | #### Example: search for top React packages ```json { "mode": "search", "searchQuery": "react", "minDownloadsLastWeek": 10000, "licenseAnyOf": ["MIT"], "maxItems": 50 } ```` #### Example: lookup specific packages ```json { "mode": "lookup", "packageNames": ["express", "lodash", "@types/node"], "includeVersions": true } ``` #### Example: TypeScript ecosystem audit ```json { "mode": "search", "searchQuery": "typescript types", "keywordAnyOf": ["typescript"], "maxItems": 200 } ``` ### Use cases - **Open-source intelligence** — track package adoption, license distribution, ecosystem trends - **Security teams** — audit dependencies, track maintainer churn, detect license changes - **DevRel & growth** — find similar / competing packages, monitor share of voice - **VC due diligence** — adoption telemetry on open-source startups (downloads, maintainers, repo activity) - **Build tools** — automate npm-trends-style dashboards - **License compliance** — bulk-fetch SPDX identifiers across an entire dependency tree ### FAQ **Does it require an NPM account?** No. The NPM registry is fully public. **Is there a rate limit?** NPM is generous on the public registry (no documented hard limit for normal scraping). The actor inserts small delays between requests to stay polite. **What's the difference between `latestVersion` and `distTags`?** `latestVersion` is the `dist-tags.latest` value (the "stable" release). `distTags` includes any other tags like `next`, `beta`, `legacy`. **Why are some fields empty?** The actor omits empty fields rather than emit nulls. For example, `bugsUrl` is only present if the package's `package.json` lists one. **Can I get download history (time series)?** Not in v1 — we emit a single point-in-time count for the chosen period. For time-series download data, query `api.npmjs.org/downloads/range/...` separately. **How do scoped packages work?** Use the full scoped name (e.g. `@types/node`) in `packageNames`. The actor URL-encodes correctly. **Are tarball / unpacked size included?** Not in v1. The registry payload exposes them per-version under `versions[].dist`. We can add a `latestTarball` field if needed. **What's the difference between `score.final` and `searchScore`?** `score.final` is NPM's general quality score (0–1) computed from quality, popularity, and maintenance. `searchScore` is the relevance score for your specific search query. # Actor input Schema ## `mode` (type: `string`): `search` queries the NPM search API; `lookup` fetches metadata for the exact package names you provide. ## `searchQuery` (type: `string`): Free-text query passed to the NPM search API. Supports keywords like `react`, `cli`, scope queries like `@types/`, and qualifiers like `keywords:web framework`. ## `packageNames` (type: `array`): Exact NPM package names. Scoped packages like `@types/node` are supported. ## `scope` (type: `string`): NPM organization / scope name. Examples: `types`, `babel`, `aws-sdk`, `vue`, `nestjs`. Leading `@` is optional. Lists packages whose name starts with `@/`. ## `downloadsPeriod` (type: `string`): Time period for download counts (NPM downloads API). ## `minDownloadsLastWeek` (type: `integer`): Drop packages with fewer downloads last week than this. ## `keywordAnyOf` (type: `array`): Only emit packages tagged with at least one of these keywords. ## `licenseAnyOf` (type: `array`): Only emit packages whose license matches one of these SPDX identifiers (e.g. `MIT`, `Apache-2.0`). ## `includeVersions` (type: `boolean`): Emit the full list of published versions per package. ## `includeMaintainers` (type: `boolean`): Emit the maintainers array (name + email). ## `includeDownloads` (type: `boolean`): Fetch download counts from `api.npmjs.org/downloads`. ## `maxItems` (type: `integer`): Hard cap on emitted records. ## Actor input object example ```json { "mode": "search", "searchQuery": "react", "packageNames": [ "express", "lodash" ], "downloadsPeriod": "last-week", "keywordAnyOf": [], "licenseAnyOf": [], "includeVersions": false, "includeMaintainers": true, "includeDownloads": true, "maxItems": 50 } ``` # Actor output Schema ## `packages` (type: `string`): Dataset containing all scraped NPM packages. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "search", "searchQuery": "react", "packageNames": [ "express", "lodash" ], "downloadsPeriod": "last-week", "keywordAnyOf": [], "licenseAnyOf": [], "includeVersions": false, "includeMaintainers": true, "includeDownloads": true, "maxItems": 50 }; // Run the Actor and wait for it to finish const run = await client.actor("crawlerbros/npm-registry-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "search", "searchQuery": "react", "packageNames": [ "express", "lodash", ], "downloadsPeriod": "last-week", "keywordAnyOf": [], "licenseAnyOf": [], "includeVersions": False, "includeMaintainers": True, "includeDownloads": True, "maxItems": 50, } # Run the Actor and wait for it to finish run = client.actor("crawlerbros/npm-registry-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "search", "searchQuery": "react", "packageNames": [ "express", "lodash" ], "downloadsPeriod": "last-week", "keywordAnyOf": [], "licenseAnyOf": [], "includeVersions": false, "includeMaintainers": true, "includeDownloads": true, "maxItems": 50 }' | apify call crawlerbros/npm-registry-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=crawlerbros/npm-registry-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "NPM Registry Scraper", "description": "Scrape NPM package metadata, version history, maintainers, dependents, and download stats from the public NPM registry. Search packages or pull a specific list of package names.", "version": "1.0", "x-build-id": "WlerrBehb3akKDoCD" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/crawlerbros~npm-registry-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-crawlerbros-npm-registry-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/crawlerbros~npm-registry-scraper/runs": { "post": { "operationId": "runs-sync-crawlerbros-npm-registry-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/crawlerbros~npm-registry-scraper/run-sync": { "post": { "operationId": "run-sync-crawlerbros-npm-registry-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "mode" ], "properties": { "mode": { "title": "Mode", "enum": [ "search", "lookup", "byScope" ], "type": "string", "description": "`search` queries the NPM search API; `lookup` fetches metadata for the exact package names you provide.", "default": "search" }, "searchQuery": { "title": "Search query (mode=search)", "type": "string", "description": "Free-text query passed to the NPM search API. Supports keywords like `react`, `cli`, scope queries like `@types/`, and qualifiers like `keywords:web framework`.", "default": "react" }, "packageNames": { "title": "Package names (mode=lookup)", "type": "array", "description": "Exact NPM package names. Scoped packages like `@types/node` are supported.", "default": [], "items": { "type": "string" } }, "scope": { "title": "Scope (mode=byScope)", "type": "string", "description": "NPM organization / scope name. Examples: `types`, `babel`, `aws-sdk`, `vue`, `nestjs`. Leading `@` is optional. Lists packages whose name starts with `@/`." }, "downloadsPeriod": { "title": "Downloads period", "enum": [ "last-day", "last-week", "last-month" ], "type": "string", "description": "Time period for download counts (NPM downloads API).", "default": "last-week" }, "minDownloadsLastWeek": { "title": "Min downloads (last week)", "minimum": 0, "maximum": 1000000000, "type": "integer", "description": "Drop packages with fewer downloads last week than this." }, "keywordAnyOf": { "title": "Keyword filter", "type": "array", "description": "Only emit packages tagged with at least one of these keywords.", "default": [], "items": { "type": "string" } }, "licenseAnyOf": { "title": "License filter (SPDX)", "type": "array", "description": "Only emit packages whose license matches one of these SPDX identifiers (e.g. `MIT`, `Apache-2.0`).", "default": [], "items": { "type": "string" } }, "includeVersions": { "title": "Include version history", "type": "boolean", "description": "Emit the full list of published versions per package.", "default": false }, "includeMaintainers": { "title": "Include maintainers", "type": "boolean", "description": "Emit the maintainers array (name + email).", "default": true }, "includeDownloads": { "title": "Include download stats", "type": "boolean", "description": "Fetch download counts from `api.npmjs.org/downloads`.", "default": true }, "maxItems": { "title": "Max items", "minimum": 1, "maximum": 5000, "type": "integer", "description": "Hard cap on emitted records.", "default": 50 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```