# Package Stats API: npm, PyPI, crates.io, GitHub & Docker (`informed_almandine_vzb/github-npm-pypi-docker-package-scraper`) Actor

Get one record per package across five ecosystems, with downloads, versions, license, stars, and maintainer activity, so you can audit dependencies or feed an AI agent. A test run returned 16 packages per ecosystem (80 total).

- **URL**: https://apify.com/informed_almandine_vzb/github-npm-pypi-docker-package-scraper.md
- **Developed by:** [Almandine Data](https://apify.com/informed_almandine_vzb) (community)
- **Categories:** Developer tools, AI, Lead generation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.50 / 1,000 package or repository intelligence record scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## GitHub, npm, PyPI, Crates & Docker Package Intelligence

Scrape and enrich open-source packages and repos across GitHub, npm, PyPI, Rust Crates.io, and Docker Hub with stars, downloads, licenses, and maintainers.

### What it does

Searches and enriches open-source repositories and software packages across GitHub, npm, PyPI, Rust Crates.io, and Docker Hub with GitHub stars/forks, 30-day/90-day download velocity, Docker pull counts, SPDX license compliance, maintainer contacts, repository URLs, and release timestamps in one unified schema. Export clean, deduplicated data as **JSON, CSV, Excel, XML, or RSS**, or connect directly via the **Apify REST API, Python/JavaScript SDKs, CLI, MCP Server, Clay, n8n, Make, or Zapier** — with **no login, cookies, or browser overhead**.

### Use cases

- **DevTool GTM & Developer Relations Lead Generation:** Identify fast-growing open-source packages, maintainers (`authorOrPublisher`, `maintainerEmail`), and GitHub organizations across AI (`mcp`, `langchain`), observability (`opentelemetry`), and data tooling.
- **VC & Private Equity Open-Source Traction Due Diligence:** Benchmark developer adoption by combining GitHub `starsCount`/`forksCount` with real npm 30-day downloads, Crates.io 90-day downloads, and Docker Hub `pull_count` (`downloadsRecent`, `downloadsTotal`).
- **Enterprise AppSec & SBOM License Compliance Auditing:** Verify SPDX `license` declarations, `latestVersion`, `repositoryUrl`, and release recency (`publishedOrPushedAt`) across Python, TypeScript/JavaScript, Rust, and container dependencies.
- **AI Coding Agent & RAG Tool Discovery:** Track newly published MCP servers, LLM SDKs, and developer packages with `onlyNew: true`.

### Why use this scraper?

| Feature | This Actor | Generic Browser / HTML Scrapers |
|---|---|---|
| **Architecture** | Direct structured API / JSON / SSR requests (`curl_cffi`) | Heavy headless browser automation |
| **Speed & reliability** | **2–25 seconds** for 50–200 items | Minutes of high compute burn |
| **Pricing model** | **Pay per result (`$0.00` actor-start fee)** | Monthly rental or high CU overhead |
| **Error handling** | Failed inputs logged in `ERRORS` key-value record and **never charged** | Bills compute time even on failures |

### Input parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `queries` | `array` | `["mcp", "langchain", "opentelemetry", "pydant...` | Keywords or package/repository names to search across GitHub, npm, Rust Crates.io, and Docker Hub. |
| `ecosystems` | `array` | `["github", "npm", "pypi", "crates", "docker"]` | Developer registries to query and normalize ('github', 'npm', 'pypi', 'crates', 'docker'). |
| `pypiPackages` | `array` | `["fastapi", "pydantic", "openai", "anthropic"...` | Exact Python package names on PyPI to enrich via pypi.org/pypi/{pkg}/json. |
| `enrichNpmDownloads` | `boolean` | `true` | Attach 30-day monthly download velocity for npm packages. |
| `onlyNew` | `boolean` | `false` | Deduplicate across runs via Apify Key-Value Store and emit only newly seen packages or releases. |
| `maxItemsPerEcosystem` | `integer` | `16` | Maximum number of normalized package/repository records to emit per enabled ecosystem (default 5 ecosystems x 16 = 80 records). |
| `delaySeconds` | `number` | `0.15` | Polite pause between upstream HTTP requests (seconds). |

#### Parameter details

- **Search keywords or package names** (`queries`, `array`): Keywords or package/repository names to search across GitHub, npm, Rust Crates.io, and Docker Hub.
- **Package registries & developer ecosystems** (`ecosystems`, `array`): Developer registries to query and normalize ('github', 'npm', 'pypi', 'crates', 'docker').
- **PyPI package names to enrich** (`pypiPackages`, `array`): Exact Python package names on PyPI to enrich via pypi.org/pypi/{pkg}/json.
- **Include 30-day npm download counts** (`enrichNpmDownloads`, `boolean`): Attach 30-day monthly download velocity for npm packages.
- **Only new packages / versions** (`onlyNew`, `boolean`): Deduplicate across runs via Apify Key-Value Store and emit only newly seen packages or releases.
- **Max items per ecosystem** (`maxItemsPerEcosystem`, `integer`): Maximum number of normalized package/repository records to emit per enabled ecosystem (default 5 ecosystems x 16 = 80 records).
- **Delay between requests** (`delaySeconds`, `number`): Polite pause between requests (default `0.15`).

#### Example input

```json
{
  "queries": [
    "mcp",
    "langchain",
    "opentelemetry",
    "pydantic"
  ],
  "ecosystems": [
    "github",
    "npm",
    "pypi",
    "crates",
    "docker"
  ],
  "pypiPackages": [
    "fastapi",
    "pydantic",
    "openai",
    "anthropic",
    "langchain",
    "httpx",
    "polars",
    "duckdb",
    "uvicorn",
    "sqlalchemy",
    "pytest",
    "ruff",
    "litellm",
    "tiktoken",
    "celery",
    "ray"
  ],
  "enrichNpmDownloads": true,
  "onlyNew": false,
  "maxItemsPerEcosystem": 16
}
```

### Quickstart: API, Python & Automation (Clay / n8n / Make)

Run directly via the Apify REST API (`POST /v2/acts/informed_almandine_vzb~github-npm-pypi-docker-package-scraper/runs`) or Python SDK:

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("informed_almandine_vzb/github-npm-pypi-docker-package-scraper").call(run_input={"queries": ["mcp", "langchain", "opentelemetry", "pydantic"], "ecosystems": ["github", "npm", "pypi", "crates", "docker"], "pypiPackages": ["fastapi", "pydantic", "openai", "anthropic", "langchain", "httpx", "polars", "duckdb", "uvicorn", "sqlalchemy", "pytest", "ruff", "litellm", "tiktoken", "celery", "ray"], "enrichNpmDownloads": true, "onlyNew": false, "maxItemsPerEcosystem": 16})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)
```

### Output & example output

One dataset row per result. Default table view columns: `ecosystem, packageName, latestVersion, starsCount, downloadsRecent, license, authorOrPublisher, publishedOrPushedAt, repositoryUrl, url`. Full structured JSON is stored in the run's default dataset.

#### Example output item

```json
{
  "ecosystem": "github",
  "packageName": "morluto/rea",
  "latestVersion": "main",
  "description": "Reverse engineer anything with agents, from app behavior down to native binaries.",
  "starsCount": 382870,
  "forksCount": 95792,
  "downloadsRecent": 382870,
  "downloadsTotal": 382870,
  "openIssuesCount": 1062,
  "language": "TypeScript",
  "license": "MIT",
  "authorOrPublisher": "morluto",
  "maintainerEmail": "https://github.com/morluto",
  "keywordsOrTopics": [
    "agent-skills",
    "ai-agents",
    "binary-analysis"
  ],
  "repositoryUrl": "https://github.com/morluto/rea",
  "homepageUrl": "https://rea.tools",
  "publishedOrPushedAt": "2026-10-25T16:19:34Z",
  "url": "https://github.com/morluto/rea",
  "scrapedAt": "2026-10-25T16:20:34+00:00"
}
```

### Pricing (Pay-Per-Result)

This Actor uses transparent **Pay-Per-Event** pricing with a **`$0.00` actor-start fee**:

| Event | What is charged | Price |
|---|---|---|
| `package-result` | Package or repository intelligence record scraped (One normalized open-source package or repository record across GitHub, npm, PyPI, Crates.io, or Docker Hub) | **$0.0015 / result** (**$1.50 / 1,000 results**) |
| `apify-actor-start` | Actor start fee | **$0.00** |

#### Concrete pricing math

- **`package-result` (Package or repository intelligence record scraped):** `1,000 results = $1.50` (`50 results = $0.07`, `10,000 results = $15.00`). Apify's **$5.00/month Free Tier** covers up to **3,333 results/month** at **$0.00** out-of-pocket.
- **Empty runs (`0` results) & skipped duplicates (`onlyNew` / `onlyChanged`):** **$0.00**.

### Data and compliance

100% public developer registry metadata accessed via official GitHub REST API, npm Registry API, PyPI JSON API, Crates.io API v1, and Docker Hub API v2. Requests are rate-limited and routed through Apify Proxy. Per-input errors are recorded in the run's `ERRORS` key-value record and are never charged.

### FAQ

**Do I need an account, login, or API key?**\
No. This Actor queries public endpoints directly and requires zero credentials.

**What happens if I start a run with empty `{}` input via API or CLI?**\
Every input defines identical `prefill` and `default` values in `.actor/input_schema.json`, so an empty `{}` run automatically returns **50+ live records** out of the box.

**How does incremental deduplication (`onlyNew`) work?**\
When `onlyNew: true` is enabled, the Actor stores seen record IDs in a named Apify Key-Value Store (`open-source-package-intel-state`) and emits only newly discovered records on subsequent scheduled runs.

**Can I export to CSV, Excel, Google Sheets, Clay, or n8n?**\
Yes. Every dataset item uses flat, normalized top-level keys (`ecosystem, packageName, latestVersion, starsCount, downloadsRecent, license`, etc.) for one-click CSV/Excel download or native webhook delivery.

# Actor input Schema

## `queries` (type: `array`):

Keywords or package/repository names to search across GitHub, npm, Rust Crates.io, and Docker Hub.

## `ecosystems` (type: `array`):

Developer registries to query and normalize ('github', 'npm', 'pypi', 'crates', 'docker').

## `pypiPackages` (type: `array`):

Exact Python package names on PyPI to enrich via pypi.org/pypi/{pkg}/json.

## `enrichNpmDownloads` (type: `boolean`):

Attach 30-day monthly download velocity for npm packages.

## `onlyNew` (type: `boolean`):

Deduplicate across runs via Apify Key-Value Store and emit only newly seen packages or releases.

## `maxItemsPerEcosystem` (type: `integer`):

Maximum number of normalized package/repository records to emit per enabled ecosystem (default 5 ecosystems x 16 = 80 records).

## `delaySeconds` (type: `number`):

Polite pause between requests.

## `proxyConfiguration` (type: `object`):

Apify Proxy settings.

## Actor input object example

```json
{
  "queries": [
    "mcp",
    "langchain",
    "opentelemetry",
    "pydantic"
  ],
  "ecosystems": [
    "github",
    "npm",
    "pypi",
    "crates",
    "docker"
  ],
  "pypiPackages": [
    "fastapi",
    "pydantic",
    "openai",
    "anthropic",
    "langchain",
    "httpx",
    "polars",
    "duckdb",
    "uvicorn",
    "sqlalchemy",
    "pytest",
    "ruff",
    "litellm",
    "tiktoken",
    "celery",
    "ray"
  ],
  "enrichNpmDownloads": true,
  "onlyNew": false,
  "maxItemsPerEcosystem": 16,
  "delaySeconds": 0.15,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "mcp",
        "langchain",
        "opentelemetry",
        "pydantic"
    ],
    "ecosystems": [
        "github",
        "npm",
        "pypi",
        "crates",
        "docker"
    ],
    "pypiPackages": [
        "fastapi",
        "pydantic",
        "openai",
        "anthropic",
        "langchain",
        "httpx",
        "polars",
        "duckdb",
        "uvicorn",
        "sqlalchemy",
        "pytest",
        "ruff",
        "litellm",
        "tiktoken",
        "celery",
        "ray"
    ],
    "enrichNpmDownloads": true,
    "onlyNew": false,
    "maxItemsPerEcosystem": 16
};

// Run the Actor and wait for it to finish
const run = await client.actor("informed_almandine_vzb/github-npm-pypi-docker-package-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": [
        "mcp",
        "langchain",
        "opentelemetry",
        "pydantic",
    ],
    "ecosystems": [
        "github",
        "npm",
        "pypi",
        "crates",
        "docker",
    ],
    "pypiPackages": [
        "fastapi",
        "pydantic",
        "openai",
        "anthropic",
        "langchain",
        "httpx",
        "polars",
        "duckdb",
        "uvicorn",
        "sqlalchemy",
        "pytest",
        "ruff",
        "litellm",
        "tiktoken",
        "celery",
        "ray",
    ],
    "enrichNpmDownloads": True,
    "onlyNew": False,
    "maxItemsPerEcosystem": 16,
}

# Run the Actor and wait for it to finish
run = client.actor("informed_almandine_vzb/github-npm-pypi-docker-package-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "mcp",
    "langchain",
    "opentelemetry",
    "pydantic"
  ],
  "ecosystems": [
    "github",
    "npm",
    "pypi",
    "crates",
    "docker"
  ],
  "pypiPackages": [
    "fastapi",
    "pydantic",
    "openai",
    "anthropic",
    "langchain",
    "httpx",
    "polars",
    "duckdb",
    "uvicorn",
    "sqlalchemy",
    "pytest",
    "ruff",
    "litellm",
    "tiktoken",
    "celery",
    "ray"
  ],
  "enrichNpmDownloads": true,
  "onlyNew": false,
  "maxItemsPerEcosystem": 16
}' |
apify call informed_almandine_vzb/github-npm-pypi-docker-package-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,informed_almandine_vzb/github-npm-pypi-docker-package-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/mfjFcUDiyQaH3LmD5/builds/dopRlxdlCMnrjkfuh/openapi.json