# PyPI Package Scraper (`ef12/pypi-package-scraper`) Actor

Search PyPI packages, get package metadata, download counts, and version history using the free PyPI JSON API. Track trending packages by download volume.

- **URL**: https://apify.com/ef12/pypi-package-scraper.md
- **Developed by:** [Daniel Wilson](https://apify.com/ef12) (community)
- **Categories:** Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## PyPI Package Scraper

### What does PyPI Package Scraper do?

**PyPI Package Scraper** extracts detailed metadata, version history, and download statistics from the [Python Package Index (PyPI)](https://pypi.org/) — the official repository of over 500,000 Python packages. It uses the official PyPI JSON API and pypistats.org to deliver structured data without requiring a browser or HTML parsing.

You can search for packages by keyword, look up a specific package's full metadata and version history, or browse trending packages sorted by download volume. All data is returned as clean JSON in your Apify dataset, ready for analysis, integration, or export.

The Apify platform handles scheduling, scaling, and monitoring — no infrastructure to manage. Results are accessible via API, webhook, or downloadable in JSON, CSV, HTML, or Excel.

### Why use PyPI Package Scraper?

**Data-driven decision making:** Evaluate which packages to use in your project by comparing download numbers, license types, Python version requirements, and maintenance recency side by side.

**Dependency auditing:** Automate identification of dependency chain metadata — author, repository URL, license — across your entire project for compliance or reporting.

**Market intelligence:** Track the PyPI ecosystem. Monitor download trends for competing packages, spot rising stars, and identify packages with stale maintenance records.

**CI/CD integration:** Fetch a package's latest version programmatically inside your build pipeline to alert on outdated dependencies or license changes.

**Data enrichment:** Enrich your own dataset with authoritative PyPI metadata — package name, summary, author, home page, repository URL, keywords.

### How to use PyPI Package Scraper

1. **Open the Actor** — Go to the PyPI Package Scraper page on Apify Console and click **Try** or **Run**.
2. **Configure input** — Choose a mode and fill in the fields. You can run a search, fetch a single package, or get trending packages.
3. **Run the Actor** — Hit the start button. The scraper fetches data from PyPI in seconds using concurrent HTTP requests.
4. **Get your data** — Results appear in the **Output** tab as a table. Download in JSON, CSV, HTML, Excel, or access via the Apify API.

If you prefer to run programmatically, use the Apify API or one of the SDKs (Python, JavaScript).

### Input

The Actor accepts a JSON object with the following fields:

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `mode` | string | `"search"` | One of `search`, `package`, or `trending` |
| `query` | string | `""` | Keyword search term (required for **search** mode) |
| `package_name` | string | `""` | Exact PyPI package name (required for **package** mode) |
| `max_results` | integer | `25` | Maximum results (1–250) |
| `include_downloads` | boolean | `true` | Fetch daily download counts from pypistats.org |

#### Example input (search mode)

```json
{
    "mode": "search",
    "query": "async http client",
    "max_results": 10,
    "include_downloads": true
}
````

#### Example input (package mode)

```json
{
    "mode": "package",
    "package_name": "httpx",
    "include_downloads": true
}
```

#### Example input (trending mode)

```json
{
    "mode": "trending",
    "max_results": 20,
    "include_downloads": true
}
```

### Output

Each result in the dataset contains these fields:

```json
{
    "name": "httpx",
    "version": "0.28.1",
    "description": "The next generation HTTP client.",
    "author": "Tom Christie",
    "license": "BSD-3-Clause",
    "home_page": "https://www.python-httpx.org",
    "repository": "https://github.com/encode/httpx",
    "keywords": ["http", "async", "client", "rest", "api"],
    "requires_python": ">=3.8",
    "last_modified": "2024-12-15T10:30:00",
    "downloads": 1256789,
    "versions": ["0.28.0", "0.28.1"]
}
```

The `versions` array is included only in **package mode**. The `downloads` field appears only when `include_downloads` is `true`.

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel from the Apify Console.

### Data fields

| Field | Type | Description |
| --- | --- | --- |
| `name` | string | Package name |
| `version` | string | Latest version string |
| `description` | string | One-line summary from setup.py/pyproject.toml |
| `author` | string | Package author |
| `license` | string | SPDX license identifier |
| `home_page` | string | Project home page URL |
| `repository` | string | Source code repository URL |
| `keywords` | array of strings | Classifier keywords |
| `requires_python` | string | Minimum Python version constraint |
| `last_modified` | string | ISO date of most recent file upload |
| `downloads` | integer | Last-day download count (optional) |
| `versions` | array of strings | All published version strings (package mode only) |

### Pricing / Cost estimation

The Actor uses Apify's **pay-per-event (PPE)** pricing model:

- **10 results are free** per run, regardless of mode.
- After the first 10, each additional result costs **$0.002** (one-fifth of a cent).
- The Actor start event costs **$0.00005**.

A typical search run returning 25 results costs approximately 15 × $0.002 = **$0.03**plus negligible start cost. A trending run with 50 results costs roughly 40 × $0.002 = **$0.08**.

Running locally with `apify run` does not incur any charges. Costs are only generated when running on the Apify platform.

### Use Cases

#### Open-source maintainers

Track download trends for your own packages. Run the scraper weekly on Apify's scheduler and compare daily download counts over time. Detect drops that might indicate a packaging issue or a competing library gaining traction.

#### Security and compliance teams

Audit your organization's Python dependencies at scale. Run package mode for each dependency in your requirements.txt to collect license identifiers, author names, and repository URLs. Cross-reference licenses against your approved list and flag unmaintained packages (those with old `last_modified` dates).

#### Data scientists and analysts

Enrich package recommendation models with real PyPI metadata. Collect keyword tags, download volumes, and Python version constraints across hundreds of packages to identify clusters, popularity trends, and compatibility patterns. Export to CSV for analysis in pandas, R, or your BI tool of choice.

#### DevOps and platform engineers

Integrate into your CI/CD pipeline via the Apify API. On every deploy, check your critical dependencies' latest versions and repository URLs. Alert on packages whose latest version is more than 12 months old. Feed results into your dependency update bot (Renovate, Dependabot).

#### Content and documentation teams

Generate up-to-date package reference tables for tutorials, comparison articles, or internal wikis. Search mode returns summary metadata that's ready to embed in documentation. The deterministic output structure makes it easy to diff changes between runs.

#### Research and education

Study the Python ecosystem quantitatively. Pull metadata for hundreds of packages via trending mode and analyze license prevalence, author diversity, description vocabulary, and version velocity across the top packages. All data is structured and repeatable — perfect for longitudinal studies.

### Tips and advanced options

- **Maximize throughput** — The Actor uses up to 10 concurrent HTTP requests. Running with `max_results` up to 50 usually completes in under 30 seconds.
- **Minimize cost** — Set `include_downloads: false` to skip the pypistats.org API call. This cuts execution time roughly in half and reduces the number of API roundtrips.
- **Package mode for depth** — Use package mode when you need the full version history. The `versions` array includes every release, not just the latest.
- **Search pagination** — PyPI search returns a single results page. For broader coverage, use a lower `max_results` and vary your `query` terms across runs.
- **Retries** — The Actor automatically retries failed requests with exponential backoff. Transient network errors on PyPI's side are handled transparently.

### FAQ, disclaimers, and support

**Is it legal to scrape PyPI?** Yes. PyPI's JSON API is designed for programmatic access and has no rate-limiting restrictions for reasonable use. pypistats.org also provides a public API. Both APIs are free and require no authentication.

**Can I use this for commercial purposes?** Yes. All PyPI data is publicly available. The Apify platform itself requires a paid plan for production use above the free tier.

**Rate limits** — PyPI does not enforce strict rate limits but may throttle aggressive requests. The Actor uses a concurrency limit of 10 connections and retries with backoff to stay within reasonable bounds.

**Known limitations** — Search mode uses PyPI's web search results page, which returns at most 20 results per query. For exhaustive package discovery, consider iterating the simple index directly. The pypistats.org API reports download counts with a ~24-hour delay.

**Need help?** Open an issue on the [GitHub repository](https://github.com/apify-fleet/pypi-package-scraper) for bugs or feature requests. For custom scraping solutions or enterprise support, contact the Apify team through the platform.

**Disclaimer** — This Actor is provided as-is. The author is not responsible for how you use the scraped data. Always respect PyPI's terms of service and applicable laws.

# Actor input Schema

## `mode` (type: `string`):

Scraping mode: search packages by keyword, fetch a single package, or get trending packages by download volume

## `query` (type: `string`):

Keyword search term (required for search mode)

## `package_name` (type: `string`):

Exact PyPI package name to fetch (required for package mode)

## `max_results` (type: `integer`):

Maximum number of packages to return (max 250)

## `include_downloads` (type: `boolean`):

Fetch recent download counts from pypistats.org for each package

## Actor input object example

```json
{
  "mode": "search",
  "query": "",
  "package_name": "",
  "max_results": 25,
  "include_downloads": true
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("ef12/pypi-package-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("ef12/pypi-package-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call ef12/pypi-package-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=ef12/pypi-package-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "PyPI Package Scraper",
        "description": "Search PyPI packages, get package metadata, download counts, and version history using the free PyPI JSON API. Track trending packages by download volume.",
        "version": "0.0",
        "x-build-id": "uzlBMPzd6XIrNtyKD"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/ef12~pypi-package-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-ef12-pypi-package-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/ef12~pypi-package-scraper/runs": {
            "post": {
                "operationId": "runs-sync-ef12-pypi-package-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/ef12~pypi-package-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-ef12-pypi-package-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "properties": {
                    "mode": {
                        "title": "Mode",
                        "enum": [
                            "search",
                            "package",
                            "trending"
                        ],
                        "type": "string",
                        "description": "Scraping mode: search packages by keyword, fetch a single package, or get trending packages by download volume",
                        "default": "search"
                    },
                    "query": {
                        "title": "Search Query",
                        "type": "string",
                        "description": "Keyword search term (required for search mode)",
                        "default": ""
                    },
                    "package_name": {
                        "title": "Package Name",
                        "type": "string",
                        "description": "Exact PyPI package name to fetch (required for package mode)",
                        "default": ""
                    },
                    "max_results": {
                        "title": "Max Results",
                        "minimum": 1,
                        "maximum": 250,
                        "type": "integer",
                        "description": "Maximum number of packages to return (max 250)",
                        "default": 25
                    },
                    "include_downloads": {
                        "title": "Include Download Stats",
                        "type": "boolean",
                        "description": "Fetch recent download counts from pypistats.org for each package",
                        "default": true
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```