# PyPI Scraper - Python Package Data (`benthepythondev/pypi-scraper`) Actor

Look up Python packages on PyPI and get clean metadata: version, summary, author, license, dependencies, classifiers, project links and release history. Fast and reliable via the public PyPI JSON API, no key. Bulk-process any list of package names.

- **URL**: https://apify.com/benthepythondev/pypi-scraper.md
- **Developed by:** [ben](https://apify.com/benthepythondev) (community)
- **Categories:** Developer tools, Business, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## 🐍 PyPI Scraper

Look up **Python packages** on **PyPI** and get clean, structured metadata — version, summary, author, license, dependencies, classifiers, project links and release history. Powered by the public PyPI JSON API, so it's fast and reliable: no browser, no login, no API key. Bulk-process any list of package names in one run.

Built for dependency analysis, software supply-chain research, dev dashboards and package datasets. Export to JSON/CSV/Excel, run on a schedule, call via API, or connect to Make, Zapier or n8n.

### 🔎 What is the PyPI Scraper?

Give it a list of package names (e.g. `httpx`, `requests`, `fastapi`) and it returns one clean record per package — current version, who maintains it, its license, what it depends on and when it was last released. Perfect for auditing dependencies or building a package catalog.

#### What data does it extract?

- **Name** and current **version**
- **Summary** and content type
- **Author** and **maintainer** (with email)
- **License** (SPDX expression or classifier)
- **Home page**, **repository** and **documentation** links
- **Required Python** version
- **Dependencies** (`requires_dist`) and a dependency count
- **Classifiers** and **keywords**
- **Release count** and **latest release date**
- **Latest distribution files** (wheels/sdists) with size and URL

### ⬇️ Input

| Field | Type | Description |
|-------|------|-------------|
| `packages` | array | Exact PyPI package names, e.g. `httpx`, `requests`. |

#### Example input

```json
{
  "packages": ["httpx", "requests", "fastapi", "pydantic"]
}
````

### ⬆️ Output

One record per package:

```json
{
  "name": "httpx",
  "version": "0.28.1",
  "summary": "The next generation HTTP client.",
  "author": "Tom Christie",
  "maintainer": null,
  "license": "BSD-3-Clause",
  "home_page": "https://github.com/encode/httpx",
  "package_url": "https://pypi.org/project/httpx/",
  "repository": "https://github.com/encode/httpx",
  "documentation": "https://www.python-httpx.org",
  "requires_python": ">=3.8",
  "requires_dist": ["certifi", "httpcore==1.*", "anyio", "idna", "sniffio"],
  "dependency_count": 5,
  "keywords": null,
  "classifiers": ["Development Status :: 4 - Beta", "Programming Language :: Python :: 3"],
  "yanked": false,
  "release_count": 74,
  "latest_release_date": "2024-12-06T18:30:00.000000Z",
  "latest_files": [
    {"filename": "httpx-0.28.1-py3-none-any.whl", "size": 73517, "python_version": "py3", "url": "https://files.pythonhosted.org/.../httpx-0.28.1-py3-none-any.whl"}
  ],
  "query": "httpx"
}
```

### 💡 Use cases

- 🔐 **Supply-chain research** — audit licenses, maintainers and dependencies.
- 📊 **Dependency analysis** — map what your stack actually pulls in.
- 🗂️ **Package catalogs** — build a structured dataset of Python libraries.
- 🤖 **Automation** — alert when a dependency ships a new release.

### 📈 Why scrape PyPI?

PyPI is the default package registry for the Python ecosystem, which means its metadata is a high-value source for security, compliance, developer tools and market research. Package versions, classifiers, maintainers, project links and dependency declarations can reveal whether a library is maintained, whether it supports your Python version, how often it ships, and where its source code or documentation lives.

For software supply-chain workflows, this actor helps turn a plain package list into an auditable dataset. You can enrich `requirements.txt` files, dependency manifests, SBOMs, internal package catalogs or customer-submitted project lists. Because the output includes links, release metadata and dependency counts, it is also useful for dashboards, security triage, LLM context packs and engineering productivity tools.

### 🔁 Automation workflow

Run the actor daily or weekly with the packages your product depends on. Store the previous result, compare `version` and `latest_release_date`, and trigger an alert when a new release appears. For compliance, export the output to CSV and filter by license, maintainer email or missing repository links. For research, run curated lists of packages by category and analyze release velocity or dependency complexity.

You can also chain this actor with GitHub and Docker Hub scrapers. PyPI gives you package metadata; GitHub adds stars, forks, issues and repository activity; Docker Hub adds image adoption. Together, they produce a much clearer picture of open-source project health than any single source.

### ✅ Data quality notes

The actor calls PyPI's package JSON endpoint directly, so results come from the registry rather than a scraped HTML page. Package names are de-duplicated before processing, missing packages are logged without stopping the run, and dependency lists are returned in PyPI's original `requires_dist` format so you can parse extras, environment markers and version constraints downstream.

### ❓ FAQ

**Do I need an API key or login?** No — it uses the public PyPI JSON API.

**Can I look up many packages at once?** Yes — pass an array of names; they run concurrently.

**Do I get dependencies?** Yes — the `requires_dist` list and a dependency count.

**Is the license included?** Yes — the SPDX expression or license classifier.

**Does it include release history?** Yes — the release count and latest release date.

**What if a package doesn't exist?** It's skipped and logged — the run continues.

**How does pricing work?** Pay per package returned. No subscription.

**Is it legal?** PyPI metadata is public. Use responsibly and within PyPI's terms.

### ⚙️ How it works

The scraper calls the PyPI JSON API for each package directly — no browser and no key. It fetches packages concurrently, de-duplicates by name, and normalizes each response into consistent fields (version, license, dependencies, links, releases), so you get a tidy table instead of raw JSON. Runs are fast and dependable, which is why the actor keeps passing its daily health check. The same input shape works for one package or a list of hundreds.

### 👥 Who uses PyPI data?

Package data is valuable to engineering teams, security analysts, researchers and tool builders. A security analyst audits the licenses and maintainers of every dependency; a platform team tracks which libraries are current; a researcher studies the Python ecosystem; a tool builder enriches a dashboard with package metadata. Because every record is plain JSON with consistent fields, it drops straight into a spreadsheet, database, BI tool or LLM pipeline with no custom parsing.

### 📤 Export, schedule & integrate

Every run is saved to a dataset you can export to **JSON, CSV, Excel, XML or RSS**, or pull through the **Apify API**. Wire it into **Make, Zapier, n8n, Google Sheets, Slack** or your **own database**, run it on a **schedule** to track new releases, and call it from AI agents through the **Apify MCP server**.

### 💡 Tips for best results

- Paste your `requirements.txt` package names to audit a whole project.
- Schedule a run and diff `version` to get a new-release alert.
- Use `classifiers` to filter packages by Python version or status.
- Combine with the GitHub scraper to enrich a package with repo stars.

### ❓ More FAQ

**How fresh is the data?** It is fetched live on each run.

**Can I run it automatically?** Yes — use Apify Schedules (cron).

**Are duplicates removed?** Yes — by package name within each run.

**Which export formats?** JSON, CSV, Excel, XML and RSS, plus the Apify API.

**Can AI agents use it?** Yes — via the Apify API and MCP server.

### 🔗 You might also like

- [GitHub Repository Scraper](https://apify.com/benthepythondev/github-repository-scraper) — repo metadata & stars.
- [Docker Hub Scraper](https://apify.com/benthepythondev/dockerhub-scraper) — image data & pulls.
- [Dev.to Articles Scraper](https://apify.com/benthepythondev/devto-articles-scraper) — developer articles.

***

**Keywords:** pypi scraper, pypi api, python package data, python packages, dependency analysis, requirements.txt, package metadata, software supply chain, python libraries, package scraper, pip packages, open source data, python dependencies, package dataset

# Actor input Schema

## `packages` (type: `array`):

Exact PyPI package names to look up, e.g. 'httpx', 'requests'.

## Actor input object example

```json
{
  "packages": [
    "httpx",
    "requests"
  ]
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "packages": [
        "httpx",
        "requests"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("benthepythondev/pypi-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "packages": [
        "httpx",
        "requests",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("benthepythondev/pypi-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "packages": [
    "httpx",
    "requests"
  ]
}' |
apify call benthepythondev/pypi-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=benthepythondev/pypi-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "PyPI Scraper - Python Package Data",
        "description": "Look up Python packages on PyPI and get clean metadata: version, summary, author, license, dependencies, classifiers, project links and release history. Fast and reliable via the public PyPI JSON API, no key. Bulk-process any list of package names.",
        "version": "1.0",
        "x-build-id": "2Wxt5pU67LUXAvMKe"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/benthepythondev~pypi-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-benthepythondev-pypi-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/benthepythondev~pypi-scraper/runs": {
            "post": {
                "operationId": "runs-sync-benthepythondev-pypi-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/benthepythondev~pypi-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-benthepythondev-pypi-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "properties": {
                    "packages": {
                        "title": "Package names",
                        "type": "array",
                        "description": "Exact PyPI package names to look up, e.g. 'httpx', 'requests'.",
                        "items": {
                            "type": "string"
                        }
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```