PyPI Package Scraper — Downloads, Deps, Metadata & Version Intel

Extract rich Python package intelligence from PyPI: version, author, license, dependencies, release date, and 30-day download counts — all without an API key.

What does PyPI Package Scraper do?

PyPI Package Scraper pulls structured data from two official, open endpoints: the PyPI JSON API (https://pypi.org/pypi/{package}/json) for full package metadata and the PyPI Stats API (https://pypistats.org/api/packages/{package}/recent) for download counts. No authentication or API key is needed.

The actor supports three modes. topPackages mode fetches the top-N most-downloaded packages of the last 30 days from the community-maintained hugovk/top-pypi-packages list, then enriches each package in parallel. packages mode accepts an explicit list of package names and enriches them. packageDetail is a single-package alias. For each package the actor resolves the latest stable version, full metadata (author, license, classifiers, requires-dist, requires-python, project URLs), and the last-month download figure from PyPI Stats.

Runs use bounded concurrency (6 workers) to stay well within rate limits while keeping high throughput. All network errors are retried with exponential backoff; a failed package logs a warning and the run continues. The actor exits cleanly even when zero valid packages are found.

Who is it for?

• Developer tools and OSS researchers who need bulk Python package metadata for trend analysis, tech-stack detection, or SBOM generation.
• Security and DevOps teams tracking dependency licenses, Python version requirements, and known package metadata across hundreds of projects.
• Data scientists and analysts building dashboards of PyPI download trends and ecosystem growth.
• Recruiters and VC analysts mining which Python packages power production workloads or which packages are gaining traction.
• Product managers and founders validating market size by measuring how widely a package (or category) is downloaded each month.

Use cases

• Dependency auditing: pull the metadata for every package in a requirements.txt and flag those with missing licenses or incompatible Python requirements.
• Ecosystem mapping: grab the top 500 most-downloaded packages and group them by classifier (web framework, ML, data, etc.) to visualize the Python landscape.
• Download trend tracking: run the scraper weekly and diff recentDownloads over time to spot packages gaining or losing momentum.
• Tech-stack fingerprinting: enrich a list of packages detected via import analysis to get author, homepage, and project URLs for lead generation.
• OSS intelligence: identify the most active PyPI maintainers and their contact emails for partnership or recruitment outreach.

Why use PyPI Package Scraper?

• 100% keyless — both the PyPI JSON API and PyPI Stats are public endpoints requiring zero credentials.
• 15 data fields per package — name, version, summary, author, email, license, homepage, project URLs, dependencies, Python requirements, keywords, release date, downloads, classifiers, and PyPI URL.
• Bulk-first design — the topPackages mode can enrich 5,000 packages in a single run with concurrent workers and smart rate-limiting.
• Resilient — automatic retries with exponential backoff on 429 and 5xx; failed packages skip gracefully without stopping the run.
• Export-ready — download results as CSV, JSON, Excel, or XML directly from the Apify dataset UI, or pipe to Google Sheets via integration.
• Pay-per-result — you only pay for the data you get, making large research runs extremely cost-efficient.

What data can you extract?

Field	Type	Description
name	string	PyPI package name (e.g., requests)
version	string	Latest published version (e.g., 2.31.0)
summary	string	One-line description from the package metadata
author	string	Primary author name
authorEmail	string	Author's email address (if provided)
license	string	SPDX identifier or license string (e.g., MIT, Apache 2.0)
homePage	string	Project homepage URL
projectUrls	string	JSON object of named project URLs (Source, Documentation, Tracker, etc.)
requiresDist	string	JSON array of dependency specifiers (e.g., ['urllib3>=1.21.1', 'certifi>=2017.4.17'])
requiresPython	string	Python version constraint (e.g., >=3.8)
keywords	string	Comma-separated keyword tags
releaseDate	string	ISO 8601 upload timestamp of the latest release
recentDownloads	number	30-day download count from PyPI Stats
pypiUrl	string	Direct URL to the package's PyPI page
classifiers	string	JSON array of Trove classifiers (programming language, topic, license)
scrapedAt	string	ISO 8601 timestamp when the record was collected

JSON output example:

{
  "name": "requests",
  "version": "2.31.0",
  "summary": "Python HTTP for Humans.",
  "author": "Kenneth Reitz",
  "authorEmail": "me@kennethreitz.org",
  "license": "Apache 2.0",
  "homePage": "https://requests.readthedocs.io",
  "projectUrls": "{\"Documentation\":\"https://requests.readthedocs.io\",\"Source\":\"https://github.com/psf/requests\"}",
  "requiresDist": "[\"charset-normalizer<4,>=2\",\"idna<4,>=2.5\",\"urllib3<3,>=1.21.1\",\"certifi>=2017.4.17\"]",
  "requiresPython": ">=3.7",
  "keywords": "requests, http, https, rest, api",
  "releaseDate": "2023-05-22T15:12:44",
  "recentDownloads": 87432109,
  "pypiUrl": "https://pypi.org/project/requests/",
  "classifiers": "[\"License :: OSI Approved :: Apache Software License\",\"Programming Language :: Python :: 3\"]",
  "scrapedAt": "2026-07-08T10:00:00.000Z"
}

How to use

Option A — Top N packages by download count

The most common use case: get the top-ranked PyPI packages with all metadata and download stats.

1. Set mode to topPackages.
2. Set topN to the number of packages you want (e.g., 200 for the top 200 most-downloaded packages).
3. Optionally adjust maxResults to cap output.

Input example:

{
  "mode": "topPackages",
  "topN": 200,
  "maxResults": 200
}

Option B — Specific package list

Enrich an explicit list of package names — useful when you have a requirements.txt or a curated set.

1. Set mode to packages.
2. Provide packages as a list of PyPI package names.

Input example:

{
  "mode": "packages",
  "packages": ["requests", "numpy", "pandas", "flask", "django", "fastapi", "pydantic", "boto3"],
  "maxResults": 100
}

Option C — Single package detail

Retrieve all fields for one package, ideal for quick lookups.

Input example:

{
  "mode": "packageDetail",
  "packages": ["httpx"]
}

Input parameters

Parameter	Type	Default	Description
mode	string	topPackages	packages, topPackages, or packageDetail
packages	array	[]	List of package names (used in packages / packageDetail modes)
topN	integer	100	Number of top packages to fetch (used in topPackages mode, max 5000)
maxResults	integer	1000	Hard cap on total output records
proxyConfiguration	object	{}	Apify proxy settings

Full input JSON:

{
  "mode": "topPackages",
  "packages": [],
  "topN": 500,
  "maxResults": 500,
  "proxyConfiguration": {}
}

Output example

{
  "name": "numpy",
  "version": "1.26.4",
  "summary": "Fundamental package for array computing in Python",
  "author": "Travis E. Oliphant et al.",
  "authorEmail": null,
  "license": "BSD",
  "homePage": "https://numpy.org",
  "projectUrls": "{\"Bug Tracker\":\"https://github.com/numpy/numpy/issues\",\"Documentation\":\"https://numpy.org/doc/1.26\",\"Source Code\":\"https://github.com/numpy/numpy\"}",
  "requiresDist": null,
  "requiresPython": ">=3.9",
  "keywords": "numpy, scientific, math, array",
  "releaseDate": "2024-02-05T18:22:11",
  "recentDownloads": 121000000,
  "pypiUrl": "https://pypi.org/project/numpy/",
  "classifiers": "[\"License :: OSI Approved :: BSD License\",\"Programming Language :: Python :: 3\",\"Topic :: Scientific/Engineering :: Mathematics\"]",
  "scrapedAt": "2026-07-08T10:00:00.000Z"
}

Tips for best results

• Use topPackages for discovery: it pulls from the real 30-day download ranking, so you always get the freshest list of high-impact packages.
• Set topN to 5000 for a full market scan — the run completes in under 30 minutes at 6x concurrency.
• Use packages mode for targeted enrichment: paste your requirements.txt contents directly into the package list for a SBOM-style audit.
• Filter by requiresPython in your downstream analysis to identify packages that haven't migrated to Python 3.12+.
• Sort by recentDownloads to find the fastest-growing or highest-usage packages within a category.
• Parse requiresDist as JSON to build a full dependency tree for any package or group of packages.
• Check classifiers for license type: the JSON array includes License :: OSI Approved :: ... entries for accurate license identification.
• Run on a schedule (weekly or monthly) and compare recentDownloads over time to detect download trend shifts.
• Cross-reference authorEmail with your CRM to find which open-source authors are already your users or prospects.
• Combine with the NPM Package Intelligence Scraper to compare Python vs JavaScript ecosystem adoption for the same category of tools.

Integrations

Google Sheets: use the Google Sheets integration in the Apify Console to stream dataset rows directly into a spreadsheet in real time — ideal for collaborative team analysis.

Slack: configure a Webhook notification to post a summary to your Slack channel when the run finishes — useful for weekly tracking jobs.

Zapier / Make: trigger downstream workflows (enrich a CRM, update a Notion database, send alerts) whenever new packages are scraped.

Scheduled runs: set up a weekly cron in Apify to keep your dataset fresh and compare download trends over time.

Webhooks: receive a POST request to your own endpoint when the run succeeds, with a link to the dataset.

API usage

cURL:

curl -X POST \
  "https://api.apify.com/v2/acts/logiover~pypi-package-scraper/runs?token=YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode":"topPackages","topN":100}'

Node.js (Apify client):

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });
const run = await client.actor('logiover/pypi-package-scraper').call({
  mode: 'topPackages',
  topN: 200,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items[0]);

Python (Apify client):

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")
run = client.actor("logiover/pypi-package-scraper").call(run_input={
    "mode": "packages",
    "packages": ["requests", "numpy", "pandas"],
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["name"], item["recentDownloads"])

Use with AI agents (MCP)

PyPI Package Scraper is available as an MCP tool for AI agents. Connect it through the Apify MCP Server and invoke it from Claude, GPT, or any MCP-compatible agent:

"Fetch the top 100 PyPI packages and return those with MIT or Apache licenses and over 1 million monthly downloads."

The agent will call the actor, receive the dataset, and filter or analyze the results — no manual API calls required.

FAQ

Does this actor require an API key?

No. Both the PyPI JSON API and the pypistats API are completely public. No PyPI account, no token, no login.

How many packages can I scrape in one run?

The topPackages mode supports up to 5,000 packages per run. The packages mode accepts any list size, capped by maxResults (max 10,000). A 5,000-package run typically finishes in under 30 minutes.

What is the source for the "top packages" list?

It uses the hugovk/top-pypi-packages dataset — a widely-used community resource updated monthly with the top 5,000 PyPI packages by 30-day download count.

Why is recentDownloads null for some packages?

The pypistats API may return no data for very new packages (less than 30 days old) or packages with extremely low download counts. The field is set to null rather than 0 in these cases.

Why are some fields like requiresDist stored as JSON strings?

The PyPI JSON API returns these as arrays/objects. To stay compatible with the Apify output schema (which requires all top-level fields to be primitive strings), complex structures are serialized as compact JSON strings. Parse them with JSON.parse() in your downstream code.

Can I filter by license or Python version requirement?

Not directly in the actor — apply these filters in your downstream tool (e.g., filter dataset items in Google Sheets, Pandas, or a Zapier step) using the license and requiresPython fields.

How do I export the results?

In the Apify Console, click the dataset tab after the run completes. Download as JSON, CSV, Excel, XML, or JSONL — all formats are available for free.

How fast does the actor run?

With default concurrency (6 workers), expect roughly 100 packages per minute. A 500-package run takes around 5 minutes; a 5,000-package run takes 30–45 minutes depending on pypistats response times.

Is there a cost?

Yes — Apify charges per compute unit (CPU/memory time) and per dataset item write. PyPI and pypistats have no costs. A typical 500-package run costs under $0.05 on the Apify platform.

How often should I run this?

The pypistats "recent" endpoint reflects the last 30 days. Weekly runs give you a rolling trend. For monitoring the top-100 packages, a monthly run is usually sufficient.

What if a package returns zero results?

If a package name doesn't exist on PyPI (404 response), the actor logs a warning and skips it. It doesn't stop the run. You'll see the final count in the run log.

Does this work for private PyPI mirrors?

No — the actor is hardcoded to pypi.org and pypistats.org. For private mirrors you would need a custom fork.

Is it legal?

This actor only accesses publicly available data from pypi.org and pypistats.org — the official Python Package Index operated by the Python Software Foundation. Both endpoints are designed for programmatic access: the PyPI JSON API is documented and encouraged by PSF, and pypistats.org publishes aggregate download statistics as a public service.

No authentication is bypassed, no personal data beyond what package maintainers voluntarily publish (author name, email) is collected, and data is fetched at a rate that respects server resources. Always ensure your use of this data complies with PyPI's Terms of Use and applicable data protection regulations in your jurisdiction.

Related scrapers

• NPM Package Intelligence Scraper — Same concept for the Node.js ecosystem: search, keyword filter, download stats, and full metadata from the NPM registry.
• GitHub Activity Stream — Track commits, releases, issues, and PRs for any GitHub repository — great for monitoring PyPI packages' upstream activity.
• Hacker News Search Scraper — Find HN discussions about any Python package or library to gauge community sentiment.
• CVE Security Advisory Monitor — Cross-reference PyPI package names against the NVD CVE database to surface known security vulnerabilities.