🐍 PyPI Scraper

Look up Python packages on PyPI and get clean, structured metadata — version, summary, author, license, dependencies, classifiers, project links and release history. Powered by the public PyPI JSON API, so it's fast and reliable: no browser, no login, no API key. Bulk-process any list of package names in one run.

Built for dependency analysis, software supply-chain research, dev dashboards and package datasets. Export to JSON/CSV/Excel, run on a schedule, call via API, or connect to Make, Zapier or n8n.

🔎 What is the PyPI Scraper?

Give it a list of package names (e.g. httpx, requests, fastapi) and it returns one clean record per package — current version, who maintains it, its license, what it depends on and when it was last released. Perfect for auditing dependencies or building a package catalog.

What data does it extract?

• Name and current version
• Summary and content type
• Author and maintainer (with email)
• License (SPDX expression or classifier)
• Home page, repository and documentation links
• Required Python version
• Dependencies (requires_dist) and a dependency count
• Classifiers and keywords
• Release count and latest release date
• Latest distribution files (wheels/sdists) with size and URL

⬇️ Input

Field	Type	Description
packages	array	Exact PyPI package names, e.g. httpx, requests.

Example input

{
  "packages": ["httpx", "requests", "fastapi", "pydantic"]
}

⬆️ Output

One record per package:

{
  "name": "httpx",
  "version": "0.28.1",
  "summary": "The next generation HTTP client.",
  "author": "Tom Christie",
  "maintainer": null,
  "license": "BSD-3-Clause",
  "home_page": "https://github.com/encode/httpx",
  "package_url": "https://pypi.org/project/httpx/",
  "repository": "https://github.com/encode/httpx",
  "documentation": "https://www.python-httpx.org",
  "requires_python": ">=3.8",
  "requires_dist": ["certifi", "httpcore==1.*", "anyio", "idna", "sniffio"],
  "dependency_count": 5,
  "keywords": null,
  "classifiers": ["Development Status :: 4 - Beta", "Programming Language :: Python :: 3"],
  "yanked": false,
  "release_count": 74,
  "latest_release_date": "2024-12-06T18:30:00.000000Z",
  "latest_files": [
    {"filename": "httpx-0.28.1-py3-none-any.whl", "size": 73517, "python_version": "py3", "url": "https://files.pythonhosted.org/.../httpx-0.28.1-py3-none-any.whl"}
  ],
  "query": "httpx"
}

💡 Use cases

• 🔐 Supply-chain research — audit licenses, maintainers and dependencies.
• 📊 Dependency analysis — map what your stack actually pulls in.
• 🗂️ Package catalogs — build a structured dataset of Python libraries.
• 🤖 Automation — alert when a dependency ships a new release.

📈 Why scrape PyPI?

PyPI is the default package registry for the Python ecosystem, which means its metadata is a high-value source for security, compliance, developer tools and market research. Package versions, classifiers, maintainers, project links and dependency declarations can reveal whether a library is maintained, whether it supports your Python version, how often it ships, and where its source code or documentation lives.

For software supply-chain workflows, this actor helps turn a plain package list into an auditable dataset. You can enrich requirements.txt files, dependency manifests, SBOMs, internal package catalogs or customer-submitted project lists. Because the output includes links, release metadata and dependency counts, it is also useful for dashboards, security triage, LLM context packs and engineering productivity tools.

🔁 Automation workflow

Run the actor daily or weekly with the packages your product depends on. Store the previous result, compare version and latest_release_date, and trigger an alert when a new release appears. For compliance, export the output to CSV and filter by license, maintainer email or missing repository links. For research, run curated lists of packages by category and analyze release velocity or dependency complexity.

You can also chain this actor with GitHub and Docker Hub scrapers. PyPI gives you package metadata; GitHub adds stars, forks, issues and repository activity; Docker Hub adds image adoption. Together, they produce a much clearer picture of open-source project health than any single source.

✅ Data quality notes

The actor calls PyPI's package JSON endpoint directly, so results come from the registry rather than a scraped HTML page. Package names are de-duplicated before processing, missing packages are logged without stopping the run, and dependency lists are returned in PyPI's original requires_dist format so you can parse extras, environment markers and version constraints downstream.

❓ FAQ

Do I need an API key or login? No — it uses the public PyPI JSON API.

Can I look up many packages at once? Yes — pass an array of names; they run concurrently.

Do I get dependencies? Yes — the requires_dist list and a dependency count.

Is the license included? Yes — the SPDX expression or license classifier.

Does it include release history? Yes — the release count and latest release date.

What if a package doesn't exist? It's skipped and logged — the run continues.

How does pricing work? Pay per package returned. No subscription.

Is it legal? PyPI metadata is public. Use responsibly and within PyPI's terms.

⚙️ How it works

The scraper calls the PyPI JSON API for each package directly — no browser and no key. It fetches packages concurrently, de-duplicates by name, and normalizes each response into consistent fields (version, license, dependencies, links, releases), so you get a tidy table instead of raw JSON. Runs are fast and dependable, which is why the actor keeps passing its daily health check. The same input shape works for one package or a list of hundreds.

👥 Who uses PyPI data?

Package data is valuable to engineering teams, security analysts, researchers and tool builders. A security analyst audits the licenses and maintainers of every dependency; a platform team tracks which libraries are current; a researcher studies the Python ecosystem; a tool builder enriches a dashboard with package metadata. Because every record is plain JSON with consistent fields, it drops straight into a spreadsheet, database, BI tool or LLM pipeline with no custom parsing.

📤 Export, schedule & integrate

Every run is saved to a dataset you can export to JSON, CSV, Excel, XML or RSS, or pull through the Apify API. Wire it into Make, Zapier, n8n, Google Sheets, Slack or your own database, run it on a schedule to track new releases, and call it from AI agents through the Apify MCP server.

💡 Tips for best results

• Paste your requirements.txt package names to audit a whole project.
• Schedule a run and diff version to get a new-release alert.
• Use classifiers to filter packages by Python version or status.
• Combine with the GitHub scraper to enrich a package with repo stars.

❓ More FAQ

How fresh is the data? It is fetched live on each run.

Can I run it automatically? Yes — use Apify Schedules (cron).

Are duplicates removed? Yes — by package name within each run.

Which export formats? JSON, CSV, Excel, XML and RSS, plus the Apify API.

Can AI agents use it? Yes — via the Apify API and MCP server.

🔗 You might also like

• GitHub Repository Scraper — repo metadata & stars.
• Docker Hub Scraper — image data & pulls.
• Dev.to Articles Scraper — developer articles.

---

Keywords: pypi scraper, pypi api, python package data, python packages, dependency analysis, requirements.txt, package metadata, software supply chain, python libraries, package scraper, pip packages, open source data, python dependencies, package dataset