Package Intel Scraper — PyPI & npm Metadata, Downloads, Deps & Vulns

One-row-per-package intelligence for the two biggest software ecosystems, built only on free official APIs — no browser, no proxies, no API keys, zero anti-bot blocks.

What it does

Pick any of five modes:

Mode	What you get
fullReport (default)	Everything below in a single row
packageDetails	Metadata: versions, author, license, homepage, repo URL, release timeline
downloads	Download stats: last day / week / month
dependencies	Direct dependency graph (count + top 10)
vulnerabilities	Known vulnerabilities: count, max severity, top CVE IDs

A package missing from both registries yields a not_found row (never charged). Network or source failures yield an English setup_guide row and exit 0 — the dataset is never empty, so Apify's automatic quality check always sees data.

Data sources (all free, no key)

Source	Endpoint	Used for
PyPI JSON API	https://pypi.org/pypi/{name}/json	metadata, releases, yanked, classifiers
npm Registry	https://registry.npmjs.org/{name}	metadata, versions, maintainers
pypistats.org	https://pypistats.org/api/packages/{name}/recent	exact PyPI downloads (day/week/month)
npm downloads API	https://api.npmjs.org/downloads/range/last-month/{name}	npm downloads, last 30 days
deps.dev	https://api.deps.dev/v3/systems/{PYPI|NPM}/packages/{name}	direct dependencies (1 level)
OSV.dev	POST https://api.osv.dev/v1/query	known vulnerabilities (CVE/GHSA IDs + CVSS severity)

npm download stats are marked downloadsEstimated: true (derived from the 30-day daily series); PyPI numbers from pypistats are exact.

Input

• packages — array of {"name": "requests", "ecosystem": "auto"}. ecosystem can be auto (try PyPI first, then npm), PyPI or npm.
• mode — one of the five modes above, default fullReport.
• version — optional pin, e.g. 2.31.0. Empty = latest.
• maxResults — cap on packages per run, default 50.
• includeVulns — query OSV.dev (default on).

Output fields (one row per package)

Shared: itemType, mode, ecosystem, queriedAt, name, latestVersion, queriedVersion, description, author, license, homepage, repoUrl (Repository > Source > home page), firstReleaseAt, lastReleaseAt, releaseCount, status.

Downloads: downloadsLastDay, downloadsLastWeek, downloadsLastMonth, downloadsEstimated.

Dependencies: directDepCount, directDepsTop10 (name@requirement), devDepCount (npm).

Vulnerabilities: vulnCount, maxSeverity (Low/Medium/High/Critical), vulnIdsTop5, hasKnownVulns.

PyPI-only: requiresPython, yanked, classifiersTop (top 5). npm-only: maintainers (top 5), unpackedSizeKb.

Billing

• Pay-per-event package: one charge per package data row.
• First 20 rows per run are free; not_found and setup_guide rows are never charged.

Local development

python3 -m venv ~/.venvs-pkgintel
~/.venvs-pkgintel/bin/pip install -r requirements.txt pytest pytest-asyncio
~/.venvs-pkgintel/bin/python -m pytest tests/ -q