Package Intel Scraper — npm package metadata API + PyPI package data + dependency vulnerability data

Pull registry metadata and known-vulnerability signals for npm and PyPI packages in one normalized dataset. Give it a list of package names, get back one row per package: latest version, license, maintainer count, download volume (npm), deprecation status, and a list of known vulnerabilities from OSV.dev — all in a single run, no scraping of HTML pages, no API keys.

Why this actor

npm's registry (registry.npmjs.org) and PyPI's JSON API (pypi.org/pypi/<name>/json) are both fully public and unauthenticated — this actor talks to them directly. Vulnerability data comes from OSV.dev, Google's open, free vulnerability database that aggregates GitHub Security Advisories, PyPI's own advisory feed, and more — also public, also no key required. All three endpoints were called live against real packages before this actor shipped; see Data sources below for exactly what each returns.

Honesty note on the market for this: the closest tools in this space — Snyk, Socket — build their own dependency-graph and vulnerability infrastructure in-house rather than buying a scraped feed. This actor is useful for one-off audits, spreadsheet enrichment, and lightweight monitoring where standing up a full SCA (software composition analysis) platform is overkill, but it is not a drop-in replacement for those tools. Willingness to pay for this specific packaging of public data, via a scraper marketplace, is inferred from the total lack of competition on the Apify Store — it has not been proven by paying customers.

Data sources

npm registry

The full packument (registry.npmjs.org/<name>) includes every published version, the time map (first publish + per-version publish timestamps), maintainers, license, repository URL, and a deprecated flag on the version object when the maintainer has deprecated it. This actor requests the full packument rather than the abbreviated (

Accept: application/vnd.npm.install-v1+json

npm downloads

api.npmjs.org/downloads/point/last-week/<name> is a separate, also-public endpoint. If it fails or the package is too new to have download data, this actor emits null for weeklyDownloads — it does not treat a lookup failure as zero downloads.

PyPI JSON API

Returns the current release's metadata plus a releases map keyed by every version ever published, each with an upload timestamp — used here to compute publishedAt (earliest) and lastPublishAt (most recent).

PyPI does not expose two signals npm does, and this actor is explicit about that rather than guessing:

• No download counts. PyPI's JSON API has no download-count field (BigQuery and pypistats.org are separate rate-limited services this actor doesn't call). weeklyDownloads is always null for PyPI packages.
• No maintainer account list. PyPI exposes a single free-text maintainer string, not a list of registered accounts like npm's maintainers array. maintainerCount is always null for PyPI packages — this actor will not fabricate a count of 1 or 0 from a free-text field.

OSV.dev

POST api.osv.dev/v1/query with

{"package": {"name": ..., "ecosystem": ...}}

Input

The explicit {name, ecosystem} object form — recommended whenever you know which registry you mean:

{
  "packages": [
    { "name": "express", "ecosystem": "npm" },
    { "name": "requests", "ecosystem": "pypi" },
    { "name": "django", "ecosystem": "pypi" }
  ],
  "includeVulnerabilities": true
}

Bare strings also work, and use the ecosystem default below — but read Bare names and cross-ecosystem ambiguity first:

{
  "packages": ["express", "lodash"],
  "ecosystem": "npm",
  "includeVulnerabilities": true
}

• packages (array, required for a custom run) — bare names (uses the default ecosystem below) or {"name": ..., "ecosystem": "npm" | "pypi"} objects. Duplicates (same ecosystem + name) are deduped automatically. Leave this out entirely and the actor runs a small built-in example set (express, lodash, requests, django), all given with an explicit ecosystem — useful for a first test run.
• ecosystem (string, default npm) — default ecosystem applied to any bare-string entry in packages. This is still a guess on this actor's part, not a statement from you — see below.

Bare names and cross-ecosystem ambiguity

This is the one gotcha that matters most in this actor. npm and PyPI are separate namespaces with no coordination between them — the same string can be registered as two completely unrelated packages, one per registry. The most damaging version of this is a small, low-download package on one registry squatting the name of a much bigger, unrelated package on the other.

fastapi is a real example, verified live: on PyPI it's the real, enormously popular FastAPI web framework. On npm it's an unrelated, near-abandoned package (a handful of weekly downloads, ISC license, nothing to do with the Python framework). Ask this actor for a bare "fastapi" with no ecosystem stated and, before this behavior existed, it silently returned the npm row — wrong answer, presented with total confidence, for a paid data product.

To fix that, every bare-name entry is checked against the other ecosystem too, one extra request per bare name. If the name resolves on both:

• the row for the ecosystem actually queried is still the only row returned (no duplicate rows per ecosystem — that would inflate a pay-per-result bill for padding, not signal),
• nameAmbiguous is true on that row,
• alsoFoundIn lists the other ecosystem(s) the name also exists on,
• a WARNING-level log line names both ecosystems and tells you how to disambiguate.

Worked example — {"packages": ["fastapi"], "ecosystem": "npm"}:

{
  "ecosystem": "npm",
  "name": "fastapi",
  "found": true,
  "latestVersion": "0.0.8",
  "license": "ISC",
  "weeklyDownloads": 676,
  "nameAmbiguous": true,
  "alsoFoundIn": ["pypi"],
  "...": "other fields as usual"
}

Run log:

WARN Ambiguous name "fastapi": it also exists on pypi as an unrelated package. This row was resolved as npm/fastapi. If that's not the one you meant, disambiguate by passing this entry as { "name": "fastapi", "ecosystem": "..." } instead of a bare string.

To skip the probe entirely and get exactly the package you mean, use the explicit object form — {"name": "fastapi", "ecosystem": "pypi"}. Any entry that names its own ecosystem short-circuits the cross-ecosystem check: no extra request, and nameAmbiguous is always false for that row, because you already told this actor which registry you meant.

nameAmbiguous and alsoFoundIn are present on every row (false / [] when there's nothing to flag) — never omitted, never null.

• includeVulnerabilities (boolean, default true) — when on, queries OSV.dev for each package and populates vulnerabilityCount and vulnerabilities. Adds one extra request per package.
• maxPackages (integer, default 500) — caps how many packages are processed in one run after deduping.

Output

One dataset item per requested package. Real sample rows from an actual run (includeVulnerabilities: true):

{
  "ecosystem": "npm",
  "name": "express",
  "found": true,
  "latestVersion": "5.2.1",
  "description": "Fast, unopinionated, minimalist web framework",
  "license": "MIT",
  "homepage": "https://expressjs.com/",
  "repositoryUrl": "https://github.com/expressjs/express",
  "publishedAt": "2010-12-29T19:38:25.450Z",
  "lastPublishAt": "2025-12-01T20:49:43.268Z",
  "maintainerCount": 5,
  "versionCount": 288,
  "weeklyDownloads": 125244975,
  "deprecated": false,
  "nameAmbiguous": false,
  "alsoFoundIn": [],
  "vulnerabilityCount": 5,
  "vulnerabilities": [
    { "id": "GHSA-cm5g-3pgc-8rg4", "aliases": ["CVE-2024-10491"], "severity": "MODERATE", "summary": "Express ressource injection" },
    { "id": "GHSA-jj78-5fmv-mv28", "aliases": ["CVE-2024-9266"], "severity": "LOW", "summary": "Express Open Redirect vulnerability" }
  ],
  "errors": [],
  "scrapedAt": "2026-07-31T04:52:33.252Z"
}

{
  "ecosystem": "pypi",
  "name": "requests",
  "found": true,
  "latestVersion": "2.34.2",
  "description": "Python HTTP for Humans.",
  "license": "Apache-2.0",
  "homepage": "https://requests.readthedocs.io",
  "repositoryUrl": "https://github.com/psf/requests",
  "publishedAt": "2011-02-14T08:49:42.641660Z",
  "lastPublishAt": "2026-05-14T19:25:26.443000Z",
  "maintainerCount": null,
  "versionCount": 160,
  "weeklyDownloads": null,
  "deprecated": null,
  "nameAmbiguous": false,
  "alsoFoundIn": [],
  "vulnerabilityCount": 16,
  "vulnerabilities": [
    { "id": "GHSA-9hjg-9r4m-mvj7", "aliases": ["CVE-2024-47081"], "severity": "MODERATE", "summary": "Requests vulnerable to .netrc credentials leak via malicious URLs" },
    { "id": "GHSA-9wx4-h78v-vm56", "aliases": ["CVE-2024-35195"], "severity": "MODERATE", "summary": "Requests Session object does not verify requests after making first request with verify=False" }
  ],
  "errors": [],
  "scrapedAt": "2026-07-31T04:52:33.253Z"
}

Note maintainerCount and weeklyDownloads are null for the PyPI row — that's the ecosystem gap described above, not a failed lookup. A package that doesn't exist on the registry gets "found": false with every metadata field null and a message in errors, rather than being silently dropped from the dataset.

Run summary

At the end of a run, a per-package status summary is written to the default key-value store under PACKAGE_SUMMARY — which packages resolved, which were not found, and any fetch errors. The run only fails (non-zero exit) when every requested package fails to resolve — one bad package name in a long list won't sink the run.

Use cases

• Dependency auditing — feed your package.json/requirements.txt dependency list in and get back license, maintenance status, and known vulnerabilities for every package in one pass.
• Supply-chain monitoring — re-run weekly against a fixed list of packages your organization depends on to catch newly disclosed vulnerabilities or a package going deprecated.
• Package research — compare download volume, maintainer count, and release cadence across a set of candidate libraries before adopting one.
• Registry analytics — build a dataset of package metadata across npm/PyPI for trend analysis (license distribution, publish frequency, ecosystem health) without writing registry-API glue code yourself.

Pricing

This actor is billed pay-per-result: one dataset item = one package looked up. Enabling or disabling includeVulnerabilities does not change pricing, only the amount of data returned per item.

Limitations (please read)

• Only npm and PyPI are supported. No RubyGems, Maven, crates.io, Go modules, or other ecosystems — those would each need their own normalizer and haven't been built or verified.
• Vulnerability data reflects "any version currently known affected," not your exact installed version. OSV.dev's package-level query (no version pinned) returns every vulnerability with an affected-version range that includes at least one published version — it does not tell you whether the specific version you have installed is in that range. Cross-reference latestVersion and the advisory's affected-range detail (visible on the linked GHSA/CVE page) if you need per-version precision.
• maintainerCount and weeklyDownloads are always null for PyPI. PyPI's public JSON API doesn't expose either signal — see Data sources above. This is a gap in what PyPI's API offers, not something this actor can recover without a separate paid/rate-limited service.
• deprecated means different things per ecosystem. For npm it's the deprecated field a maintainer can set on a version. For PyPI it reflects a release being yanked (pulled from install resolution) — PyPI has no separate "deprecated" concept, so a PyPI package that's simply unmaintained but not yanked will show deprecated: null, not true.
• License strings are not normalized to SPDX. npm and PyPI both allow free-text license fields; this actor passes through what the registry reports (falling back to PyPI's license classifiers when the free-text field is empty) rather than attempting SPDX normalization.
• The cross-ecosystem ambiguity probe only checks existence, not relevance. It answers "does this name also exist elsewhere" — it does not try to guess which of the two same-named packages you actually meant. If nameAmbiguous comes back true, treat it as "verify this before trusting it," not as this actor resolving the ambiguity for you.
• The probe adds one extra request per bare-name package (skipped entirely for entries using the explicit {name, ecosystem} form). For large bare-name batches this roughly doubles registry request volume, though it does not change pay-per-result pricing, which is billed per dataset row, not per request.
• Data reflects a single point-in-time snapshot of each registry and of OSV.dev at the moment the actor ran — new versions and newly disclosed vulnerabilities won't appear until the next run.