npm + PyPI Package Scraper

A package index exists to be read, so npm and PyPI serve their metadata to anyone: no key, no login, no anti-bot layer to work around. What they do not agree on is shape. npm's package document has no version field at all, the current one sits behind dist-tags.latest, while PyPI puts it plainly in info.version.

This flattens both into one record each: version, license, author, repository, homepage, keywords, and for npm the last month's download count. Same output shape for both registries, so you can put them in one table.

What you get back

{
  "ok": true,
  "registry": "npm",
  "name": "zustand",
  "version": "5.0.3",
  "description": "🐻 Bear necessities for state management in React",
  "author": "Paul Henschel",
  "homepage": "https://github.com/pmndrs/zustand",
  "repository": "https://github.com/pmndrs/zustand",
  "license": "MIT",
  "keywords": ["react", "state", "manager", "management"],
  "score": 0.4732,
  "monthlyDownloads": 6218440,
  "url": "https://www.npmjs.com/package/zustand"
}

repository is normalised to something you can click — git+ssh://git@github.com/x/y.git comes back as https://github.com/x/y. score is npm's own search relevance number and is only present on search results. PyPI rows have the same keys with registry: "pypi".

Two ways to ask

Search npm by keyword. Put terms in searchQuery and set maxItems.

{
  "registry": "npm",
  "searchQuery": "react state management",
  "maxItems": 25,
  "includeDownloads": true
}

Look up exact names. Works on both registries, and this is the only mode PyPI supports.

{
  "registry": "pypi",
  "packageNames": ["requests", "fastapi", "pydantic"]
}

Scoped npm names like @types/node work fine in packageNames.

Field	Default	Notes
registry	npm	npm or pypi.
searchQuery	—	npm only. Ignored for PyPI.
packageNames	[]	Exact names. Both registries.
includeDownloads	true	npm only. One extra request per package.
maxItems	50	Caps npm search results, 1 to 1,000. Doesn't apply to name lookups.
proxyConfiguration	off	Neither registry has anti-bot. Leave it off.

Limits you should know before you plan a run

• PyPI has no keyword search. There is no clean public search endpoint, so PyPI is exact names only. Pass a searchQuery with registry: "pypi" and you get a BAD_INPUT diagnostic row telling you to use packageNames instead. It isn't billed.
• license is null on npm search rows. npm's search response simply doesn't include it. If you need licenses, take the names from the search run and put them through packageNames — the full package document has it.
• monthlyDownloads is npm-only. PyPI publishes no public download endpoint. It's also null if you switch includeDownloads off, or if that extra request fails for one package. When that happens the log names the package.
• A name that doesn't exist gets a NOT_FOUND diagnostic row and the run continues with the rest. Not billed.
• Results are deduplicated on registry plus lowercased name, so listing a package twice charges once.

Billing

$2.00 per 1,000 packages, so $0.002 each, with no run-start fee. One charge per package actually returned. Diagnostic rows are free.

Note that with includeDownloads on, npm packages take two HTTP requests instead of one. That changes how long the run takes, not what you pay.

Delivery

The dataset exports as JSON, CSV, Excel or XML. notionConnector plus notionParentId will additionally write one Notion page per package after the run finishes. Authorize the connector once under Settings → API & Integrations → MCP connectors; leave the fields empty to skip it.

Common jobs

License audits, mostly. Paste the dependency list out of a package.json or a requirements.txt into packageNames, export the CSV, sort by license, and you can see the copyleft ones without opening 200 registry pages.

It also settles arguments about which library to pick. Three names in packageNames gives you three rows with monthlyDownloads, version and repository lined up next to each other.

Point it at someone else's packages on a schedule and monthlyDownloads turns into a growth curve.