Npm Registry Scraper

Extract comprehensive metadata, download statistics, and dependency information from the NPM (Node Package Manager) registry. This Actor provides structured data about JavaScript packages, including version history, maintainer details, and popularity metrics.

Features

• Package Metadata Extraction: Get package name, description, version, license, keywords, and more
• Download Statistics: Retrieve monthly and weekly download counts for popularity analysis
• Dependency Mapping: Extract dependencies, devDependencies, peerDependencies, and optionalDependencies
• Maintainer Information: Collect author and maintainer details with contact information
• Repository Links: Capture homepage, repository URL, and bug tracker links
• Search Capability: Discover packages by search query with relevance scoring
• Bulk Processing: Process multiple packages or search results in a single run
• Full Version History: Optionally include complete version metadata for each package

Use Cases

1. Package Research & Analysis

Analyze the JavaScript ecosystem by extracting metadata from popular packages. Compare download statistics, identify trending libraries, and research dependency trees for security audits.

2. Dependency Auditing

Generate comprehensive reports of package dependencies for your projects. Identify outdated packages, security vulnerabilities, and license compatibility issues across your dependency tree.

3. Market Intelligence

Track package popularity over time with download statistics and dependent counts. Identify emerging libraries and monitor competitor packages in your niche.

4. Developer Tooling

Build internal tools that require NPM metadata. Create package recommendation engines, license compliance checkers, or dependency visualization tools.

Input Parameters

Parameter	Type	Required	Default	Description
packageNames	array	No	[]	List of package names to scrape (e.g., ["react", "lodash"])
searchQuery	string	No	-	Search NPM registry for packages matching this query
maxSearchResults	integer	No	20	Maximum search results (1-250)
includeVersions	boolean	No	false	Include all version metadata (increases data size)
includeDependencies	boolean	No	true	Include package dependencies
maxResults	integer	No	100	Maximum total packages to process (1-1000)
proxyConfiguration	object	No	Apify proxy	Proxy configuration for requests

Output Format

Example Package Record

{
  "name": "crawlee",
  "version": "3.16.0",
  "description": "The scalable web crawling and scraping library for JavaScript/Node.js",
  "keywords": ["apify", "headless", "chrome", "puppeteer", "crawler"],
  "license": "Apache-2.0",
  "homepage": "https://crawlee.dev",
  "repository": "git+https://github.com/apify/crawlee.git",
  "author": {
    "name": "Apify",
    "email": "support@apify.com",
    "url": "https://apify.com"
  },
  "maintainers": [
    {
      "username": "b4nan",
      "email": "martinadamek59@gmail.com"
    }
  ],
  "downloads": {
    "monthly": 299733,
    "weekly": 64779
  },
  "dependents": 102,
  "updated": "2026-03-21T06:45:19.017Z",
  "published": "2022-01-15T10:30:00.000Z",
  "dependencies": {
    "@apify/log": "^2.5.0",
    "@crawlee/types": "^3.16.0"
  },
  "devDependencies": {
    "typescript": "^5.0.0"
  },
  "engines": {
    "node": ">=16.0.0"
  },
  "dist": {
    "integrity": "sha512-...",
    "shasum": "abc123...",
    "tarball": "https://registry.npmjs.org/crawlee/-/crawlee-3.16.0.tgz",
    "fileCount": 150,
    "unpackedSize": 450000
  },
  "score": {
    "final": 1484.43,
    "quality": 1.0,
    "popularity": 1.0,
    "maintenance": 1.0
  }
}

Pricing

This Actor uses Pay Per Event pricing:

• $0.002 per package scraped (minimum charge: $0.001)

Example costs:

• 100 packages: $0.20
• 500 packages: $1.00
• 1000 packages: $2.00

FAQ

Q: Do I need an NPM account or API key? A: No. This Actor uses the public NPM registry API which requires no authentication.

Q: Can I scrape private packages? A: No, this Actor only accesses public packages from the NPM registry.

Q: What is the rate limit? A: The Actor respects NPM registry rate limits with built-in delays between requests (500ms). Processing is intentionally throttled to be respectful to the registry.

Q: How recent is the download data? A: Download statistics are updated daily by NPM. The data reflects the most recent 30-day rolling window.

Q: Can I get historical download data? A: The NPM registry API only provides current monthly and weekly download counts. Historical data is not available through the public API.

Limitations

• Maximum 1000 packages per run
• Download statistics are approximate and may have a 24-hour delay
• Private packages and scoped packages requiring authentication are not supported
• Very old packages may have incomplete metadata
• Rate limiting is applied to prevent registry abuse

Support

Having issues? Open an issue on this Actor's Apify page. Include the package names or search query that caused the problem.

---

Built for the Apify Store by P. McD0nald