# Deps Dev Open Source Insights Scraper (`real-estate-api/deps-dev-open-source-insights-scraper`) Actor

deps.dev Open Source Insights Scraper is a Python tool designed to extract software package dependencies, license distributions, security advisories, and supply chain graphs from Open Source Insights (deps.dev). It exports structured package datasets into clean CSV or JSON formats

- **URL**: https://apify.com/real-estate-api/deps-dev-open-source-insights-scraper.md
- **Developed by:** [real-estate-api](https://apify.com/real-estate-api) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.99 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### deps.dev Open Source Insights Scraper — Package Versions, Licenses, and Dependency Graphs

### Overview

deps.dev Open Source Insights Scraper collects publicly available information about open-source packages and projects, including package versions, license information, dependency graphs, and project statistics.

Choose a collection mode, provide package names or project identifiers, and collect structured records in an Apify dataset. Results can be exported as JSON, CSV, or Excel for package research, dependency analysis, and software supply chain reviews.

### Features

- **Package Version Discovery:** Collect available versions and publication dates for supported package ecosystems.
- **Version Details:** Retrieve license information, advisories, related projects, and other available version metadata.
- **Dependency Graphs:** Collect dependency nodes, relationships, and version requirements.
- **Project Insights:** Retrieve available project statistics, license information, and OpenSSF Scorecard details.
- **Multiple Ecosystems:** Supports npm, PyPI, Go, Maven, Cargo, and NuGet.
- **Default Version Selection:** Use the latest version marked as the default when no specific version is supplied.
- **Duplicate Filtering:** Keep duplicate version keys, project identifiers, and graph records from being stored more than once within a run.
- **Structured Exports:** Organize package and project records for further analysis.

### Use Cases

- **Package Research:** Review available releases and identify default or deprecated versions.
- **License Review:** Examine license information associated with a package version.
- **Dependency Analysis:** Explore dependency relationships and version requirements.
- **Software Supply Chain Reviews:** Examine available project and security-related metadata.
- **Project Assessment:** Review repository statistics and available OpenSSF Scorecard information.
- **Package Inventory:** Export package information for reporting and comparisons across collection runs.

### Collection Modes

| Mode | Description |
|---|---|
| `versions` | Collects available versions for a specified package. |
| `version` | Collects details for a specific version or the default version. |
| `dependencies` | Collects dependency graph nodes and edges for a package version. |
| `project` | Collects information about specified projects. |

The `includeDependencies` option can also collect a dependency graph alongside another applicable collection mode.

### Input Parameters

| Parameter | Type | Default | Description |
|---|---|---|---|
| `mode` | String | `versions` | Collection mode: `versions`, `version`, `dependencies`, or `project`. |
| `packages` | Array | `[{"system":"npm","name":"react"}]` | Package ecosystem and package name. Supported systems: `npm`, `pypi`, `go`, `maven`, `cargo`, and `nuget`. |
| `version` | String | Empty | Optional package version. When omitted, the actor selects the latest version marked as the default when available. |
| `projects` | Array | `[]` | Project identifiers to collect in project mode. |
| `includeDependencies` | Boolean | `false` | Also collect the dependency graph for the selected version when applicable. |
| `maxItems` | Integer | `100` | Maximum number of dataset rows, including error records. Minimum: 1; maximum: 1000. |

### Example Input

```json id="f4k2mz"
{
  "mode": "version",
  "packages": [
    {
      "system": "npm",
      "name": "react"
    }
  ],
  "version": "18.2.0",
  "includeDependencies": false,
  "maxItems": 100
}
```

### What Data Does This Actor Extract?

The dataset contains records for package versions, version details, dependency graph components, and project information. Available fields depend on the selected mode and the data returned for each package or project.

#### Extracted Data Fields

| Field | Type | Description |
|---|---|---|
| `versionKey` | Object | Package ecosystem, package name, and version identifier. |
| `publishedAt` | String | Publication timestamp for a package version. |
| `isDefault` | Boolean | Indicates whether the version is marked as the default. |
| `isDeprecated` | Boolean | Indicates whether the version is deprecated. |
| `deprecatedReason` | String | Available explanation for a version's deprecation. |
| `licenses` | Array | License information associated with a version. |
| `advisoryKeys` | Array | Available advisory identifiers. |
| `links` | Object | Related links included in version details, when available. |
| `registries` | Array | Registry information associated with a version, when available. |
| `relatedProjects` | Array | Projects associated with the package version. |
| `slsaProvenances` | Array | Available SLSA provenance information. |
| `attestations` | Array | Available attestation information. |
| `graphPart` | String | Identifies a dependency graph record as a `node` or `edge`. |
| `relation` | String | Relationship represented by a dependency graph edge. |
| `requirement` | String | Version requirement associated with a dependency relationship. |
| `projectKey` | Object | Project identifier and associated project information. |
| `starsCount` | Integer | Available project star count. |
| `forksCount` | Integer | Available project fork count. |
| `openIssuesCount` | Integer | Available count of open project issues. |
| `license` | String | Project license information when available. |
| `overallScore` | Number | Available overall project score. |
| `projectSummary` | String | Project summary text when available. |
| `mode` | String | Collection mode used to produce the record. |
| `scraped_at` | String | Timestamp when the record was collected. |

### Example Output

The following is an illustrative package version record.

```json id="9w2rkc"
{
  "versionKey": {
    "system": "NPM",
    "name": "react",
    "version": "18.2.0"
  },
  "publishedAt": "2022-06-14T19:46:38Z",
  "isDefault": false,
  "isDeprecated": false,
  "deprecatedReason": "",
  "mode": "versions",
  "scraped_at": "2026-10-12T00:00:00Z"
}
```

### Working with Your Dataset

After collection, review the dataset and export records in your preferred format:

- **JSON:** For structured data processing and analysis.
- **CSV:** For spreadsheet reviews and reporting.
- **Excel:** For comparing package versions, licenses, and project information.

You can compare separate exports to monitor changes in package versions, dependency relationships, and available project statistics.

### Important Notes

- Supported package ecosystems are npm, PyPI, Go, Maven, Cargo, and NuGet.
- Package and project records may contain different fields depending on the selected mode.
- The `maxItems` setting limits the total number of dataset rows, including error records.
- When the version list exceeds the remaining item capacity, the actor prioritizes the newest entries according to the order supplied in the collected version list.
- Missing or unsupported package information may prevent a record from being collected successfully.
- Dependency graphs contain separate node and edge records.
- Project details discovered through related project information may be collected when applicable.

### FAQ

**Can I collect packages from different ecosystems?**

Yes. Specify the appropriate system and package name for each entry in `packages`.

**Can I collect a specific package version?**

Yes. Set `version` to the version you want to inspect.

**Can I collect dependency graphs along with version details?**

Yes. Enable `includeDependencies` when you want the dependency graph included alongside another applicable mode.

**What information is available in project mode?**

Depending on the project, records can include repository statistics, license information, and available OpenSSF Scorecard details.

**Can I export the collected information to CSV or Excel?**

Yes. The dataset supports JSON, CSV, and Excel exports.

**How many records can one run collect?**

The default maximum is 100 dataset rows, and the configured maximum cannot exceed 1000 rows.

### Changelog

#### 1.0.0

- Added package version collection and version detail records.
- Added license and advisory information.
- Added dependency graph collection.
- Added project information collection.
- Added configurable record limits and duplicate filtering.

### Disclaimer

This actor is an independent tool and is not affiliated with, endorsed by, sponsored by, or officially associated with deps.dev, Google Open Source Insights, or their owners. Users are responsible for ensuring that their collection and use of data comply with applicable laws and relevant terms.

### Is Web Scraping Legal?

Collecting information that is already public is generally legal. Scraping is not the same thing as hacking, and there is no single law that bans it. The limits are the usual ones: do not break into accounts, do not take private or confidential data, and be careful with personal data and copyrighted works. A site's terms can also restrict how you access it. Facts such as a public name, price, or rating are not the same thing as republishing someone's article or photo.

This actor collects information from public pages. It does not log in and it does not bypass access controls.

For a longer explanation, see Apify's article [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/).

# Actor input Schema

## `mode` (type: `string`):

What to collect. Package versions lists releases. Version detail returns licenses and advisories for one version. Dependency graph returns nodes and edges. Project info reads a GitHub project from deps.dev.

## `packages` (type: `array`):

Packages to look up. Each object needs system (npm, pypi, go, maven, cargo, nuget) and name. Names are sent percent-encoded, so scoped npm names and module paths with slashes are fine.

## `version` (type: `string`):

Optional version. Leave empty to use the latest version marked isDefault. Used by version detail, the dependency graph, and project lookup when no project id is set.

## `projects` (type: `array`):

Project ids for project mode, one per line, such as github.com/facebook/react. When this list is empty, project ids come from the version detail relatedProjects list.

## `includeDependencies` (type: `boolean`):

Also fetch the dependency graph for the chosen version. When version is empty, the graph uses the default version. Ignored when mode is already Dependency graph.

## `maxItems` (type: `integer`):

Max dataset rows for the run, including error rows. Free cap 1000. When a package has more versions than the remaining cap, the newest entries from the API list are kept.

## `requestDelaySecs` (type: `number`):

Polite pacing. Seconds to wait between requests. Minimum 0.2. The public API did not return rate-limit headers in a short check, so this delay is the throttle.

## `timeoutSecs` (type: `integer`):

Stop the run after this many seconds, even if more packages are still queued.

## `requestTimeoutSecs` (type: `integer`):

How long to wait for a single response before retrying or recording that request as an error.

## `proxyConfiguration` (type: `object`):

Optional proxy for requests. Default is no proxy. Turn on Apify Proxy only if your network is blocked.

## Actor input object example

```json
{
  "mode": "versions",
  "packages": [
    {
      "system": "npm",
      "name": "react"
    }
  ],
  "version": "18.2.0",
  "projects": [
    "github.com/facebook/react"
  ],
  "includeDependencies": false,
  "maxItems": 100,
  "requestDelaySecs": 0.2,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `overview` (type: `string`):

Version key, publish time, default and deprecation flags, mode, and source URL.

## `errors` (type: `string`):

Failed lookups with the input, HTTP status, and message.

## `dataset` (type: `string`):

Every row from the run, without a table view applied.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "packages": [
        {
            "system": "npm",
            "name": "react"
        }
    ],
    "version": "",
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("real-estate-api/deps-dev-open-source-insights-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "packages": [{
            "system": "npm",
            "name": "react",
        }],
    "version": "",
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("real-estate-api/deps-dev-open-source-insights-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "packages": [
    {
      "system": "npm",
      "name": "react"
    }
  ],
  "version": "",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call real-estate-api/deps-dev-open-source-insights-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,real-estate-api/deps-dev-open-source-insights-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/4TKJoEgDDdgqlgVf8/builds/r6dB0nKIIiPX2xrOU/openapi.json