# Puppet Forge Modules Scraper (`b2b_lead_generation/puppet-forge-modules-scraper`) Actor

Search Puppet Forge modules by keyword, view module details, or list release histories. Extract metadata into structured datasets.

- **URL**: https://apify.com/b2b_lead_generation/puppet-forge-modules-scraper.md
- **Developed by:** [B2B Lead Generation](https://apify.com/b2b_lead_generation) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.99 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Puppet Forge Modules Scraper: Puppet modules search

Puppet Forge Modules Scraper collects Puppet module records and version histories. You can search by keyword, inspect specific modules using owner-name slugs, or retrieve complete release lists. All scraped results land in a clean Apify dataset ready to export as JSON, CSV, or Excel.

### Use cases

- **Module search**: Find modules matching keywords like `nginx` or `apache`, sorted by rank, downloads, or recent activity.
- **Catalog audit**: Keep structured records of module slugs, overall download counts, endorsements, maintainers, and current release details.
- **Release history**: Track version numbers, file hashes, and individual download stats for specific modules over time.
- **Scheduled exports**: Run automated extraction workflows on a schedule to deliver updated datasets directly to CSV or Excel.
- **Deprecation watch**: Monitor module lifecycle status and track replacements or deprecation dates automatically.

### Modes

- **Search (`search`)**: Search for modules matching your keywords. You can sort by rank, overall downloads, or latest release, and standard pagination automatically handles additional pages.
- **Module detail (`detail`)**: Focus on specific modules using their slug. If a requested module is unavailable, it is recorded cleanly without interrupting the rest of your run.
- **List releases (`releases`)**: Retrieve full release histories for specified module slugs. Each release version is saved as an individual row in your dataset.

### Input

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `mode` | string | `search` | Choose search, module detail, or list releases. |
| `searchQueries` | array | `["nginx"]` | Keywords for search mode. |
| `moduleSlugs` | array | `[]` | Slugs such as `puppetlabs-apache`. Used in detail and releases mode. |
| `sortBy` | string | `rank` | Search sorting options: `rank`, `downloads`, or `latest_release`. |
| `maxItems` | integer | `100` | Target maximum rows to save. |
| `requestDelaySecs` | number | `0.3` | Delay between requests in seconds to manage pacing. |
| `timeoutSecs` | integer | `300` | Overall run timeout in seconds. |
| `requestTimeoutSecs` | integer | `30` | Single request timeout in seconds. |
| `proxyConfiguration` | object | `{}` | Optional proxy settings if custom network routing is needed. |

#### Example input

```json
{
  "mode": "search",
  "searchQueries": ["nginx"],
  "moduleSlugs": [],
  "sortBy": "rank",
  "maxItems": 100,
  "requestDelaySecs": 0.3,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30
}

```

For module detail or release lists, supply your target slugs:

```json
{
  "mode": "detail",
  "moduleSlugs": ["puppetlabs-apache"],
  "maxItems": 100,
  "requestDelaySecs": 0.3,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30
}

```

### What data does this actor extract?

A standard search result for a module like `nginx` extracts fields structured like this:

```json
{
  "slug": "puppet-nginx",
  "name": "nginx",
  "downloads": 48759335,
  "created_at": "2016-09-02 02:48:40 -0700",
  "updated_at": "2026-10-04 04:33:15 -0700",
  "deprecated_at": null,
  "supported": false,
  "endorsement": "validated",
  "module_group": "base",
  "owner": {
    "slug": "puppet",
    "username": "puppet"
  },
  "premium": false,
  "followers_count": 2,
  "current_release": {
    "slug": "puppet-nginx-8.3.1",
    "version": "8.3.1"
  },
  "mode": "search",
  "scraped_at": "2026-10-12T00:00:00+00:00"
}

```

| Field | Type | Description |
| --- | --- | --- |
| `slug` | string | Unique module or release slug. |
| `name` | string | Name of the module. |
| `downloads` | integer | Total download count for the module or specific release. |
| `created_at` | string | Creation timestamp. |
| `updated_at` | string | Last updated timestamp. |
| `deprecated_at` | string | Deprecation timestamp, if applicable. |
| `endorsement` | string | Status endorsement level (e.g., supported, validated). |
| `owner` | object | Details about the maintainer or owner account. |
| `current_release` | object | Metadata about the latest release version. |
| `version` | string | Specific release version tag (releases mode). |
| `file_sha256` | string | File checksum for the release archive. |
| `mode` | string | Mode used during extraction (`search`, `detail`, or `releases`). |
| `scraped_at` | string | UTC timestamp recorded when the data was saved. |
| `error` | string | Error message if a specific item failed to load. |
| `input` | string | The requested query or slug associated with an error. |
| `status` | integer | Status code returned during a failed item lookup. |

### FAQ

**How fresh is the extracted data?**
Every row reflects the exact information available when your run executes. The `scraped_at` timestamp marks the precise time the record was stored.

**What happens if a requested module slug doesn't exist?**
If a module slug cannot be found, the actor logs an error row containing the status and error details, then immediately continues processing the rest of your list.

**How many records can I extract in a single run?**
You can set `maxItems` according to your needs. The actor will gather results page by page until it hits your item limit or runs out of results.

**Can I export data to Excel or CSV?**
Yes. Once your run finishes, you can export your dataset directly from Apify as JSON, CSV, or Excel.

### Trademark Notice

Puppet, Puppet Forge, and related names, logos, and trademarks belong to their respective owners. Their use here is strictly to identify the software ecosystem and data context, and does not imply any official endorsement, sponsorship, or affiliation.

### Disclaimer

This Actor is an independent tool developed by B2B Lead Generation and is not affiliated with or endorsed by Puppet. All data processed by this tool comes from standard public web sources. Users are responsible for ensuring their usage complies with relevant terms, rate guidelines, and licensing requirements.

### Is web scraping legal?

This Actor only collects publicly available data without bypassing any access controls or technical barriers. Scraping public data is widely recognized as legal, but how you store and handle that data remains your responsibility:

- **Personal data:** If your extractions include personal details (such as developer names or contact info), privacy regulations like GDPR or CCPA may apply. Only process personal data with a valid business or legal justification.
- **Copyright:** Always ensure you respect intellectual property rights when storing or displaying scraped module descriptions or codebase assets.
- **Usage policies:** Be mindful of reasonable request pacing and standard data retention practices.
- **Legal advice:** This guide provides general context, not legal advice. If you have questions about your specific implementation, consult a legal professional.

To learn more, read Apify's full guide: [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/)

# Actor input Schema

## `mode` (type: `string`):

Search modules by keyword, fetch one module by owner-name slug, or list releases for a slug. Only the public Forge v3 routes are called.

## `searchQueries` (type: `array`):

Keywords sent as the query parameter to GET /v3/modules. Used in search mode. One keyword per line.

## `moduleSlugs` (type: `array`):

owner-name slugs for detail and releases, such as puppetlabs-apache. Ignored in search mode.

## `sortBy` (type: `string`):

Search sort. rank, downloads, or latest_release. Sent as sort_by on GET /v3/modules. Ignored for detail and releases.

## `maxItems` (type: `integer`):

Max results (free cap 1000). Stops the run once this many dataset rows are collected, including error rows.

## `requestDelaySecs` (type: `number`):

Polite pacing. Seconds to wait between requests. Minimum 0.3. Requests run one at a time.

## `timeoutSecs` (type: `integer`):

Stop the run after this many seconds, even if more queries, slugs, or pages are still queued.

## `requestTimeoutSecs` (type: `integer`):

How long to wait for a single Forge API response before that request is recorded as an error.

## `proxyConfiguration` (type: `object`):

Optional proxy for Forge API requests. Leave Apify Proxy off unless your IP is blocked.

## Actor input object example

```json
{
  "mode": "search",
  "searchQueries": [
    "nginx"
  ],
  "moduleSlugs": [
    "puppetlabs-apache"
  ],
  "sortBy": "rank",
  "maxItems": 100,
  "requestDelaySecs": 0.3,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `datasetUrl` (type: `string`):

Default dataset for this run.

## `overviewUrl` (type: `string`):

Dataset rows in the overview table.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "nginx"
    ],
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("b2b_lead_generation/puppet-forge-modules-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQueries": ["nginx"],
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("b2b_lead_generation/puppet-forge-modules-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "nginx"
  ],
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call b2b_lead_generation/puppet-forge-modules-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,b2b_lead_generation/puppet-forge-modules-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/TQJJ49fS1XehWowIo/builds/BXnKIDmA7BfpTGQTL/openapi.json