# Rubygems Scraper (`kawsar/rubygems-scraper`) Actor

Collect public Ruby gem search results, stats, version lists, and reverse dependencies. Customize queries and limits, then export your structured dataset directly as JSON, CSV, or Excel for easy analysis and tracking.

- **URL**: https://apify.com/kawsar/rubygems-scraper.md
- **Developed by:** [Kawsar](https://apify.com/kawsar) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.99 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## RubyGems Scraper: Ruby gems search and stats

RubyGems Scraper collects gem search results, gem stats, version lists, and reverse dependencies. Each run writes a flat dataset you can export as JSON, CSV, or Excel.

Requests are sequential, paced, and capped so a refresh stays inside a polite rate limit.

### Use cases

- **Gem search**: export matching gems for a keyword such as json, including downloads, license, and project links
- **Dependency review**: load exact gems such as rails or rack and keep the current version stats
- **Release tracking**: pull the version list for a gem and see downloads, platform, and Ruby version requirements
- **Impact checks**: list reverse dependency names for a gem before you ship a breaking change
- **Scheduled inventories**: rerun the same input on a timer and diff downloads or new versions
- **Open data export**: hand a CSV of gem stats to a spreadsheet without manual copying

### Input

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `searchQueries` | array | `["json"]` | Keywords sent for search, one per line. Pages continue until a page is empty or the item cap is reached. |
| `gemNames` | array | empty | Exact gem names to look up details for. |
| `includeVersions` | boolean | `false` | Also fetch the version list for each gem name. |
| `includeReverseDependencies` | boolean | `false` | Also fetch reverse dependency names for each gem name. |
| `maxItems` | integer | `100` | Maximum successful records in one run. Minimum 1, maximum 1000. |
| `requestDelaySecs` | number | `0.15` | Pause between calls. Values below 0.15 are raised to 0.15. |
| `timeoutSecs` | integer | `300` | Overall time limit in seconds. |
| `requestTimeoutSecs` | integer | `30` | Per-request timeout in seconds. |
| `proxyConfiguration` | object | `{"useApifyProxy": false}` | Optional proxy configuration. Direct connection is default. |

#### Example input

```json
{
  "searchQueries": ["json"],
  "gemNames": ["rails"],
  "includeVersions": false,
  "includeReverseDependencies": false,
  "maxItems": 100,
  "requestDelaySecs": 0.15,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": { "useApifyProxy": false }
}

```

### What data does this actor extract?

Successful rows extract standard gem fields, plus `mode` and `scraped_at`. Version text from the versions view is stored in `info` so the dataset field list stays valid. Failed requests become error rows with `error`, `input`, and `status`, allowing the run to continue without halting.

```json
{
  "name": "json",
  "version": "3.0.2",
  "downloads": 1398968346,
  "version_downloads": 5363548,
  "platform": "ruby",
  "licenses": ["Ruby"],
  "project_uri": "[https://rubygems.org/gems/json](https://rubygems.org/gems/json)",
  "homepage_uri": "[https://github.com/ruby/json](https://github.com/ruby/json)",
  "documentation_uri": "[https://docs.ruby-lang.org/en/master/JSON.html](https://docs.ruby-lang.org/en/master/JSON.html)",
  "source_code_uri": "[https://github.com/ruby/json](https://github.com/ruby/json)",
  "bug_tracker_uri": "[https://github.com/ruby/json/issues](https://github.com/ruby/json/issues)",
  "changelog_uri": "[https://github.com/ruby/json/blob/master/CHANGES.md](https://github.com/ruby/json/blob/master/CHANGES.md)",
  "gem_uri": "[https://rubygems.org/gems/json-3.0.2.gem](https://rubygems.org/gems/json-3.0.2.gem)",
  "mode": "search",
  "scraped_at": "2026-10-12T00:00:00+00:00"
}

```

| Field | Type | Description |
| --- | --- | --- |
| `name` | string | Gem name. |
| `version` | string | Version string. |
| `downloads` | integer | Download count. |
| `platform` | string | Platform, usually ruby. |
| `authors` | string | Author string when available. |
| `info` | string | Summary or longer description text. |
| `licenses` | array | License names. |
| `project_uri` | string | RubyGems project page. |
| `metadata` | object | Metadata object. |
| `mode` | string | `search`, `detail`, `versions`, or `reverse_dependencies`. |
| `scraped_at` | string | UTC timestamp. |
| `error` | string | Present on failed rows. |
| `input` | string | Query or gem name that failed. |
| `status` | integer | HTTP status on failed rows. |

Duplicate records are automatically skipped. Search and detail use the gem name. Versions use gem name plus version. Reverse dependencies use the parent gem plus the dependent name.

### How it works

1. Reads search queries and gem names. Empty values are dropped.
2. Collects matching items page by page (30 results per page) until a page is empty or `maxItems` is reached.
3. Retrieves detailed metrics for each target gem name.
4. If requested, fetches the version list and reverse dependency names for each gem.
5. Pauses for at least `requestDelaySecs` between network requests to maintain a polite browsing cadence.
6. Saves rows in batches. A missing page or non-existent gem is logged as an error row while the scraper continues processing the remaining list.

### FAQ

**How fresh is the gem data?**
Each run extracts live metrics at the time of execution. You can schedule runs whenever you need a fresh export of downloads, versions, or reverse dependencies.

**How many gems can one run save?**
Set `maxItems` from 1 to 1000. The default is 100. Error rows do not count toward that limit.

**What happens if a gem name does not exist?**
The tool stores an error row with status 404 and continues with the remaining gem names.

#### Trademark Notice

RubyGems and related names, logos, and trademarks belong to their respective owners. Their use in this documentation is solely to identify the source of the data and does not imply affiliation, sponsorship, endorsement, or partnership.

#### Disclaimer

This Actor is an independent tool and is not affiliated with, endorsed by, sponsored by, or otherwise officially associated with RubyGems or its owners. All data is collected from publicly available pages. Users are responsible for complying with the source's terms of use, rate limits, and attribution or licensing requirements.

#### Is web scraping legal?

This Actor only collects publicly available data without bypassing any access controls. Scraping public data is generally legal, but how you use it matters:

- **Personal data:** If the results include personal data (for example names, emails, or profile details), privacy laws such as GDPR in the EU or CCPA in California may apply. Only collect and store personal data when you have a legitimate reason.
- **Copyright:** Some content (articles, images, descriptions) may be protected by copyright. Do not republish full copyrighted content without permission.
- **Terms of use:** Respect stated rate limits, terms of use, and any attribution or non-commercial-use requirements.
- **Legal advice:** This is general information, not legal advice. If you're unsure, consult a legal professional.

To learn more, read Apify's full guide: [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/)

# Actor input Schema

## `searchQueries` (type: `array`):

Keywords to search on the public RubyGems search endpoint. One query per line. Each query is paginated until the page is empty or the item cap is reached.

## `gemNames` (type: `array`):

Exact gem names to load from the gem detail endpoint, such as rails or rack. One name per line.

## `includeVersions` (type: `boolean`):

When enabled, also fetch the version list for each gem name.

## `includeReverseDependencies` (type: `boolean`):

When enabled, also fetch reverse dependency names for each gem name from the public reverse dependencies endpoint.

## `maxItems` (type: `integer`):

Maximum number of successful records to save in one run. The free cap is 1000.

## `requestDelaySecs` (type: `number`):

Seconds to wait between HTTP calls. The minimum is 0.15 so the run stays under the public RubyGems rate limit.

## `timeoutSecs` (type: `integer`):

Overall run time limit in seconds. The actor stops scheduling new requests after this elapses.

## `requestTimeoutSecs` (type: `integer`):

Per-request HTTP timeout in seconds.

## `proxyConfiguration` (type: `object`):

Optional proxy settings. Leave useApifyProxy false to call RubyGems directly. Enable Apify Proxy or supply your own proxy URLs only if you need another IP.

## Actor input object example

```json
{
  "searchQueries": [
    "json"
  ],
  "gemNames": [],
  "includeVersions": false,
  "includeReverseDependencies": false,
  "maxItems": 100,
  "requestDelaySecs": 0.15,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `dataset` (type: `string`):

Default dataset with one row per gem, version, reverse dependency, or error.

## `overview` (type: `string`):

Table view of gem name, version, downloads, and mode.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "json"
    ],
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("kawsar/rubygems-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQueries": ["json"],
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("kawsar/rubygems-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "json"
  ],
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call kawsar/rubygems-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,kawsar/rubygems-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/K1dd7aW2a34qaJjzY/builds/qUyRvulUJRNNbfmkq/openapi.json