# RubyGems Scraper: Gem Metadata & Version History (`arman-bd/rubygems-scraper`) Actor

Pull gem data from RubyGems: version, total and per-version downloads, licences, runtime and development dependencies, source code URI and documentation links. Bulk gems, no auth.

- **URL**: https://apify.com/arman-bd/rubygems-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** Developer tools, Automation, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.37 / 1,000 gem scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## RubyGems Scraper: Gem Metadata & Version History

![RubyGems Scraper: Latest version, download counts, licences, dependencies and every declared link for any RubyGems package](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/rubygems-scraper.jpg)

**RubyGems Scraper** turns any list of gem names into flat, structured records, latest version, total and per-version download counts, licences, runtime and development dependencies, and every link the gem declares: homepage, source, docs, changelog, bug tracker and funding.

It reads the same RubyGems metadata Bundler resolves against. **No proxy setup, no browser, no credentials to manage.** A gem lookup is a single request.

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/rubygems-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/rubygems-scraper.md
```

### What you get

| Output field | Meaning |
|---|---|
| `name`, `version`, `platform` | Gem name as RubyGems spells it, latest version, and its platform (`ruby`, `java`, …) |
| `downloads`, `versionDownloads` | All-time downloads for the gem, and downloads for the latest version alone |
| `lastRelease` | When the latest version was pushed |
| `authors`, `info` | Author line and the gem's description, verbatim from the gemspec |
| `licenses` | Declared SPDX licence identifiers |
| `yanked`, `sha` | Whether the latest version was withdrawn, plus its checksum |
| `homepageUri`, `sourceCodeUri`, `documentationUri`, `changelogUri`, `bugTrackerUri`, `mailingListUri`, `wikiUri`, `fundingUri` | The gem's declared links, each `null` when unset |
| `projectUri`, `gemUri` | RubyGems page and the direct `.gem` download URL |
| `metadata` | The raw gemspec metadata map, including flags like `rubygems_mfa_required` |
| `runtimeDependencies`, `developmentDependencies` | Dependency name and version requirement for the latest version |
| `releaseCount`, `firstRelease`, `releases` | Version history, total count, project age, and per-version detail (opt-in) |
| `scrapedAt` | Run timestamp |

A `RUN_SUMMARY` record in the key-value store holds per-run counts, the filters used, unknown gem names and any lookup that failed.

### Common use cases

- **Dependency and licence audits.** Resolve a `Gemfile.lock` into licences, maintainership and transitive requirements.
- **Adoption tracking.** `downloads` versus `versionDownloads` shows how fast a new release is actually being taken up.
- **Abandonment detection.** A large `releaseCount` with an old `lastRelease` is the signal you are looking for.
- **Release monitoring.** Run with no input on a schedule to capture whatever RubyGems published since the last sweep.

### Quick start

Three gems, default depth:

```json
{
 "packages": ["rails", "rspec", "sidekiq"]
}
```

A dependency audit with full history:

```json
{
 "packages": ["rails", "puma", "nokogiri", "devise"],
 "includeVersionHistory": true,
 "maxVersions": 50,
 "includeDependencies": true
}
```

Release monitor, no input at all:

```json
{}
```

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `packages` | array | `[]` | Gem names or `rubygems.org/gems/.` URLs. Empty pulls the most recently published gems from the activity feed. |
| `includeVersionHistory` | boolean | `false` | Add the `releases` array and fill in `releaseCount` and `firstRelease`. Costs one extra request per gem. |
| `maxVersions` | integer | `20` | Cap on entries in `releases`, newest first. `releaseCount` still reports the true total. |
| `includeDependencies` | boolean | `true` | Include `runtimeDependencies` and `developmentDependencies`. They arrive free with the gem lookup. |
| `maxPackages` | integer | `25` | Cap on gems per run. Applies to both an explicit list and the activity-feed fallback. |

Blanks, duplicates and trailing `.json` are stripped before any request goes out, and two inputs that resolve to the same gem are only saved once.

### Gem names are case-sensitive

This trips people up more than anything else about the RubyGems API. `rubygems.org/api/v1/gems/RedCloth.json` returns 200; the same URL with `redcloth` returns 404. Paste the name exactly as it appears on the gem's RubyGems page.

As a safety net, a mixed-case name that 404s is retried once in lowercase, so `Rails` still finds `rails`. The reverse is not recoverable: nothing about `redcloth` tells the Actor where the capitals belong.

### Output example

```json
{
 "name": "rails",
 "version": "8.1.3.1",
 "platform": "ruby",
 "lastRelease": "2026-07-29T15:02:41.060Z",
 "downloads": 772174102,
 "versionDownloads": 981377,
 "authors": "David Heinemeier Hansson",
 "info": "Ruby on Rails is a full-stack web framework optimized for programmer happiness and sustainable productivity.",
 "licenses": ["MIT"],
 "yanked": false,
 "sha": "ccd11a36bfc171bf9c66d585d14c0ece91c0c9dde840aae60c0118d6f5c9c52a",
 "projectUri": "https://rubygems.org/gems/rails",
 "gemUri": "https://rubygems.org/gems/rails-8.1.3.1.gem",
 "homepageUri": "https://rubyonrails.org",
 "documentationUri": "https://api.rubyonrails.org/v8.1.3.1/",
 "sourceCodeUri": "https://github.com/rails/rails/tree/v8.1.3.1",
 "bugTrackerUri": "https://github.com/rails/rails/issues",
 "changelogUri": "https://github.com/rails/rails/releases/tag/v8.1.3.1",
 "mailingListUri": "https://discuss.rubyonrails.org/c/rubyonrails-talk",
 "wikiUri": null,
 "fundingUri": null,
 "metadata": { "rubygems_mfa_required": "true" },
 "runtimeDependencies": [
 { "name": "activesupport", "requirements": "= 8.1.3.1" },
 { "name": "bundler", "requirements": ">= 1.15.0" }
 ],
 "developmentDependencies": [],
 "releaseCount": 519,
 "firstRelease": "2009-07-25T18:01:56.000Z",
 "releases": [
 {
 "version": "8.1.3.1",
 "platform": "ruby",
 "releasedAt": "2026-07-29T15:02:41.060Z",
 "downloads": 984407,
 "prerelease": false,
 "rubyVersion": ">= 3.2.0",
 "licenses": ["MIT"],
 "sha": "ccd11a36bfc171bf9c66d585d14c0ece91c0c9dde840aae60c0118d6f5c9c52a"
 }
 ],
 "scrapedAt": "2026-08-06T11:41:07.204Z"
}
```

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~rubygems-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "packages": ["rails", "sidekiq"],
 "includeVersionHistory": true,
 "maxVersions": 10
 }'
```

### Limits and behaviour

- **`authors` is a single string, not a list.** RubyGems stores it as free text and plenty of gems use a comma inside one name, "Datadog, Inc.", so splitting on commas would invent authors that do not exist. The field is passed through exactly as published.
- **Version history is a second endpoint.** `releaseCount`, `firstRelease` and `releases` come from `/api/v1/versions/{gem}.json`, which is why they are `null` unless `includeVersionHistory` is on. That response is large, Rails returns 519 versions and about 450 KB.
- **The no-input path costs one request.** The activity feed returns fully-formed gem objects, so the 25 default records are built without a single per-gem lookup. It lists a gem once per push, so repeats are collapsed before the cap is applied.
- **A missing gem is a per-item failure.** RubyGems answers 404 with a plain-text body, not JSON; that name lands in `RUN_SUMMARY.unknownPackages` and the run carries on. The Actor errors out only when *every* lookup failed.
- **Transient errors are retried.** 429 and 5xx get three attempts with backoff, honouring `Retry-After` when RubyGems sends one. Requests are spaced 200 ms apart, comfortably inside the documented public-API budget.
- **Dependencies describe the latest version only.** The API does not expose historical dependency graphs; `releases` entries carry version, date, downloads and platform, not requirements.

**Defaults:** 1 GB memory, 15 minute timeout.

### FAQ

**Do I need a RubyGems account or API key?** No. You supply no credentials.

**Does it need a proxy?** No. There is nothing for a proxy to solve.

**Why is `versionDownloads` so much smaller than `downloads`?** `downloads` is all-time across every version ever published; `versionDownloads` counts only the latest one. Their ratio is a decent proxy for upgrade velocity.

**Can I get metadata for a specific old version?** Turn on `includeVersionHistory`, each entry carries its own date, download count, Ruby requirement and checksum.

**What does `platform` mean?** Most gems are `ruby` (pure Ruby, any platform). Native extensions publish per-platform variants such as `java` or `x86_64-linux`, and each is a separate row in the version history.

**Can I schedule it?** Yes. Diff on `version` or `lastRelease` between runs to detect new releases, and use the empty-input mode to catch gems you weren't already tracking.

**Can I integrate it with something else?** Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.

# Actor input Schema

## `packages` (type: `array`):

Gem names. the slug in the gem's RubyGems URL (rubygems.org/gems/rails → 'rails'). Full URLs are accepted and reduced to the name. Gem names are case-sensitive on RubyGems: 'RedCloth' resolves and 'redcloth' does not, so paste the name exactly as it appears. A mixed-case miss is retried once in lowercase. Leave empty to pull the most recently published gems instead.

## `includeVersionHistory` (type: `boolean`):

Add a 'releases' array from the separate versions endpoint: version number, release date, per-version downloads, platform and prerelease flag, newest first. Costs one extra request per gem and also fills in 'releaseCount' and 'firstRelease'.

## `maxVersions` (type: `integer`):

How many versions the 'releases' array keeps, counting back from the newest. Ignored when Include version history is off. Rails has over 500 versions, so leaving this low keeps the dataset readable; 'releaseCount' still reports the true total.

## `includeDependencies` (type: `boolean`):

Include 'runtimeDependencies' and 'developmentDependencies' for the latest version, each entry carrying the dependency name and its version requirement. They arrive with the gem lookup at no extra cost. turn this off only to slim the dataset.

## `maxPackages` (type: `integer`):

Cap on how many gems are looked up in one run. Each gem is one request, or two with version history on. Applies to both an explicit gem list and the recently-published fallback, which itself tops out at 50.

## Actor input object example

```json
{
  "packages": [
    "rails",
    "puma",
    "nokogiri"
  ],
  "includeVersionHistory": false,
  "maxVersions": 20,
  "includeDependencies": true,
  "maxPackages": 25
}
```

# Actor output Schema

## `items` (type: `string`):

Every record the run produced.

## `runsummary` (type: `string`):

The RUN\_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "packages": [
        "rails",
        "rspec",
        "sidekiq"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/rubygems-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "packages": [
        "rails",
        "rspec",
        "sidekiq",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/rubygems-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "packages": [
    "rails",
    "rspec",
    "sidekiq"
  ]
}' |
apify call arman-bd/rubygems-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/rubygems-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/PxtT3rHWZ9DVgE0e0/builds/TktLJH5Ri3Px9XfH2/openapi.json