# Go Module Scraper: Versions, Info & Checksums (`arman-bd/go-module-scraper`) Actor

Query the official Go module proxy for any module: available versions, publish timestamps and go.mod contents. The canonical source for Go dependency data.

- **URL**: https://apify.com/arman-bd/go-module-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** Developer tools, Automation, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.37 / 1,000 module scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Go Module Scraper: Versions, Info & Checksums

![Go Module Scraper: Every published version, the latest release, go.mod dependencies and deprecation notices, straight from the official Go module proxy](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/go-module-scraper.jpg)

**Go Module Scraper** queries **proxy.golang.org**, the official Go module mirror, for any module you name, every published version, the latest release and when it was cut, the `go` directive, the full dependency list from `go.mod`, and whether the module has been formally deprecated.

This is the same source the `go` command itself resolves against, so it is the canonical answer rather than a scrape of a package page: **no browser, no proxies, no login.**

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/go-module-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/go-module-scraper.md
```

### What you get

| Output field | Meaning |
|---|---|
| `module` | The module path exactly as you asked for it |
| `encodedPath` | The proxy-encoded form actually requested, see the note below |
| `latestVersion` | What `go get <module>` would resolve to right now |
| `publishedAt` | Commit timestamp of that version, straight from the proxy |
| `versionCount` | How many versions the proxy has ever served for this module |
| `versions` | Every version, newest first, when `includeAllVersions` is on |
| `goVersion` | The `go` directive from `go.mod`, the minimum toolchain the module claims |
| `requires` | Every requirement: `{ module, version, indirect }` |
| `isDeprecated`, `deprecationNotice` | Whether `go.mod` carries a `// Deprecated:` marker, and what it says |
| `repoUrl` | Upstream VCS URL the proxy resolved the module from |
| `docsUrl` | pkg.go.dev page for the module |
| `scrapedAt` | Run timestamp |

A `RUN_SUMMARY` record in the key-value store holds per-run counts, the options used, and any module that failed.

**On `encodedPath`.** The Go proxy protocol forbids uppercase letters in a request path, every capital must be written as `!` followed by its lowercase form, so that a case-insensitive file system can never conflate two different modules. `github.com/Masterminds/semver` has to be requested as `github.com/!masterminds/semver`; asking for the raw path returns a flat 404. You pass the normal path and the Actor does the encoding; `encodedPath` shows you what actually went over the wire.

### Common use cases

**Audit Go dependencies across repos.** Feed it every module in your `go.mod` files and check what is stale, what is deprecated and what the transitive surface looks like.

```json
{
 "modules": [
 "github.com/gin-gonic/gin",
 "github.com/golang/protobuf",
 "golang.org/x/net",
 "gopkg.in/yaml.v2"
 ],
 "includeGoMod": true,
 "includeAllVersions": false
}
```

**Track module release cadence.** Pull the full version list and derive time-between-releases per module.

```json
{
 "modules": ["github.com/gorilla/mux", "github.com/pelletier/go-toml/v2"],
 "includeAllVersions": true,
 "includeGoMod": false
}
```

**Build a Go package explorer.** Everything on, for a curated catalogue.

```json
{
 "modules": ["github.com/spf13/cobra", "github.com/stretchr/testify", "github.com/Masterminds/semver/v3"],
 "includeGoMod": true,
 "includeAllVersions": true
}
```

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `modules` | array | - | **Required.** Full import paths (`github.com/gorilla/mux`). pkg.go.dev URLs, `https://` prefixes and trailing `@v1.2.3` are stripped for you. |
| `includeGoMod` | boolean | `true` | Fetch and parse `go.mod` for the latest version. Populates `goVersion`, `requires`, `isDeprecated`. **+1 request per module.** |
| `includeAllVersions` | boolean | `true` | Write the full sorted `versions` array. `versionCount` is populated regardless. |

**Which combinations make sense.** `includeGoMod` is the one that costs a request; `includeAllVersions` only controls how much lands in each row. A module like `golang.org/x/net` has hundreds of versions, so turning `includeAllVersions` off is worth it when you are sweeping a large catalogue and only need the latest. If you want a dependency audit, `includeGoMod` is the whole point, leave it on.

### Output example

```json
{
 "module": "github.com/gin-gonic/gin",
 "encodedPath": "github.com/gin-gonic/gin",
 "latestVersion": "v1.10.0",
 "publishedAt": "2024-05-07T09:12:18Z",
 "versionCount": 47,
 "versions": ["v1.10.0", "v1.9.1", "v1.9.0", "v1.8.2", "…"],
 "goVersion": "1.20",
 "requires": [
 { "module": "github.com/bytedance/sonic", "version": "v1.11.6", "indirect": false },
 { "module": "github.com/gin-contrib/sse", "version": "v0.1.0", "indirect": false },
 { "module": "github.com/go-playground/validator/v10","version": "v10.20.0","indirect": false },
 { "module": "github.com/cloudwego/base64x", "version": "v0.1.4", "indirect": true }
 ],
 "isDeprecated": false,
 "deprecationNotice": null,
 "repoUrl": "https://github.com/gin-gonic/gin",
 "docsUrl": "https://pkg.go.dev/github.com/gin-gonic/gin",
 "scrapedAt": "2026-08-06T12:00:00.000Z"
}
```

A deprecated module looks like this, `github.com/golang/protobuf` carries the marker in its `go.mod`:

```json
{
 "module": "github.com/golang/protobuf",
 "latestVersion": "v1.5.4",
 "publishedAt": "2024-03-06T06:45:40Z",
 "isDeprecated": true,
 "deprecationNotice": "Use the \"google.golang.org/protobuf\" module instead."
}
```

`RUN_SUMMARY` in the key-value store:

```json
{
 "modulesRequested": 4,
 "modulesSaved": 3,
 "modulesFailed": 1,
 "failures": [
 { "module": "github.com/nonexistent/nope-xyz", "error": "not found (404): not found: module github.com/nonexistent/nope-xyz: git ls-remote …" }
 ],
 "filters": { "includeGoMod": true, "includeAllVersions": false },
 "finishedAt": "2026-08-06T12:00:03.402Z"
}
```

### Limits and behaviour

- **The proxy returns plain text, not JSON, for two of the three endpoints.** `/@v/list` is a newline-separated list of version strings and `/@v/{version}.mod` is raw `go.mod` source; only `/@latest` and `/@v/{version}.info` are JSON. This Actor parses all three and hands you structured fields.
- **Module paths are case-encoded before every request.** Uppercase letters become `!` + lowercase, per the proxy protocol. Skipping this is the single most common way to get a spurious 404 out of proxy.golang.org.
- **The version list is unordered on the wire.** `/@v/list` returns tags in arbitrary order, `v1.3.0` can precede `v1.8.1`. The Actor sorts them by semver precedence, newest first, with prereleases ranked below their release and Go pseudo-versions ordered by their embedded timestamp.
- **Modules with no tags still work.** Some modules were never tagged and only ever had pseudo-versions; `/@v/list` comes back empty or 404 while `/@latest` still resolves. That is logged as a warning, not a failure, and the row is still saved.
- **No advertised rate limit.** proxy.golang.org publishes no rate-limit headers, but it is a shared public service, the Actor requests modules sequentially rather than in a burst.
- **One bad module never aborts the run.** A typo or a private repo returns 404 and is recorded in `RUN_SUMMARY.failures`; the run continues. The Actor only throws if *every* module failed.
- **Transient errors are retried.** 429 and 5xx get three attempts with linear backoff. A 404/410 is fatal for that module and is not retried, because the proxy's answer will not change.
- **Duplicates are removed** before any request is made.
- **Public data only.** No authentication, no personal data, no private modules, anything behind `GOPRIVATE` is invisible to the public proxy by design.

### Finding a module path

The module path is the first line of the project's `go.mod`, and it is what you write in an `import`. Three rules cover almost every case:

| What you have | Module path |
|---|---|
| `https://github.com/gorilla/mux` | `github.com/gorilla/mux` |
| `https://pkg.go.dev/github.com/spf13/cobra` | `github.com/spf13/cobra` |
| A v2-or-later release | Add the suffix: `github.com/pelletier/go-toml/v2` |

The major-version suffix trips people up most often: from v2 onward, Go treats each major version as a *different module*, so `github.com/pelletier/go-toml` and `github.com/pelletier/go-toml/v2` are two separate rows with two separate version histories.

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~go-module-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "modules": ["github.com/gorilla/mux", "github.com/gin-gonic/gin"],
 "includeGoMod": true,
 "includeAllVersions": false
 }'
```

### JavaScript example

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/go-module-scraper').call({
 modules: ['github.com/gorilla/mux', 'github.com/golang/protobuf'],
 includeGoMod: true,
 includeAllVersions: false,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const m of items) {
 const flag = m.isDeprecated ? ' ⚠ DEPRECATED' : '';
 console.log(`${m.module}@${m.latestVersion} (go ${m.goVersion}, ${m.versionCount} versions)${flag}`);
}
```

### FAQ

**Do I need a proxy?** No. Proxy configuration is not required to run this Actor.

**Do I need a Go installation or a Google account?** Neither. This talks to the proxy over plain HTTPS.

**Why did a module path with capitals return 404 when I tried it myself?** Because the proxy protocol requires `!`-escaping of uppercase letters. `github.com/Masterminds/semver` must go over the wire as `github.com/!masterminds/semver`. The Actor handles it; `encodedPath` shows you the result.

**Why is `versionCount` sometimes 0 when `latestVersion` is set?** The module has never been tagged. `go get` resolves it to a pseudo-version derived from the newest commit, which is what `latestVersion` reports, but there is nothing to list.

**Does it give me checksums?** The proxy exposes hashes through `sum.golang.org`, a separate transparency log with its own signed-tree format. This Actor reports the resolved commit hash via `repoUrl` and the version identity, not the go.sum lines.

**What happens if the proxy is unavailable?** 5xx responses are retried three times, then that module is recorded in `RUN_SUMMARY.failures` and the run continues.

**Can I schedule it?** Yes, it is designed for it. Run your dependency list nightly and diff on `latestVersion` and `isDeprecated`.

**Can I integrate it with something else?** Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.

# Actor input Schema

## `modules` (type: `array`):

Full import paths as they appear in a go.mod require line. host first, e.g. github.com/gorilla/mux or golang.org/x/net. Major versions from v2 onward carry the suffix: github.com/pelletier/go-toml/v2. pkg.go.dev URLs and trailing @version suffixes are stripped automatically, and uppercase letters are proxy-encoded for you.

## `includeGoMod` (type: `boolean`):

Fetch the go.mod of the latest version and parse it. This is what populates goVersion, requires and isDeprecated. with it off those three fields come back null. Costs one extra request per module.

## `includeAllVersions` (type: `boolean`):

Write the full sorted version list into each row. Turn it off for a much smaller dataset when you only care about the latest release. versionCount is still populated either way.

## Actor input object example

```json
{
  "modules": [
    "github.com/gorilla/mux",
    "golang.org/x/net",
    "github.com/Masterminds/semver/v3"
  ],
  "includeGoMod": true,
  "includeAllVersions": true
}
```

# Actor output Schema

## `items` (type: `string`):

Every record the run produced.

## `runsummary` (type: `string`):

The RUN\_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "modules": [
        "github.com/gorilla/mux",
        "github.com/gin-gonic/gin"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/go-module-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "modules": [
        "github.com/gorilla/mux",
        "github.com/gin-gonic/gin",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/go-module-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "modules": [
    "github.com/gorilla/mux",
    "github.com/gin-gonic/gin"
  ]
}' |
apify call arman-bd/go-module-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/go-module-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/QpdT4urlyIljsPtxw/builds/wh9hmp0qrtUh6u1TW/openapi.json