Go Module Scraper: Versions, Info & Checksums

Go Module Scraper queries proxy.golang.org, the official Go module mirror, for any module you name, every published version, the latest release and when it was cut, the go directive, the full dependency list from go.mod, and whether the module has been formally deprecated.

This is the same source the go command itself resolves against, so it is the canonical answer rather than a scrape of a package page: no browser, no proxies, no login.

Agent skill: SKILL.md

https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/go-module-scraper.md

What you get

Output field	Meaning
module	The module path exactly as you asked for it
encodedPath	The proxy-encoded form actually requested, see the note below
latestVersion	What go get <module> would resolve to right now
publishedAt	Commit timestamp of that version, straight from the proxy
versionCount	How many versions the proxy has ever served for this module
versions	Every version, newest first, when includeAllVersions is on
goVersion	The go directive from go.mod, the minimum toolchain the module claims
requires	Every requirement: { module, version, indirect }
isDeprecated, deprecationNotice	Whether go.mod carries a // Deprecated: marker, and what it says
repoUrl	Upstream VCS URL the proxy resolved the module from
docsUrl	pkg.go.dev page for the module
scrapedAt	Run timestamp

A RUN_SUMMARY record in the key-value store holds per-run counts, the options used, and any module that failed.

On encodedPath. The Go proxy protocol forbids uppercase letters in a request path, every capital must be written as ! followed by its lowercase form, so that a case-insensitive file system can never conflate two different modules. github.com/Masterminds/semver has to be requested as github.com/!masterminds/semver; asking for the raw path returns a flat 404. You pass the normal path and the Actor does the encoding; encodedPath shows you what actually went over the wire.

Common use cases

Audit Go dependencies across repos. Feed it every module in your go.mod files and check what is stale, what is deprecated and what the transitive surface looks like.

{
 "modules": [
 "github.com/gin-gonic/gin",
 "github.com/golang/protobuf",
 "golang.org/x/net",
 "gopkg.in/yaml.v2"
 ],
 "includeGoMod": true,
 "includeAllVersions": false
}

Track module release cadence. Pull the full version list and derive time-between-releases per module.

{
 "modules": ["github.com/gorilla/mux", "github.com/pelletier/go-toml/v2"],
 "includeAllVersions": true,
 "includeGoMod": false
}

Build a Go package explorer. Everything on, for a curated catalogue.

{
 "modules": ["github.com/spf13/cobra", "github.com/stretchr/testify", "github.com/Masterminds/semver/v3"],
 "includeGoMod": true,
 "includeAllVersions": true
}

Input

Field	Type	Default	Notes
modules	array	-	Required. Full import paths (github.com/gorilla/mux). pkg.go.dev URLs, https:// prefixes and trailing @v1.2.3 are stripped for you.
includeGoMod	boolean	true	Fetch and parse go.mod for the latest version. Populates goVersion, requires, isDeprecated. +1 request per module.
includeAllVersions	boolean	true	Write the full sorted versions array. versionCount is populated regardless.

Which combinations make sense. includeGoMod is the one that costs a request; includeAllVersions only controls how much lands in each row. A module like golang.org/x/net has hundreds of versions, so turning includeAllVersions off is worth it when you are sweeping a large catalogue and only need the latest. If you want a dependency audit, includeGoMod is the whole point, leave it on.

Output example

{
 "module": "github.com/gin-gonic/gin",
 "encodedPath": "github.com/gin-gonic/gin",
 "latestVersion": "v1.10.0",
 "publishedAt": "2024-05-07T09:12:18Z",
 "versionCount": 47,
 "versions": ["v1.10.0", "v1.9.1", "v1.9.0", "v1.8.2", "…"],
 "goVersion": "1.20",
 "requires": [
 { "module": "github.com/bytedance/sonic", "version": "v1.11.6", "indirect": false },
 { "module": "github.com/gin-contrib/sse", "version": "v0.1.0", "indirect": false },
 { "module": "github.com/go-playground/validator/v10","version": "v10.20.0","indirect": false },
 { "module": "github.com/cloudwego/base64x", "version": "v0.1.4", "indirect": true }
 ],
 "isDeprecated": false,
 "deprecationNotice": null,
 "repoUrl": "https://github.com/gin-gonic/gin",
 "docsUrl": "https://pkg.go.dev/github.com/gin-gonic/gin",
 "scrapedAt": "2026-08-06T12:00:00.000Z"
}

A deprecated module looks like this, github.com/golang/protobuf carries the marker in its go.mod:

{
 "module": "github.com/golang/protobuf",
 "latestVersion": "v1.5.4",
 "publishedAt": "2024-03-06T06:45:40Z",
 "isDeprecated": true,
 "deprecationNotice": "Use the \"google.golang.org/protobuf\" module instead."
}

RUN_SUMMARY in the key-value store:

{
 "modulesRequested": 4,
 "modulesSaved": 3,
 "modulesFailed": 1,
 "failures": [
 { "module": "github.com/nonexistent/nope-xyz", "error": "not found (404): not found: module github.com/nonexistent/nope-xyz: git ls-remote …" }
 ],
 "filters": { "includeGoMod": true, "includeAllVersions": false },
 "finishedAt": "2026-08-06T12:00:03.402Z"
}

Limits and behaviour

• The proxy returns plain text, not JSON, for two of the three endpoints. /@v/list is a newline-separated list of version strings and /@v/{version}.mod is raw go.mod source; only /@latest and /@v/{version}.info are JSON. This Actor parses all three and hands you structured fields.
• Module paths are case-encoded before every request. Uppercase letters become ! + lowercase, per the proxy protocol. Skipping this is the single most common way to get a spurious 404 out of proxy.golang.org.
• The version list is unordered on the wire. /@v/list returns tags in arbitrary order, v1.3.0 can precede v1.8.1. The Actor sorts them by semver precedence, newest first, with prereleases ranked below their release and Go pseudo-versions ordered by their embedded timestamp.
• Modules with no tags still work. Some modules were never tagged and only ever had pseudo-versions; /@v/list comes back empty or 404 while /@latest still resolves. That is logged as a warning, not a failure, and the row is still saved.
• No advertised rate limit. proxy.golang.org publishes no rate-limit headers, but it is a shared public service, the Actor requests modules sequentially rather than in a burst.
• One bad module never aborts the run. A typo or a private repo returns 404 and is recorded in RUN_SUMMARY.failures; the run continues. The Actor only throws if every module failed.
• Transient errors are retried. 429 and 5xx get three attempts with linear backoff. A 404/410 is fatal for that module and is not retried, because the proxy's answer will not change.
• Duplicates are removed before any request is made.
• Public data only. No authentication, no personal data, no private modules, anything behind GOPRIVATE is invisible to the public proxy by design.

Finding a module path

The module path is the first line of the project's go.mod, and it is what you write in an import. Three rules cover almost every case:

What you have	Module path
https://github.com/gorilla/mux	github.com/gorilla/mux
https://pkg.go.dev/github.com/spf13/cobra	github.com/spf13/cobra
A v2-or-later release	Add the suffix: github.com/pelletier/go-toml/v2

The major-version suffix trips people up most often: from v2 onward, Go treats each major version as a different module, so github.com/pelletier/go-toml and github.com/pelletier/go-toml/v2 are two separate rows with two separate version histories.

API example

curl -X POST "https://api.apify.com/v2/acts/arman-bd~go-module-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "modules": ["github.com/gorilla/mux", "github.com/gin-gonic/gin"],
 "includeGoMod": true,
 "includeAllVersions": false
 }'

JavaScript example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/go-module-scraper').call({
 modules: ['github.com/gorilla/mux', 'github.com/golang/protobuf'],
 includeGoMod: true,
 includeAllVersions: false,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const m of items) {
 const flag = m.isDeprecated ? ' ⚠ DEPRECATED' : '';
 console.log(`${m.module}@${m.latestVersion} (go ${m.goVersion}, ${m.versionCount} versions)${flag}`);
}

FAQ

Do I need a proxy? No. Proxy configuration is not required to run this Actor.

Do I need a Go installation or a Google account? Neither. This talks to the proxy over plain HTTPS.

Why did a module path with capitals return 404 when I tried it myself? Because the proxy protocol requires !-escaping of uppercase letters. github.com/Masterminds/semver must go over the wire as github.com/!masterminds/semver. The Actor handles it; encodedPath shows you the result.

Why is versionCount sometimes 0 when latestVersion is set? The module has never been tagged. go get resolves it to a pseudo-version derived from the newest commit, which is what latestVersion reports, but there is nothing to list.

Does it give me checksums? The proxy exposes hashes through sum.golang.org, a separate transparency log with its own signed-tree format. This Actor reports the resolved commit hash via repoUrl and the version identity, not the go.sum lines.

What happens if the proxy is unavailable? 5xx responses are retried three times, then that module is recorded in RUN_SUMMARY.failures and the run continues.

Can I schedule it? Yes, it is designed for it. Run your dependency list nightly and diff on latestVersion and isDeprecated.

Can I integrate it with something else? Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.