Go Modules Scraper

Go Modules Scraper helps you easily gather Go module versions and package details. Whether you need to inspect existing projects, track fresh releases, or audit dependencies, you can pick a mode, provide your module paths or a start timestamp, and collect structured data ready for analysis.

Outputs can be exported seamlessly into JSON, CSV, or Excel formats.

Use Cases

• Dependency Inventory: Keep track of published versions for modules used in your Go services.
• Release Tracking: Monitor the latest versions, publish timestamps, and commit details for specific module paths.
• Index Monitoring: Catch newly published module versions over time based on specific timestamps.
• Deprecation Checks: Quickly check for deprecated packages in your software stack.
• Spreadsheet Exports: Export your dataset as a spreadsheet for quick reviews or scheduled audits.

Operation Modes

• Versions (versions): Collects published versions for your selected modules line by line.
• Latest (latest): Pulls the newest release version, publish timestamp, source URL, and commit hash.
• New Versions Feed (index_feed): Scans recently published module versions using page limits and optional timestamp filters.
• Metadata (metadata): Retrieves detailed package information per version, including deprecation flags.

Uppercase letters in module paths are automatically formatted according to standard Go path rules before processing.

Input Parameters

Parameter	Type	Default	Description
mode	string	versions	Select from: versions, latest, index_feed, or metadata.
modules	array	github.com/gin-gonic/gin	List of module paths to process.
since	string	empty	Timestamp filter for feed tracking.
maxItems	integer	100	Maximum rows to collect per run (capped at 1000).
requestDelaySecs	number	1	Pause duration between consecutive requests.
userAgent	string	BigiByteActorResearch/1.0 (contact: research@bigibyte.com)	Custom identifier sent with requests.
timeoutSecs	integer	300	Total run timeout in seconds.
requestTimeoutSecs	integer	30	Timeout per individual request.
proxyConfiguration	object	Default settings	Optional proxy setup if required.

Example Input

{
  "mode": "versions",
  "modules": ["[github.com/gin-gonic/gin](https://github.com/gin-gonic/gin)"],
  "since": "",
  "maxItems": 100,
  "requestDelaySecs": 1,
  "userAgent": "BigiByteActorResearch/1.0 (contact: research@bigibyte.com)",
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": { "useApifyProxy": false }
}

Extracted Data Format

Here is an example of the output structure generated in latest mode:

{
  "module": "[github.com/gin-gonic/gin](https://github.com/gin-gonic/gin)",
  "Version": "v1.12.0",
  "Time": "2026-02-28T10:10:09Z",
  "Origin_URL": "[https://github.com/gin-gonic/gin](https://github.com/gin-gonic/gin)",
  "Origin_Hash": "73726dc606796a025971fe451f0aa6f1b9b847f6",
  "scraped_at": "2026-10-12T00:00:00+00:00"
}

Field	Type	Description
module	string	Module path or target identifier.
Version	string	Extracted version string.
Time	string	Publication timestamp.
Origin_URL	string	Source repository URL.
Origin_Hash	string	Associated commit hash.
versions	array	List of version strings collected in versions mode.
isDeprecated	boolean	Indicates whether the package version is deprecated.
scraped_at	string	UTC timestamp when the record was generated.
error	boolean	Included if an issue occurred during lookup.
url	string	Target URL involved in an error.
status	integer	HTTP response status code.
message	string	Explanation of why a lookup failed.

Pricing & Usage Limits

This tool is free to use. You only pay for standard Apify platform runtime and storage fees.

To prevent unnecessary rate limits, requests are automatically spaced out with default delay settings and automatic retries. Runs are capped at a maximum of 1,000 items (maxItems) or governed by your specified runtime timeout (timeoutSecs).

How It Works

1. Configuration: The tool reads your requested mode, module list, item limits, and timing preferences.
2. Data Extraction: Performs structured requests tailored to your selected operation mode.
3. Error Handling: Handles rate limits gracefully with built-in retries while generating clear error records for missing modules.
4. Output Storage: Formats the extracted records and saves them cleanly into your dataset.

FAQ

Can I export module data to CSV?

Yes, datasets can be downloaded in JSON, CSV, or Excel formats anytime after a run completes.

What happens if a module path is incorrect?

If a path cannot be found, an error row is recorded with the status code and explanation. The rest of your requested modules will continue processing normally.

How many versions can I collect per run?

maxItems defaults to 100 items per run and can be set up to 1,000. Total execution is also subject to your configured timeoutSecs.

Related Tools

If you plan to organize, aggregate, or process these module dependencies, a CSV cleanup or custom data pipeline is often the next step. BigiByte also builds custom scrapers and automation tools for other package registries and data portals when you need tailored data workflows.

Need a scheduled dependency monitor, custom data feed, or specialized integration? BigiByte builds custom scrapers and APIs: https://bigibyte.com

Trademark Notice

Go Modules and related names, logos, and trademarks belong to their respective owners. Their mention in this documentation is strictly for identification purposes and does not imply affiliation, sponsorship, or endorsement.

Disclaimer

This scraper is an independent automation tool. It is not affiliated with, endorsed by, or officially associated with Go Modules or its parent entities. All data collected originates from publicly available resources. Users are responsible for adhering to relevant terms of use and data handling policies.

Is web scraping legal?

This tool only collects publicly available data without bypassing any security or access controls. Collecting public data is generally legal, but how you manage the resulting data is important:

• Personal Data: If your datasets contain personal information, regulations such as GDPR or CCPA may apply. Ensure you handle personal data under valid legal bases.
• Copyright: Certain text descriptions or media may be protected by copyright. Avoid republishing copyrighted content without authorization.
• Terms of Use: Respect standard rate limits and source guidelines.
• Legal Advice: This overview provides general information, not formal legal counsel. Consult a qualified attorney for specific legal guidance.

For further reading, check out Apify's full guide: Is web scraping legal?