# pkg.go.dev Scraper - Go Packages & Modules (`ninhothedev/pkg-go-dev-scraper`) Actor

$0.5/1K 🔥 pkg.go.dev scraper! Go packages by search — import path, synopsis, license & imported-by. No key. JSON, CSV, Excel or API in seconds. Discover & research Go modules ⚡

- **URL**: https://apify.com/ninhothedev/pkg-go-dev-scraper.md
- **Developed by:** [ninhothedev](https://apify.com/ninhothedev) (community)
- **Categories:** Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## pkg.go.dev Scraper - Go Packages & Modules

Discover and extract **Go packages and modules** from [pkg.go.dev](https://pkg.go.dev) by search query - no API key, no login. Feed it search terms like `http router`, `json`, or `cli` and get back clean, structured records for every matching Go module: import path, synopsis, latest version, published date, **imported-by count**, license and canonical URL.

Perfect for **Go package discovery**, **dependency research**, building **developer tooling**, and **open-source intelligence** on the Go ecosystem.

> Roughly **$0.5 per 1,000 packages** on the Apify platform.

### Features

- 🔎 **Search-driven** - one or many queries, results merged and de-duplicated by import path
- 📦 **Rich metadata** - import path, package name, synopsis, version, published date, imported-by count, license
- 🔢 **Imported-by as a number** - `"99,459"` is parsed to `99459` so you can sort and rank by popularity
- 🚫 **No key required** - scrapes the public pkg.go.dev search directly (curl\_cffi Chrome fingerprint + urllib fallback)
- ⚙️ **Configurable limits** - cap total items from 1 up to 1,000
- 🧱 **Clean schema** - every field nullable, permissive dataset schema, tidy `overview` table view

### Input

| Field | Type | Description |
|-------|------|-------------|
| `mode` | select | Scraping mode. Currently `search` (queries → Go packages). |
| `queries` | array | Search terms to look up on pkg.go.dev. Default: `["http router", "json", "cli"]`. |
| `maxItems` | integer | Max total package items across all queries. Default `100`, max `1000`. |

#### Example input

```json
{
  "mode": "search",
  "queries": ["http router", "orm", "grpc"],
  "maxItems": 200
}
```

### Output

Each dataset item is one Go package/module:

```json
{
  "path": "github.com/gorilla/mux",
  "name": "mux",
  "synopsis": "Package mux implements a request router and dispatcher.",
  "version": "v1.8.1",
  "published": "Oct 18, 2023",
  "imported_by": 99459,
  "license": "BSD-3-Clause",
  "url": "https://pkg.go.dev/github.com/gorilla/mux",
  "source": "pkg-go-dev",
  "scraped_at": "2026-07-24T12:00:00+00:00"
}
```

### Use cases

- **Go package discovery** - find the most-used libraries for a task, ranked by imported-by count
- **Dependency research** - audit alternatives, licenses and adoption before adding a dependency
- **Developer tooling** - power internal catalogs, package browsers and CI policy checks
- **OSS intelligence** - track the Go ecosystem, licensing mix and popularity trends

### Related scrapers

- [GitHub Scraper](https://apify.com/ninhothedev/github-scraper)
- [npms Search Scraper](https://apify.com/ninhothedev/npms-search-scraper)
- [Bundlephobia Scraper](https://apify.com/ninhothedev/bundlephobia-scraper)
- [crates.io Scraper](https://apify.com/ninhothedev/crates-io-scraper)

### Notes

This Actor scrapes publicly available pages from pkg.go.dev. It reads only public package metadata and stores no personal data. Respect pkg.go.dev's terms and use reasonable request volumes.

# Actor input Schema

## `mode` (type: `string`):

Scraping mode. 'search' sends each query to pkg.go.dev search and returns the matching Go packages/modules.

## `queries` (type: `array`):

List of search terms to look up on pkg.go.dev (e.g. 'http router', 'json', 'cli'). Each query returns its matching Go packages/modules.

## `maxItems` (type: `integer`):

Maximum number of package items to scrape in total across all queries. Hard-capped at 1000.

## Actor input object example

```json
{
  "mode": "search",
  "queries": [
    "http router",
    "json",
    "cli"
  ],
  "maxItems": 100
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "http router",
        "json",
        "cli"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("ninhothedev/pkg-go-dev-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": [
        "http router",
        "json",
        "cli",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("ninhothedev/pkg-go-dev-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "http router",
    "json",
    "cli"
  ]
}' |
apify call ninhothedev/pkg-go-dev-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=ninhothedev/pkg-go-dev-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/hjSg6Q8cvgXLHggsi/builds/8g47Nnf38zkcKT5ZM/openapi.json