# GitHub Repository Scraper (`superslowsloth/github-repository-scraper`) Actor

Search GitHub repositories with full search syntax, or look them up by owner and name. Returns stars, forks, open issues, language, topics, licence, default branch, size and the created, updated and pushed timestamps. Add your own GitHub token to lift the unauthenticated rate limit.

- **URL**: https://apify.com/superslowsloth/github-repository-scraper.md
- **Developed by:** [Superslow Sloth](https://apify.com/superslowsloth) (community)
- **Categories:** Developer tools, AI, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.40 / 1,000 repositories

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## GitHub Repository Scraper

Search GitHub for repositories using GitHub's own search syntax, or look up
specific repositories by `owner/repo`, and get a clean row per repository:
stars, forks, open issues, language, topics, licence, timestamps and more.

Everything comes from GitHub's public REST API, so the numbers are GitHub's own
rather than something scraped off a rendered page.

### Read this first: the rate limit is the thing that will bite you

GitHub meters its API, and how much this actor can do in one run depends almost
entirely on whether you give it a token. These figures were measured against the
live API on 2026-08-24, not copied from the documentation.

| | Without a token | With a token |
|---|---|---|
| Repository lookups (`owner/repo`) | ~60 requests per hour, counted **per IP address** | 5,000 requests per hour, counted per token |
| Search requests | **10 per minute** | 30 per minute |
| Repositories per search request | up to 100 | up to 100 |

So without a token, a search run gets roughly 1,000 repositories per minute at
best, and a run that asks for many specific repositories is capped near 60 of
them per hour on a given address. With a token those ceilings are about 50 times
higher.

**Supply a token.** Any GitHub account can create a personal access token at
<https://github.com/settings/tokens>, it is free, and this actor only reads
public data, so you can create it with **no scopes selected at all**. Paste it
into the *GitHub token* field, which is stored encrypted by Apify.

The actor reads GitHub's `X-RateLimit-Remaining` and `X-RateLimit-Reset` headers
on every response - including the refusals - and prints what is left in the run
log, so you can see exactly how close a run came to the ceiling.

When GitHub does refuse a request, the actor treats it as temporary and retries
from a **different proxy address**. That genuinely helps for unauthenticated
runs, because the limit is counted per IP, which is why the residential proxy is
prefilled. With a token the budget follows the token instead, so rotation buys
nothing and the actor does not bother.

### The other limit: search stops at 1,000 results

GitHub will not page past the 1,000th search result. Asking for result 1,001
answers `HTTP 422: Only the first 1000 search results are available` - verified
against the live API, not quoted. A query that reports 5,505 matches will still
yield only 1,000 rows, whatever you set *Max items* to.

This is a limit of GitHub's search, not of this actor, and there is a clean way
around it: split the query with qualifiers so each slice holds under 1,000
matches. For example, instead of `language:python stars:>1000`, run
`language:python stars:1000..2000`, `language:python stars:2000..5000` and
`language:python stars:>5000` as three queries. The actor de-duplicates across
queries, so overlapping slices cost you nothing.

When a run hits the ceiling, the log says so explicitly.

### Input

| Field | What it does |
|---|---|
| `searchQueries` | GitHub search queries, e.g. `language:rust stars:>500`, `topic:llm pushed:>2026-01-01`. |
| `repositories` | Specific repositories, as `owner/repo` or as a github.com URL. |
| `maxItems` | Cap on repositories returned across everything, after de-duplication. Default 100. |
| `sort` | `stars`, `forks`, `updated`, `help-wanted-issues`, or best match. Search only. |
| `order` | `desc` or `asc`. GitHub ignores it unless `sort` is set. |
| `githubToken` | Optional, secret. See above - this is the single biggest lever on what a run can do. |
| `proxyConfiguration` | Recommended for unauthenticated runs. |

At least one of `searchQueries` or `repositories` must be supplied; a run with
neither fails immediately with a message saying so rather than quietly producing
an empty dataset.

Repositories are de-duplicated by GitHub's numeric repository id before anything
is charged, so a repository that appears in two queries, or in both a query and
the explicit list, is delivered and billed exactly once.

### Output

One row per repository:

| Field | Meaning |
|---|---|
| `id` | GitHub's numeric repository id. Stable across renames. |
| `full_name` | `owner/repo`. |
| `name` | Repository name alone. |
| `owner`, `owner_type`, `owner_url` | Account name, `User` or `Organization`, and its profile URL. |
| `description` | Repository description, or `null` when there is none. |
| `url` | The github.com page. |
| `homepage` | Project website, or `null`. |
| `stars` | Stargazers. |
| `forks` | Forks. |
| `open_issues` | Open issues **and** open pull requests - GitHub counts pull requests as issues here. |
| `watchers` | People subscribed to notifications. See the caveat below. |
| `language` | GitHub's primary-language guess, or `null` for an empty or unrecognised repository. |
| `topics` | Array of topic strings, `[]` when none are set. |
| `license`, `license_spdx_id` | e.g. `Apache License 2.0` and `Apache-2.0`. Both `null` when no licence is detected. |
| `created_at`, `updated_at`, `pushed_at` | ISO 8601 UTC. `pushed_at` is the one that tells you whether a project is alive. |
| `default_branch` | e.g. `main`, `master`. |
| `archived`, `fork` | Booleans. |
| `size` | Repository size in **kilobytes**, as GitHub reports it. |

#### What the source will not give you

- **`watchers` is `null` for search results.** This is not an omission on our
  side. GitHub's search response does contain a `watchers_count` field, but its
  value is byte-for-byte the star count under an old name, not a watcher count.
  The real number lives in `subscribers_count` and GitHub returns it only from
  the single-repository endpoint. So rows that came from `repositories` carry a
  true watcher count and rows that came from `searchQueries` carry `null`. If
  you need watchers for search hits, feed the `full_name` values back in through
  `repositories` - at the cost of one API request each.
- **Missing values are `null`, never `0` or `""`.** A repository with no licence
  reports `license: null`, not `"None"`. A zero would read as a measurement.
- **Private repositories are never returned**, with or without a token; this
  actor asks only for public data.
- **Contributor counts, commit counts, release data and traffic statistics** are
  not included. Each needs one or more extra API requests per repository, which
  against a 60-per-hour unauthenticated budget would cripple the run.

### Errors and what they mean

- Rate limits, `403`, `429` and `5xx` are treated as temporary: the actor backs
  off, rotates its address and retries.
- A `404` (private, renamed or deleted repository) and a rejected token (`401`)
  are permanent. The actor records them, skips them and carries on with the rest
  of the batch rather than retrying something that cannot succeed.
- Failures are listed individually in the run log, so a partially successful run
  tells you exactly which inputs it could not read.

# Actor input Schema

## `searchQueries` (type: `array`):

GitHub repository search queries, one per line, using GitHub's own search syntax - for example 'language:python stars:>1000', 'topic:machine-learning pushed:>2026-01-01' or 'awesome list in:name'. GitHub returns at most 1,000 results per query however many matches it reports, so narrow a broad query with qualifiers instead of raising Max items. Supply this, or Repositories, or both.

## `repositories` (type: `array`):

Specific repositories to look up as 'owner/repo' or as a github.com URL - for example 'psf/requests' or 'https://github.com/psf/requests'. These are fetched directly rather than searched, and unlike search results they carry a real watcher count. Supply this, or Search queries, or both.

## `maxItems` (type: `integer`):

Maximum number of repositories to return across all queries and lookups combined, after duplicates are removed. Each page of search results costs one GitHub API request and holds up to 100 repositories, which matters against the rate limits described in the README.

## `sort` (type: `string`):

How GitHub should order search results. 'best match' is GitHub's own relevance ranking and is the default. Ignored for the Repositories lookups, which are fetched by name.

## `order` (type: `string`):

Descending or ascending. GitHub ignores this unless a sort field is chosen above.

## `githubToken` (type: `string`):

Optional GitHub personal access token, stored encrypted. Without one, GitHub allows about 60 API requests per hour per IP address and only 10 search requests per minute, so large runs stop early. A free token with no scopes at all raises that to 5,000 requests per hour and 30 searches per minute. Create one at https://github.com/settings/tokens - this actor only reads public data, so grant it no scopes.

## `proxyConfiguration` (type: `object`):

Recommended when running without a token, because GitHub counts the unauthenticated rate limit per IP address and this actor rotates to a fresh address when it is refused. With a token the limit follows the token instead, so a proxy adds nothing.

## Actor input object example

```json
{
  "searchQueries": [
    "language:python stars:>10000"
  ],
  "repositories": [],
  "maxItems": 100,
  "sort": "",
  "order": "desc",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `repositories` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "searchQueries": [
        "language:python stars:>10000"
    ],
    "repositories": [],
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("superslowsloth/github-repository-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "searchQueries": ["language:python stars:>10000"],
    "repositories": [],
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("superslowsloth/github-repository-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "searchQueries": [
    "language:python stars:>10000"
  ],
  "repositories": [],
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}' |
apify call superslowsloth/github-repository-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,superslowsloth/github-repository-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/lvCztKDTlPdGY08Dc/builds/VjZ2ZB3lzlxVzmBPn/openapi.json