# The Better GitHub Repositories Scraper (`betterscrapers/the-better-github-repos-scraper`) Actor

Affordable, HTTP-only GitHub repository scraper. Search GitHub repos by keyword, language, or topic and export stars, forks, description, owner, issues, topics and more as JSON/CSV. Low compute usage.

- **URL**: https://apify.com/betterscrapers/the-better-github-repos-scraper.md
- **Developed by:** [Better Scrapers](https://apify.com/betterscrapers) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.60 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## The Better GitHub Repositories Scraper — Fast, Affordable GitHub Repo Data Extraction

Scrape **GitHub repositories** at scale and export clean, structured data — **stars, forks, description, primary language, owner, open issues, topics, and repository URL** — straight to JSON, CSV, or Excel. This **GitHub repository scraper** searches the official GitHub REST search API by any keyword, language, or topic and returns normalized results you can drop into a spreadsheet, a database, or your own app in seconds.

Because it runs **HTTP-only** — no headless browser, no residential proxy — it uses **fewer compute units per run** than browser-based scrapers, which keeps it **affordable, efficient, and low compute usage** even when you pull thousands of repositories. It's a low-cost way to build GitHub datasets for market research, developer analytics, dependency discovery, tech trend analysis, and lead generation.

### What it extracts

For every matching repository the actor returns:

- **fullName** — `owner/repo` full name (e.g. `facebook/react`)
- **description** — the repository's short description
- **stars** — stargazer count
- **forks** — fork count
- **language** — primary programming language
- **url** — canonical `https://github.com/...` repository URL
- **owner** — the owning user or organization login
- **openIssues** — number of open issues
- **updatedAt** — ISO timestamp of the last update
- **topics** — array of repository topic tags

### Features

- **Keyword, language, and topic search** using the exact same qualifiers as the GitHub search bar (`language:python`, `topic:cli`, `stars:>1000`, and more).
- **Sort and order controls** — sort by stars, forks, or most recently updated, ascending or descending.
- **Pagination** — fetch up to 1000 repositories per query (100 per page, up to 10 pages).
- **Optional GitHub token** to raise the API rate limit from 10 to 30 search requests per minute.
- **HTTP-only, no browser** — minimal CPU and memory, so runs finish fast and cost less.
- **Clean, stable output schema** ready for JSON, CSV, Excel, or the Apify dataset API.
- **Robust parsing** — missing fields are normalized (numbers default to `0`, absent topics become `[]`) so your downstream pipeline never breaks.

### Input

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `query` | string | *(required)* | GitHub search query using standard GitHub qualifiers, e.g. `machine learning language:python stars:>1000`. |
| `sort` | string (`stars` \| `forks` \| `updated`) | `stars` | Which field to sort results by. |
| `order` | string (`desc` \| `asc`) | `desc` | Sort direction. |
| `maxPages` | integer (1–10) | `3` | Number of result pages to fetch, 100 repositories per page. |
| `githubToken` | string (secret) | *(none)* | Optional GitHub personal access token to raise the rate limit. Only public data is read. |
| `proxyConfiguration` | object | `{ "useApifyProxy": false }` | Optional proxy settings. Datacenter proxies are sufficient; no residential proxy required. |

#### Example input

```json
{
  "query": "web scraper language:typescript stars:>500",
  "sort": "stars",
  "order": "desc",
  "maxPages": 3
}
````

### Output

Each item in the dataset looks like this:

```json
{
  "fullName": "facebook/react",
  "description": "The library for web and native user interfaces.",
  "stars": 228000,
  "forks": 46600,
  "language": "JavaScript",
  "url": "https://github.com/facebook/react",
  "owner": "facebook",
  "openIssues": 950,
  "updatedAt": "2026-07-24T08:15:30Z",
  "topics": ["react", "javascript", "frontend", "ui", "library"]
}
```

Export the whole dataset as **JSON, CSV, Excel, HTML, or XML** from the Apify console or the API.

### Example use cases

- **Developer market research** — find the most-starred repositories in a language or topic and track their growth over time.
- **Open-source trend analysis** — discover trending GitHub projects, rising libraries, and popular frameworks by stars and recent updates.
- **Competitive & ecosystem intelligence** — map every repository around a topic, framework, or company to understand a technology ecosystem.
- **Lead generation for developer tools** — build lists of active repositories and their owning organizations for outreach.
- **Dependency & tooling discovery** — surface popular libraries, CLIs, and SDKs for a given language or use case.
- **Data science datasets** — assemble labeled GitHub repository datasets (stars, forks, language, topics) for analysis or ML.

### How it works

1. You provide a `query` plus optional sort, order, and page settings.
2. The actor calls the public **GitHub REST search API** (`/search/repositories`) over HTTPS, requesting 100 repositories per page.
3. Each JSON response is parsed by a small, pure parsing function that normalizes every repository into the stable output schema above.
4. Results are pushed to the Apify dataset, ready to download or consume via the API.

No browser is launched and no page is rendered — the actor talks directly to a JSON API, which is why it is so fast and cheap to run.

### Why it's efficient and affordable

- **HTTP-only, no headless browser.** Browser-based scrapers spend most of their compute rendering pages. This actor makes plain HTTP requests to a JSON API, so it needs far less CPU and memory and **consumes fewer compute units per run**.
- **No residential proxy required.** The GitHub API is a public endpoint, so datacenter connectivity is enough. You avoid the biggest recurring cost of most scrapers.
- **Batched requests.** Up to 100 repositories are returned per request, minimizing the number of round trips.
- **Lean dependencies and a tight parser.** Fast startup and predictable, low-cost runs whether you fetch 100 repositories or 1000.

The result is a **low-cost, efficient GitHub scraper** that gives you the same structured data as heavier tools at a fraction of the compute.

### FAQ

**Do I need a GitHub token to use this GitHub repository scraper?**
No. It works without any credentials. Adding an optional personal access token simply raises the search rate limit from 10 to 30 requests per minute, which helps when you fetch many pages.

**How many repositories can I scrape per query?**
The GitHub search API caps results at 1000 per query (10 pages of 100). Refine your `query` with qualifiers like `stars:>1000` or `language:go` to slice a large space into targeted, high-value result sets.

**What output formats are supported?**
The dataset can be exported as JSON, CSV, Excel, HTML, RSS, or XML from the Apify console or fetched programmatically via the API.

**Why is this scraper cheaper than browser-based scrapers?**
It never opens a browser and never needs residential proxies. It reads a public JSON API over HTTP, so it uses less compute and finishes faster, which means **fewer compute units and lower cost per run**.

**Can I search by topic or programming language?**
Yes. Use standard GitHub qualifiers in the `query` field, for example `topic:machine-learning`, `language:rust`, or combine them: `topic:cli language:go stars:>500`.

### Legal and ethical note

This actor scrapes **only public data** through GitHub's official public REST API. Use it responsibly: respect GitHub's Terms of Service and API rate limits, honor `robots.txt` and applicable site policies, and do not collect or store personal data in violation of privacy laws such as the GDPR or CCPA. You are responsible for how you use the data you collect.

# Actor input Schema

## `query` (type: `string`):

GitHub search query, using the same qualifiers as the GitHub search bar. Examples: 'machine learning language:python', 'topic:cli stars:>1000', 'web scraper'.

## `sort` (type: `string`):

Which field to sort results by.

## `order` (type: `string`):

Sort direction for the selected field.

## `maxPages` (type: `integer`):

How many result pages to fetch (100 repositories per page). The GitHub search API returns at most 1000 results (10 pages) per query.

## `githubToken` (type: `string`):

Optional GitHub personal access token. Not required, but sending one raises the API rate limit from 10 to 30 search requests per minute. Only public data is read.

## `proxyConfiguration` (type: `object`):

Optional proxy settings. This actor talks only to the public GitHub API over HTTP, so datacenter proxies are sufficient — no residential proxy is needed. Leave disabled to run at the lowest cost.

## Actor input object example

```json
{
  "query": "web scraper language:typescript stars:>500",
  "sort": "stars",
  "order": "desc",
  "maxPages": 3,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "web scraper language:typescript stars:>500",
    "sort": "stars",
    "order": "desc",
    "maxPages": 3,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("betterscrapers/the-better-github-repos-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "web scraper language:typescript stars:>500",
    "sort": "stars",
    "order": "desc",
    "maxPages": 3,
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("betterscrapers/the-better-github-repos-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "web scraper language:typescript stars:>500",
  "sort": "stars",
  "order": "desc",
  "maxPages": 3,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call betterscrapers/the-better-github-repos-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=betterscrapers/the-better-github-repos-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "The Better GitHub Repositories Scraper",
        "description": "Affordable, HTTP-only GitHub repository scraper. Search GitHub repos by keyword, language, or topic and export stars, forks, description, owner, issues, topics and more as JSON/CSV. Low compute usage.",
        "version": "0.1",
        "x-build-id": "NNoGhz7W6mG5Muo1E"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/betterscrapers~the-better-github-repos-scraper/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-betterscrapers-the-better-github-repos-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/betterscrapers~the-better-github-repos-scraper/runs": {
            "post": {
                "operationId": "runs-sync-betterscrapers-the-better-github-repos-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/betterscrapers~the-better-github-repos-scraper/run-sync": {
            "post": {
                "operationId": "run-sync-betterscrapers-the-better-github-repos-scraper",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "query"
                ],
                "properties": {
                    "query": {
                        "title": "Search query",
                        "type": "string",
                        "description": "GitHub search query, using the same qualifiers as the GitHub search bar. Examples: 'machine learning language:python', 'topic:cli stars:>1000', 'web scraper'."
                    },
                    "sort": {
                        "title": "Sort by",
                        "enum": [
                            "stars",
                            "forks",
                            "updated"
                        ],
                        "type": "string",
                        "description": "Which field to sort results by.",
                        "default": "stars"
                    },
                    "order": {
                        "title": "Sort order",
                        "enum": [
                            "desc",
                            "asc"
                        ],
                        "type": "string",
                        "description": "Sort direction for the selected field.",
                        "default": "desc"
                    },
                    "maxPages": {
                        "title": "Max pages",
                        "minimum": 1,
                        "maximum": 10,
                        "type": "integer",
                        "description": "How many result pages to fetch (100 repositories per page). The GitHub search API returns at most 1000 results (10 pages) per query.",
                        "default": 3
                    },
                    "githubToken": {
                        "title": "GitHub token (optional)",
                        "type": "string",
                        "description": "Optional GitHub personal access token. Not required, but sending one raises the API rate limit from 10 to 30 search requests per minute. Only public data is read."
                    },
                    "proxyConfiguration": {
                        "title": "Proxy configuration",
                        "type": "object",
                        "description": "Optional proxy settings. This actor talks only to the public GitHub API over HTTP, so datacenter proxies are sufficient — no residential proxy is needed. Leave disabled to run at the lowest cost.",
                        "default": {
                            "useApifyProxy": false
                        }
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```