The Better GitHub Repositories Scraper — Fast, Affordable GitHub Repo Data Extraction

Scrape GitHub repositories at scale and export clean, structured data — stars, forks, description, primary language, owner, open issues, topics, and repository URL — straight to JSON, CSV, or Excel. This GitHub repository scraper searches the official GitHub REST search API by any keyword, language, or topic and returns normalized results you can drop into a spreadsheet, a database, or your own app in seconds.

Because it runs HTTP-only — no headless browser, no residential proxy — it uses fewer compute units per run than browser-based scrapers, which keeps it affordable, efficient, and low compute usage even when you pull thousands of repositories. It's a low-cost way to build GitHub datasets for market research, developer analytics, dependency discovery, tech trend analysis, and lead generation.

What it extracts

For every matching repository the actor returns:

• fullName — owner/repo full name (e.g. facebook/react)
• description — the repository's short description
• stars — stargazer count
• forks — fork count
• language — primary programming language
• url — canonical https://github.com/... repository URL
• owner — the owning user or organization login
• openIssues — number of open issues
• updatedAt — ISO timestamp of the last update
• topics — array of repository topic tags

Features

• Keyword, language, and topic search using the exact same qualifiers as the GitHub search bar (language:python, topic:cli, stars:>1000, and more).
• Sort and order controls — sort by stars, forks, or most recently updated, ascending or descending.
• Pagination — fetch up to 1000 repositories per query (100 per page, up to 10 pages).
• Optional GitHub token to raise the API rate limit from 10 to 30 search requests per minute.
• HTTP-only, no browser — minimal CPU and memory, so runs finish fast and cost less.
• Clean, stable output schema ready for JSON, CSV, Excel, or the Apify dataset API.
• Robust parsing — missing fields are normalized (numbers default to 0, absent topics become []) so your downstream pipeline never breaks.

Input

Field	Type	Default	Description
query	string	(required)	GitHub search query using standard GitHub qualifiers, e.g. machine learning language:python stars:>1000.
sort	string (stars | forks | updated)	stars	Which field to sort results by.
order	string (desc | asc)	desc	Sort direction.
maxPages	integer (1–10)	3	Number of result pages to fetch, 100 repositories per page.
githubToken	string (secret)	(none)	Optional GitHub personal access token to raise the rate limit. Only public data is read.
proxyConfiguration	object	{ "useApifyProxy": false }	Optional proxy settings. Datacenter proxies are sufficient; no residential proxy required.

Example input

{
  "query": "web scraper language:typescript stars:>500",
  "sort": "stars",
  "order": "desc",
  "maxPages": 3
}

Output

Each item in the dataset looks like this:

{
  "fullName": "facebook/react",
  "description": "The library for web and native user interfaces.",
  "stars": 228000,
  "forks": 46600,
  "language": "JavaScript",
  "url": "https://github.com/facebook/react",
  "owner": "facebook",
  "openIssues": 950,
  "updatedAt": "2026-07-24T08:15:30Z",
  "topics": ["react", "javascript", "frontend", "ui", "library"]
}

Export the whole dataset as JSON, CSV, Excel, HTML, or XML from the Apify console or the API.

Example use cases

• Developer market research — find the most-starred repositories in a language or topic and track their growth over time.
• Open-source trend analysis — discover trending GitHub projects, rising libraries, and popular frameworks by stars and recent updates.
• Competitive & ecosystem intelligence — map every repository around a topic, framework, or company to understand a technology ecosystem.
• Lead generation for developer tools — build lists of active repositories and their owning organizations for outreach.
• Dependency & tooling discovery — surface popular libraries, CLIs, and SDKs for a given language or use case.
• Data science datasets — assemble labeled GitHub repository datasets (stars, forks, language, topics) for analysis or ML.

How it works

1. You provide a query plus optional sort, order, and page settings.
2. The actor calls the public GitHub REST search API (/search/repositories) over HTTPS, requesting 100 repositories per page.
3. Each JSON response is parsed by a small, pure parsing function that normalizes every repository into the stable output schema above.
4. Results are pushed to the Apify dataset, ready to download or consume via the API.

No browser is launched and no page is rendered — the actor talks directly to a JSON API, which is why it is so fast and cheap to run.

Why it's efficient and affordable

• HTTP-only, no headless browser. Browser-based scrapers spend most of their compute rendering pages. This actor makes plain HTTP requests to a JSON API, so it needs far less CPU and memory and consumes fewer compute units per run.
• No residential proxy required. The GitHub API is a public endpoint, so datacenter connectivity is enough. You avoid the biggest recurring cost of most scrapers.
• Batched requests. Up to 100 repositories are returned per request, minimizing the number of round trips.
• Lean dependencies and a tight parser. Fast startup and predictable, low-cost runs whether you fetch 100 repositories or 1000.

The result is a low-cost, efficient GitHub scraper that gives you the same structured data as heavier tools at a fraction of the compute.

FAQ

Do I need a GitHub token to use this GitHub repository scraper? No. It works without any credentials. Adding an optional personal access token simply raises the search rate limit from 10 to 30 requests per minute, which helps when you fetch many pages.

How many repositories can I scrape per query? The GitHub search API caps results at 1000 per query (10 pages of 100). Refine your query with qualifiers like stars:>1000 or language:go to slice a large space into targeted, high-value result sets.

What output formats are supported? The dataset can be exported as JSON, CSV, Excel, HTML, RSS, or XML from the Apify console or fetched programmatically via the API.

Why is this scraper cheaper than browser-based scrapers? It never opens a browser and never needs residential proxies. It reads a public JSON API over HTTP, so it uses less compute and finishes faster, which means fewer compute units and lower cost per run.

Can I search by topic or programming language? Yes. Use standard GitHub qualifiers in the query field, for example topic:machine-learning, language:rust, or combine them: topic:cli language:go stars:>500.

Legal and ethical note

This actor scrapes only public data through GitHub's official public REST API. Use it responsibly: respect GitHub's Terms of Service and API rate limits, honor robots.txt and applicable site policies, and do not collect or store personal data in violation of privacy laws such as the GDPR or CCPA. You are responsible for how you use the data you collect.