GitHub Repository Scraper

Search GitHub for repositories using GitHub's own search syntax, or look up specific repositories by owner/repo, and get a clean row per repository: stars, forks, open issues, language, topics, licence, timestamps and more.

Everything comes from GitHub's public REST API, so the numbers are GitHub's own rather than something scraped off a rendered page.

Read this first: the rate limit is the thing that will bite you

GitHub meters its API, and how much this actor can do in one run depends almost entirely on whether you give it a token. These figures were measured against the live API on 2026-08-24, not copied from the documentation.

	Without a token	With a token
Repository lookups (owner/repo)	~60 requests per hour, counted per IP address	5,000 requests per hour, counted per token
Search requests	10 per minute	30 per minute
Repositories per search request	up to 100	up to 100

So without a token, a search run gets roughly 1,000 repositories per minute at best, and a run that asks for many specific repositories is capped near 60 of them per hour on a given address. With a token those ceilings are about 50 times higher.

Supply a token. Any GitHub account can create a personal access token at https://github.com/settings/tokens, it is free, and this actor only reads public data, so you can create it with no scopes selected at all. Paste it into the GitHub token field, which is stored encrypted by Apify.

The actor reads GitHub's X-RateLimit-Remaining and X-RateLimit-Reset headers on every response - including the refusals - and prints what is left in the run log, so you can see exactly how close a run came to the ceiling.

When GitHub does refuse a request, the actor treats it as temporary and retries from a different proxy address. That genuinely helps for unauthenticated runs, because the limit is counted per IP, which is why the residential proxy is prefilled. With a token the budget follows the token instead, so rotation buys nothing and the actor does not bother.

The other limit: search stops at 1,000 results

GitHub will not page past the 1,000th search result. Asking for result 1,001 answers HTTP 422: Only the first 1000 search results are available - verified against the live API, not quoted. A query that reports 5,505 matches will still yield only 1,000 rows, whatever you set Max items to.

This is a limit of GitHub's search, not of this actor, and there is a clean way around it: split the query with qualifiers so each slice holds under 1,000 matches. For example, instead of language:python stars:>1000, run language:python stars:1000..2000, language:python stars:2000..5000 and language:python stars:>5000 as three queries. The actor de-duplicates across queries, so overlapping slices cost you nothing.

When a run hits the ceiling, the log says so explicitly.

Input

Field	What it does
searchQueries	GitHub search queries, e.g. language:rust stars:>500, topic:llm pushed:>2026-01-01.
repositories	Specific repositories, as owner/repo or as a github.com URL.
maxItems	Cap on repositories returned across everything, after de-duplication. Default 100.
sort	stars, forks, updated, help-wanted-issues, or best match. Search only.
order	desc or asc. GitHub ignores it unless sort is set.
githubToken	Optional, secret. See above - this is the single biggest lever on what a run can do.
proxyConfiguration	Recommended for unauthenticated runs.

At least one of searchQueries or repositories must be supplied; a run with neither fails immediately with a message saying so rather than quietly producing an empty dataset.

Repositories are de-duplicated by GitHub's numeric repository id before anything is charged, so a repository that appears in two queries, or in both a query and the explicit list, is delivered and billed exactly once.

Output

One row per repository:

Field	Meaning
id	GitHub's numeric repository id. Stable across renames.
full_name	owner/repo.
name	Repository name alone.
owner, owner_type, owner_url	Account name, User or Organization, and its profile URL.
description	Repository description, or null when there is none.
url	The github.com page.
homepage	Project website, or null.
stars	Stargazers.
forks	Forks.
open_issues	Open issues and open pull requests - GitHub counts pull requests as issues here.
watchers	People subscribed to notifications. See the caveat below.
language	GitHub's primary-language guess, or null for an empty or unrecognised repository.
topics	Array of topic strings, [] when none are set.
license, license_spdx_id	e.g. Apache License 2.0 and Apache-2.0. Both null when no licence is detected.
created_at, updated_at, pushed_at	ISO 8601 UTC. pushed_at is the one that tells you whether a project is alive.
default_branch	e.g. main, master.
archived, fork	Booleans.
size	Repository size in kilobytes, as GitHub reports it.

What the source will not give you

• watchers is null for search results. This is not an omission on our side. GitHub's search response does contain a watchers_count field, but its value is byte-for-byte the star count under an old name, not a watcher count. The real number lives in subscribers_count and GitHub returns it only from the single-repository endpoint. So rows that came from repositories carry a true watcher count and rows that came from searchQueries carry null. If you need watchers for search hits, feed the full_name values back in through repositories - at the cost of one API request each.
• Missing values are null, never 0 or "". A repository with no licence reports license: null, not "None". A zero would read as a measurement.
• Private repositories are never returned, with or without a token; this actor asks only for public data.
• Contributor counts, commit counts, release data and traffic statistics are not included. Each needs one or more extra API requests per repository, which against a 60-per-hour unauthenticated budget would cripple the run.

Errors and what they mean

• Rate limits, 403, 429 and 5xx are treated as temporary: the actor backs off, rotates its address and retries.
• A 404 (private, renamed or deleted repository) and a rejected token (401) are permanent. The actor records them, skips them and carries on with the rest of the batch rather than retrying something that cannot succeed.
• Failures are listed individually in the run log, so a partially successful run tells you exactly which inputs it could not read.