GitHub Repo Scraper · Repositories, Stars, Topics & Languages

Scrape GitHub repositories by programming language, topic tag, star counts, open-source license, organization handle, and commit update windows. Returns structured repository metrics, star counts, fork counts, watcher counts, open issue counts, primary language, topics array, and update timestamps without requiring login or API tokens.

Maintained by reapX. Every row cites the Apify run that produced it — nothing is inferred, modelled or filled in, and a field absent from the source is absent from the row. The extracted archive for this source is browsable at reapx.dev/data/github-repo-scraper/ and mirrored as an open dataset on Hugging Face and Kaggle. Questions: reapxdev@proton.me

---

⬇️ Input

The actor accepts structured JSON input to define search criteria, filters, sorting options, and record capacity limits.

Input Parameters

Parameter	Type	Default / Prefill	Description
query	String	stars:>1000 language:python	Custom GitHub search query expression.
language	String	python	Filter repositories by primary programming language (e.g. python, typescript, rust, go).
topic	String	""	Filter repositories tagged with a specific topic (e.g. machine-learning, web3, cli).
minStars	Integer	0	Minimum stargazer count floor for matching repositories.
maxStars	Integer	0	Maximum stargazer count ceiling for matching repositories (0 for unlimited).
license	String	""	Filter by open-source SPDX license identifier (e.g. mit, apache-2.0, gpl-3.0).
organization	String	""	Filter repositories owned by a specific GitHub user or organization handle (e.g. facebook, google).
pushedAfter	String	""	Filter repositories with code pushed on or after this ISO date (YYYY-MM-DD).
sort	String	"stars"	Sort criterion ("stars", "forks", "help-wanted-issues", "updated").
order	String	"desc"	Sort direction ("desc", "asc").
maxItems	Integer	50	Maximum number of repository records to scrape and push to dataset (1 to 5000).
githubToken	String	""	Optional personal access token for higher rate limits (never stored or logged).

Example Input Configuration

{
  "language": "python",
  "topic": "machine-learning",
  "minStars": 1000,
  "sort": "stars",
  "order": "desc",
  "maxItems": 100
}

---

⬆️ Output

Every repository record is pushed item-by-item to the run's default dataset. Each row corresponds to a single GitHub repository identified by its canonical slug (owner/repo).

Output Fields

Field Name	Type	Description	Example
slug	String	Canonical repository slug (owner/repo). Serves as primary key.	"facebook/react"
repoId	Integer	Unique numeric GitHub repository identifier.	10270250
name	String	Repository name without owner prefix.	"react"
fullName	String	Full repository owner/name path.	"facebook/react"
owner	String	GitHub username or organization handle owning the repository.	"facebook"
ownerType	String	Account type ("Organization" or "User").	"Organization"
ownerAvatarUrl	String	Profile avatar URL for owner.	"https://avatars.githubusercontent.com/u/69631?v=4"
htmlUrl	String	Public web URL on GitHub.	"https://github.com/facebook/react"
description	String	Text bio or summary of repository.	"The library for web and native user interfaces."
stars	Integer	Total stargazer count.	225000
forks	Integer	Total forks count.	45000
watchers	Integer	Subscriber count watching repository.	6700
openIssues	Integer	Total open issues and pull requests.	1200
language	String	Primary programming language detected by GitHub.	"JavaScript"
topics	Array	Array of topic strings attached to repository.	["declarative", "frontend", "library", "react"]
license	String	SPDX license key.	"mit"
licenseName	String	Human-readable license title.	"MIT License"
isFork	Boolean	Whether repository is a fork.	false
isArchived	Boolean	Whether repository is archived (read-only).	false
isDisabled	Boolean	Whether repository is disabled by GitHub.	false
defaultBranch	String	Primary git branch name.	"main"
createdAt	String	ISO 8601 creation timestamp.	"2013-05-24T16:15:54Z"
updatedAt	String	ISO 8601 metadata update timestamp.	"2026-08-03T18:30:00Z"
pushedAt	String	ISO 8601 last push timestamp.	"2026-08-03T19:00:00Z"
scrapedAt	String	ISO 8601 UTC timestamp of scraping run.	"2026-08-03T20:00:00Z"

Sample Dataset Output Item

{
  "slug": "facebook/react",
  "repoId": 10270250,
  "name": "react",
  "fullName": "facebook/react",
  "owner": "facebook",
  "ownerType": "Organization",
  "ownerAvatarUrl": "https://avatars.githubusercontent.com/u/69631?v=4",
  "htmlUrl": "https://github.com/facebook/react",
  "description": "The library for web and native user interfaces.",
  "stars": 225000,
  "forks": 45000,
  "watchers": 6700,
  "openIssues": 1200,
  "language": "JavaScript",
  "topics": [
    "declarative",
    "frontend",
    "javascript",
    "library",
    "react",
    "ui"
  ],
  "license": "mit",
  "licenseName": "MIT License",
  "isFork": false,
  "isArchived": false,
  "isDisabled": false,
  "defaultBranch": "main",
  "createdAt": "2013-05-24T16:15:54Z",
  "updatedAt": "2026-08-03T18:30:00Z",
  "pushedAt": "2026-08-03T19:00:00Z",
  "scrapedAt": "2026-08-03T20:00:00Z"
}

---

How it works

The actor issues targeted REST HTTP queries directly to GitHub's public Search API (https://api.github.com/search/repositories).

1. Query Construction: Inputs such as language, topic, minStars, maxStars, license, organization, and pushedAfter are combined into a canonical GitHub search expression.
2. Paced Execution: Requests are executed with automatic backoff and rate-limit header inspection. Unauthenticated requests are throttled at safe intervals to comply with GitHub's 10 requests per minute search ceiling.
3. Item-by-Item Pushing: Each repository record is cleaned, formatted into a standard schema, charged at $0.001 per event, and pushed directly to Apify's default dataset in real time.
4. Resilience: In case of rate limits or transient network errors, the client backs off exponentially without failing or discarding previously pushed rows.

---

Features & Use Cases

• Open Source Ecosystem Intelligence: Track trending repositories across Rust, Python, Go, and TypeScript.
• Competitive & Vendor Analysis: Monitor open-source activity for major tech companies like Meta, Google, Microsoft, and Vercel.
• Lead Generation & Developer Marketing: Identify fast-growing developer tools and active maintainer communities.
• License & Compliance Audits: Screen repositories by SPDX license key (mit, apache-2.0, gpl-3.0) for enterprise compliance.
• AI & ML Model Discovery: Discover newly updated Large Language Model (LLM) tooling and computer vision repositories.

---

Code Example — Python Apify Client

from apify_client import ApifyClient

# Initialize the Apify Client with your API token
client = ApifyClient("YOUR_APIFY_TOKEN")

# Prepare actor input configuration
run_input = {
    "language": "rust",
    "topic": "cli",
    "minStars": 1000,
    "sort": "stars",
    "order": "desc",
    "maxItems": 50
}

# Run the actor and wait for completion
run = client.actor("reapx/github-repo-scraper").call(run_input=run_input)

# Fetch dataset items
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"{item['slug']} ({item['stars']} stars) - {item['htmlUrl']}")

---

Pricing

Pay-per-event pricing: $0.001 per repository record scraped.

• Tiered Volume Discounts: Discount tiers apply automatically from FREE through DIAMOND (up to 25% discount).
• Zero Charge Guarantee: If no repositories match your search criteria or the source is unreachable, you are charged $0.
• Platform Usage Included: Apify platform compute fees are absorbed into the event price.

---

❓ FAQ

Does this scraper require a GitHub account or API token?

No account or API token is required. The scraper reads public GitHub data over unauthenticated HTTP. Providing an optional githubToken increases rate limits from 10 to 30 requests per minute.

What is the primary key / identifier for entity pages?

The primary identifier is slug formatted as owner/repo (e.g. facebook/react). The numeric ID is saved as repoId to prevent collision with named entity fields.

How fast does the scraper execute?

The default run of 50 repositories completes in under 15 seconds.

Can I filter by last commit push date?

Yes, use pushedAfter with an ISO date (e.g. 2026-01-01) to filter for repositories actively updated after that date.

---

💬 Your feedback

We continuously improve this actor based on user feedback. If you encounter bugs, missing fields, or feature requests:

• Submit an issue on the Apify Console actor page.
• Contact maintainers directly at reapxdev@proton.me.

---

Disclaimer

Unofficial - not affiliated with GitHub. Collects public data only. reapx. Contact reapxdev@proton.me.

🧪 Example input

A real, runnable configuration — this is an actual input this Actor has run with.

{
  "language": "python",
  "minStars": 500,
  "sort": "stars",
  "order": "desc",
  "maxItems": 50
}

📄 Sample output

One real row from a real run of this Actor, unedited.

{
  "slug": "codecrafters-io/build-your-own-x",
  "repoId": 132750724,
  "name": "build-your-own-x",
  "fullName": "codecrafters-io/build-your-own-x",
  "owner": "codecrafters-io",
  "ownerType": "Organization",
  "ownerAvatarUrl": "https://avatars.githubusercontent.com/u/58904235?v=4",
  "htmlUrl": "https://github.com/codecrafters-io/build-your-own-x",
  "description": "Master programming by recreating your favorite technologies from scratch.",
  "stars": 535588,
  "forks": 50598,
  "watchers": 535588,
  "openIssues": 619,
  "language": "Markdown",
  "topics": [
    "awesome-list",
    "free",
    "programming",
    "tutorial-code",
    "tutorial-exercises",
    "tutorials"
  ],
  "license": "",
  "licenseName": "",
  "isFork": false,
  "isArchived": false,
  "isDisabled": false,
  "defaultBranch": "master",
  "createdAt": "2018-05-09T12:03:18Z",
  "updatedAt": "2026-08-03T18:07:04Z",
  "pushedAt": "2026-07-14T19:25:58Z",
  "scrapedAt": "2026-08-03T18:07:38Z"
}

⚠️ Run outcomes and error handling

This Actor reports what happened in the run's status message, and it always keeps whatever it collected. These are the outcomes you can get and what each one means.

Outcome	What it means
Success	Rows were returned and you were charged repository-scraped at $0.001 per row.
No matches	The source returned nothing for your filters. Nothing is charged. Widen the date window or drop a filter.
Partial - source refused	The source rate-limited or refused some requests. The affected items are skipped and named in the log, and everything already collected is still pushed. A block never discards a run's work.
Rejected filter	The source itself rejected the filter combination. The run fails fast with the source's own reason and nothing is charged.

What is guaranteed either way

• Every row is pushed as it is built, not buffered to the end of the run. Anything that buffers output loses everything to a timeout, a block or a migration; this does not.
• Retry-After is respected. When the source asks for a pause, the Actor waits the time it asked for instead of retrying into a harder block.
• A field absent from the source is absent from the row. Nothing is inferred, modelled or filled in to make a row look complete.