GitHub Scraper

Scrape data from GitHub — repositories, repository contributors, developer profiles, trending repos, and search results. Get stars, forks, contribution counts, languages, topics, licenses, followers, and more.

What does GitHub Scraper do?

GitHub Scraper extracts structured data from GitHub using its public GitHub API and web pages. It supports six modes:

• Repository details — Full metadata for specific repos (stars, forks, topics, license, dates)
• Developer profiles — Bio, followers, location, company, repos count for any user
• Repository contributors — Turn public repository URLs into developer lists with contribution counts
• Trending repositories — Today's/week's/month's hottest repos with star velocity
• Search repositories — Find repos by keyword, sorted by stars
• Search users — Find users with GitHub qualifiers and enrich every result into the full profile format

No GitHub API token is required. Anonymous access is limited to 60 API requests/hour; an optional secret token raises the standard GitHub allowance to 5,000 requests/hour.

Who is it for?

GitHub Scraper is for developer-tool founders, recruiters, open-source researchers, venture analysts, and data teams that need structured repository and profile data without maintaining GitHub scraping infrastructure.

Why scrape GitHub?

GitHub hosts 400M+ repositories and 100M+ developers. It's the primary source for:

• 📊 Tech trend analysis — Track which languages and frameworks are gaining traction
• 🔍 Competitive intelligence — Monitor competitor repos, stars growth, and release cadence
• 📈 Developer recruiting — Find active developers by language, location, and contribution history
• 🏗️ Open source research — Analyze licensing, dependency patterns, and community health
• 📰 Content creation — Curate trending repos for newsletters and social media

Use cases

• Newsletter creators sharing weekly trending repos
• VCs and investors tracking open-source momentum
• Hiring managers building candidate lists from active contributors
• Researchers studying open-source ecosystem dynamics
• DevRel teams monitoring mentions and competitive landscape
• Developers discovering new tools and libraries

How to scrape GitHub

1. Go to GitHub Scraper on Apify Store
2. Choose a mode: repos, profiles, contributors, trending, repository search, or user search
3. Enter repository/profile URLs or a search query depending on the mode
4. Set the max results limit
5. Click Start and wait for results
6. Download data as JSON, CSV, or Excel

Data you can extract

Repository data

Field	Type	Description
fullName	string	Owner/repo (e.g., facebook/react)
description	string	Repo description
stars	number	Star count
forks	number	Fork count
watchers	number	Watcher count
openIssues	number	Open issue count
language	string	Primary language
topics	array	Topic tags
license	string	License (MIT, Apache-2.0, etc.)
isArchived	boolean	Whether the repo is archived
createdAt	string	Creation date
updatedAt	string	Last update date
size	number	Repo size in KB

Profile data

Field	Type	Description
username	string	GitHub username
name	string	Display name
bio	string	Profile bio
company	string	Company
location	string	Location
followers	number	Follower count
following	number	Following count
publicRepos	number	Public repo count
blog	string	Website URL
twitterUsername	string	X/Twitter handle

Contributor data

Contributor mode uses GitHub's public /repos/{owner}/{repo}/contributors API endpoint. It emits one row per unique repository and username, preserving the order of your repository list.

Field	Type	Description
repositoryFullName	string	Repository in owner/repo format
username	string	Contributor's GitHub username
profileUrl	string	Public GitHub profile URL
avatarUrl	string or null	Public avatar URL
contributions	number	Contributions reported by GitHub for this repository
contributorType	string or null	GitHub account type, such as User or Bot
repositoryUrl	string	Canonical public repository URL
sourceUrl	string	Repository URL supplied in the input
scrapedAt	string	UTC extraction timestamp

Trending repo data

Field	Type	Description
fullName	string	Owner/repo
description	string	Repo description
language	string	Primary language
stars	number	Total stars
starsToday	number	Stars gained in the period
forks	number	Fork count
builtBy	array	Top contributors with avatars

Input parameters

Parameter	Type	Default	Description
mode	string	"trending"	Mode: repos, profiles, contributors, trending, search, users
urls	array	[]	Public HTTPS GitHub URLs (for repos, profiles, or contributors mode)
searchQuery	string	""	GitHub query (for repository search or users search)
githubToken	string	—	Optional encrypted GitHub PAT for higher API limits; sent only to api.github.com
trendingSince	string	"daily"	Trending period: daily, weekly, monthly
trendingLanguage	string	""	Filter by language (e.g., python)
maxResults	integer	25	Max results to return

Input example

{
    "mode": "users",
    "searchQuery": "location:berlin followers:>1000",
    "maxResults": 25
}

GitHub's user search qualifiers can filter by location, followers, repositories, language, and account creation date. User search returns the same enriched fields as profile mode and charges the existing profile event once per emitted row.

Existing inputs that omit mode still use trending, and all existing modes keep their current output fields and pricing events.

Output example

Repository

{
    "name": "react",
    "fullName": "facebook/react",
    "owner": "facebook",
    "description": "The library for web and native user interfaces.",
    "url": "https://github.com/facebook/react",
    "homepageUrl": "https://react.dev",
    "language": "JavaScript",
    "stars": 243711,
    "forks": 50668,
    "watchers": 243711,
    "openIssues": 1150,
    "topics": ["declarative", "frontend", "javascript", "library", "react", "ui"],
    "license": "MIT",
    "isArchived": false,
    "isFork": false,
    "defaultBranch": "main",
    "createdAt": "2013-05-24T16:15:54Z",
    "updatedAt": "2026-03-08T09:17:07Z",
    "pushedAt": "2026-03-05T15:52:24Z",
    "size": 942058,
    "scrapedAt": "2026-03-08T09:45:54.218Z"
}

Profile

{
    "username": "torvalds",
    "name": "Linus Torvalds",
    "bio": null,
    "company": "Linux Foundation",
    "location": "Portland, OR",
    "followers": 289246,
    "following": 0,
    "publicRepos": 11,
    "avatarUrl": "https://avatars.githubusercontent.com/u/1024025?v=4",
    "url": "https://github.com/torvalds",
    "scrapedAt": "2026-03-08T09:45:50.123Z"
}

How much does it cost to scrape GitHub?

GitHub Scraper uses pay-per-event pricing:

Event	Price
Run started	$0.005
Repository or profile extracted	$0.003 per item

Cost examples

Scenario	Items	Cost
Daily trending (25 repos)	25	~$0.08
10 repo details	10	~$0.04
50 contributor rows	50	~$0.16
Search (50 results)	50	~$0.16

Apify's free plan includes $5/month in platform credits — enough for ~60 trending scrapes.

Using GitHub Scraper with the Apify API

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('automation-lab/github-scraper').call({
    mode: 'trending',
    trendingSince: 'daily',
    maxResults: 25,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach(repo => {
    console.log(`${repo.fullName}: ${repo.stars} stars (+${repo.starsToday} today)`);
});

Python

from apify_client import ApifyClient

client = ApifyClient('YOUR_API_TOKEN')

run = client.actor('automation-lab/github-scraper').call(run_input={
    'mode': 'trending',
    'trendingSince': 'daily',
    'maxResults': 25,
})

dataset = client.dataset(run['defaultDatasetId']).list_items().items
for repo in dataset:
    print(f"{repo['fullName']}: {repo['stars']} stars (+{repo['starsToday']} today)")

cURL

curl "https://api.apify.com/v2/acts/automation-lab~github-scraper/run-sync-get-dataset-items?token=YOUR_API_TOKEN" \
  -X POST -H "Content-Type: application/json" \
  -d '{"mode": "trending", "trendingSince": "daily", "maxResults": 10}'

Integrations

GitHub Scraper works with all Apify integrations:

• Scheduled runs — Track trending repos daily or weekly
• Webhooks — Get notified when a scrape completes
• Google Sheets — Export repos and profiles to spreadsheets
• Slack — Post trending repos to your team's channel
• Zapier / Make — Automate workflows with GitHub data

Tips

• 📈 Track trending daily — Schedule runs to build a history of trending repos
• 🔍 Use search for competitive analysis — Search for keywords in your domain
• 👥 Profile scraping — Great for building lists of developers by location or company
• 🤝 Contributor discovery — Turn a public repository watchlist into developer leads ranked by repository contribution count
• 🏷️ Filter by language — Use trendingLanguage to focus on specific tech stacks
• ⚡ Rate limits — The actor follows GitHub's retry/reset headers for bounded retries; add a secret githubToken for larger jobs
• ✅ Deterministic URLs — URL modes require full public https://github.com/... URLs and deduplicate normalized targets in first-seen order
• 🚨 Clear failures — Exhausted API errors and empty/unparseable Trending pages fail the run instead of silently returning a misleading green empty result
• 📊 Combine modes — Run trending to discover repos, then repos mode for full details

Legality

GitHub Scraper collects publicly available repository and profile information. Use the data responsibly, respect GitHub's terms, and avoid collecting private, authenticated, or personal data beyond your lawful purpose.

Is it legal to scrape GitHub?

GitHub provides a public REST API specifically designed for programmatic access. This scraper uses that API and publicly accessible web pages. It does not bypass authentication, rate limits, or access private data. Always review GitHub's Terms of Service and API usage policies.

FAQ

Do I need a GitHub API token? No. The scraper works anonymously with GitHub's public API (60 requests/hour per IP). For larger jobs, set the optional githubToken secret input; Apify stores it encrypted, the Actor sends it only in the Authorization header to api.github.com, and it is never written to logs or datasets. A standard authenticated GitHub allowance is 5,000 requests/hour.

How many trending repos are shown? GitHub's trending page shows up to 25 repos per language/period combination.

Can I search for users or organizations? Yes. Use mode: "users" with qualifiers such as location:berlin followers:>1000; each match is enriched into a full public profile row. Repository search remains available as mode: "search".

How does contributors mode work? Provide one or more full public repository URLs such as https://github.com/facebook/react. The actor paginates the public GitHub API, deduplicates by repositoryFullName and username, and stops before exceeding maxResults. It does not enrich profiles or fetch commits, emails, followers, organizations, or authenticated data.

What if a contributor has no public avatar or account type? Optional public values are returned as null; the actor does not infer missing data.

What about private repos? The scraper only accesses public data. Private repos are not visible without authentication.

Use with Claude AI (MCP)

This actor is available as a tool in Claude AI through the Model Context Protocol (MCP). Add it to Claude Desktop, Cursor, Windsurf, or any MCP-compatible client.

Setup for Claude Code

$claude mcp add --transport http apify "https://mcp.apify.com?tools=automation-lab/github-scraper"

Setup for Claude Desktop, Cursor, or VS Code

Add this to your MCP config file:

{
    "mcpServers": {
        "apify": {
            "url": "https://mcp.apify.com?tools=automation-lab/github-scraper"
        }
    }
}

Example prompts

• "What are the trending GitHub repositories today? Show me the top 25 with their star counts"
• "Search GitHub for the most starred machine learning repositories and summarize what each one does"
• "Get the GitHub profile and public repo stats for these developers: torvalds, gvanrossum, dhh"

Learn more in the Apify MCP documentation.

The scraper returns an error about rate limiting. GitHub's public API allows 60 requests/hour per IP without authentication. The Actor respects Retry-After and X-RateLimit-Reset for bounded retries, then fails actionably if the limit remains exhausted. Add a secret githubToken, reduce maxResults, or split very large jobs. If failure occurs after some rows were emitted, those partial rows remain in the dataset and the run still reports failure.

Profile data shows null for some fields like bio or company. Not all GitHub users fill in their profile details. The scraper returns null for fields the user hasn't set. This is expected behavior.

Related actors

• GitHub Trending Scraper — Trending repositories from GitHub with star velocity
• Hacker News Scraper — Stories from Hacker News front page, newest, Ask HN, and more
• Homebrew Scraper — Homebrew formulas and casks with install counts
• Stack Overflow Scraper — Questions, answers, and tags from Stack Overflow
• npm Scraper — Package metadata from the npm registry
• PyPI Scraper — Python package data from PyPI
• Crates Scraper — Rust crate metadata from crates.io
• Hash Generator — Generate MD5, SHA-1, SHA-256, and SHA-512 hashes