GitHub Repository Scraper

Scrape detailed information from any public GitHub repository including stats, README content, languages, contributors, and releases.

Use with AI agents (MCP)

This actor works as a tool for AI agents through Apify's MCP server at https://mcp.apify.com, so Claude, ChatGPT, Cursor and other MCP clients can inspect GitHub repositories directly. For agents, set "compactOutput": true and pass only a few repositories per call to get short, token-friendly results.

Quick setup (sign in with your Apify account when asked):

• Claude (claude.ai or Claude Desktop): Settings → Connectors → Add custom connector, and paste https://mcp.apify.com?tools=artificially/github-repo-scraper
• Claude Code or Cursor via the Apify CLI (latest version, apify upgrade): apify mcp install claude-code --tools artificially/github-repo-scraper (use cursor instead of claude-code for Cursor)
• Any MCP client (Cursor, VS Code, Windsurf):

{
  "mcpServers": {
    "apify": { "url": "https://mcp.apify.com?tools=artificially/github-repo-scraper" }
  }
}

Try asking:

• "Compare stars, contributors and release activity of react, vue and svelte on GitHub."
• "How healthy is the vercel/next.js repository: issue close rate, recent commits, top contributors?"

Features

• Repository Stats: Stars, forks, watchers, open issues, size
• Repository Info: Description, license, topics, creation/update dates
• README Content: Full README text (optional)
• Language Breakdown: Bytes of code per language (optional)
• Contributors: List of top contributors with commit counts (optional)
• Releases: Recent releases with download links (optional)
• Commit Activity: Weekly commits over the last year, trend and frequency (default on)
• Issue Statistics: Real open/closed issue totals, average close time, top labels, good first issues (default on)
• Pull Requests: Recent PRs with real open/closed totals and merge stats (optional)
• Dependencies: Parsed from package.json, requirements.txt, Cargo.toml or go.mod (optional)
• Community Profile: GitHub community health score (default on)
• Scores: Health, activity and popularity scores (0-100)

You only pay for valid results: repositories that are not found, invalid inputs, duplicates and repositories that could not be fully fetched are skipped and not charged.

Use Cases

• Track popularity metrics across multiple repositories
• Monitor competitor open-source projects
• Build datasets for research on GitHub trends
• Aggregate documentation from multiple repos
• Track release schedules and changelogs

Input

Field	Type	Required	Default	Description
repositories	array	Yes	-	List of repositories (e.g., ["facebook/react", "https://github.com/vercel/next.js"])
compactOutput	boolean	No	false	Slim items with only key fields (see below); recommended for AI agents
includeReadme	boolean	No	true	Include full README content
includeLanguages	boolean	No	true	Include language breakdown
includeContributors	boolean	No	false	Include contributor list
maxContributors	number	No	10	Maximum contributors to fetch
includeReleases	boolean	No	false	Include release history
maxReleases	number	No	5	Maximum releases to fetch
includeCommitActivity	boolean	No	true	Include yearly commit activity
includeIssueStats	boolean	No	true	Include issue statistics
includePullRequests	boolean	No	false	Include recent pull requests and PR stats
maxPullRequests	number	No	10	Maximum pull requests to list
includeDependencies	boolean	No	false	Include parsed dependencies
includeCommunityProfile	boolean	No	true	Include GitHub community health profile
githubToken	string	No	-	Optional GitHub personal access token (no scopes needed). Raises the API limit from 60 to 5,000 requests per hour
proxyConfiguration	object	No	Apify Proxy	Proxy settings; each request uses a fresh IP

Example Input

{
    "repositories": [
        "facebook/react",
        "vercel/next.js",
        "https://github.com/microsoft/vscode"
    ],
    "includeReadme": true,
    "includeLanguages": true,
    "includeContributors": true,
    "maxContributors": 5
}

Output

Each repository produces one result:

{
    "repoId": 10270250,
    "name": "react",
    "fullName": "react/react",
    "owner": "react",
    "ownerType": "Organization",
    "ownerAvatarUrl": "https://avatars.githubusercontent.com/u/102812?v=4",
    "description": "The library for web and native user interfaces.",
    "url": "https://github.com/react/react",
    "homepage": "https://react.dev",
    "stars": 250660,
    "watchers": 250660,
    "subscribers": 6598,
    "forks": 51390,
    "openIssues": 1373,
    "language": "JavaScript",
    "topics": ["declarative", "frontend", "javascript", "library", "react", "ui"],
    "license": "MIT",
    "isPrivate": false,
    "isFork": false,
    "isArchived": false,
    "defaultBranch": "main",
    "createdAt": "2013-05-24T16:15:54Z",
    "updatedAt": "2026-09-23T08:52:58Z",
    "pushedAt": "2026-09-22T17:36:10Z",
    "size": 1098633,
    "languages": { "JavaScript": 5693822, "Rust": 2991726, "TypeScript": 2577964 },
    "languagePercentages": { "JavaScript": 49.5, "Rust": 26, "TypeScript": 22.4 },
    "readme": "# [React](https://react.dev/) ...",
    "contributors": [{ "username": "sebmarkbage", "contributions": 1939, "profileUrl": "https://github.com/sebmarkbage", "type": "User" }],
    "releases": [{ "tagName": "v19.3.0", "name": "19.3.0", "publishedAt": "2026-09-09T...", "downloadCount": 0 }],
    "commitActivity": { "totalCommitsLastYear": 790, "weeklyAverage": 15.2, "commitFrequency": "active", "trend": "stable", "lastCommitDate": "2026-09-22T17:26:59Z" },
    "issueStats": { "totalOpen": 860, "totalClosed": 13857, "closedRatio": 94, "avgCloseTimeDays": 2, "goodFirstIssueCount": 0, "sampleSize": 41 },
    "pullRequestStats": { "totalOpen": 513, "totalClosed": 19796, "totalMerged": 37, "mergeRate": 74, "avgMergeTimeDays": 2, "mergeStatsSampleSize": 50 },
    "communityProfile": { "healthPercentage": 100, "hasContributing": true, "hasCodeOfConduct": true },
    "healthScore": 100,
    "activityScore": 75,
    "popularityScore": 100,
    "scrapedAt": "2026-09-23T08:57:18.550Z"
}

Notes on fields:

• watchers mirrors GitHub's watchers_count, which equals the star count. subscribers is the real "Watch" count shown on GitHub.
• issueStats.totalOpen / totalClosed and pullRequestStats.totalOpen / totalClosed are real totals for the repository (issues exclude pull requests).
• avgCloseTimeDays, labels, goodFirstIssueCount are computed from recent issues (sampleSize). totalMerged, mergeRate and avgMergeTimeDays are computed from the most recent closed pull requests (mergeStatsSampleSize, up to 50).
• If a repository was renamed or transferred, fullName and url show its current location. Inputs that resolve to the same repository are saved once.

The key-value store also contains:

• SUMMARY: counts of saved, duplicate and failed repositories and whether the GitHub rate limit was hit.
• FAILED_REPOSITORIES: each skipped input with a reason (invalid_input, not_found, rate_limited, network, server_error, stats_pending, invalid_data).

Compact output

With "compactOutput": true each item keeps only the key decision fields: repoId, fullName, owner, description, url, homepage, stars, forks, subscribers, openIssues, language, languagePercentages, topics, license, isFork, isArchived, createdAt, pushedAt, contributorsCount, topContributor, releasesCount, latestRelease, latestReleaseDate, commitsLastYear, lastCommitDate, commitFrequency, commitTrend, issuesClosedRatio, avgIssueCloseTimeDays, goodFirstIssueCount, pullRequestMergeRate, dependenciesCount, communityHealthPercentage, healthScore, activityScore, popularityScore, scrapedAt. README text and the contributor, release, issue, pull request and dependency lists are left out; counts refer to the fetched lists and are null when that option is off.

Cost

This actor uses pay-per-event pricing; see the Pricing tab for current prices. You only pay for valid results.

Rate Limits

GitHub's unauthenticated API allows 60 requests per hour per IP, and a repository with the default options needs about 10 requests. For more than a handful of repositories:

• Add a GitHub personal access token in githubToken (5,000 requests per hour; no scopes are needed for public repositories)
• Or keep Apify Proxy enabled so requests are spread across IPs

When the limit is exhausted the actor stops, saves nothing it could not fetch completely, and lists the skipped repositories in FAILED_REPOSITORIES.

Tips

1. Repository formats accepted:

   • owner/repo (e.g., facebook/react)
   • Full URL (e.g., https://github.com/facebook/react)
   • URL with .git suffix is handled automatically

2. Minimize API calls: Disable the include* options you don't need; each one costs extra GitHub API requests.

3. Large repos: For repositories with many contributors/releases, limit the count to avoid rate limits.

Support

• Built by: Artificially
• Issues: Report bugs or request features via Apify Console

Related actors

• arXiv Papers Scraper - research papers from arXiv with authors, abstracts, categories and PDF links
• Career Site Jobs Scraper - job postings straight from company career sites