# GitHub Repository Intelligence (`crawlerbros/github-repo-intelligence`) Actor Fetch rich metadata (stars, forks, README, languages, topics, license) from GitHub repositories. Search by query or provide direct URLs. Optional GitHub token for 80x higher rate limit. - **URL**: https://apify.com/crawlerbros/github-repo-intelligence.md - **Developed by:** [Crawler Bros](https://apify.com/crawlerbros) (community) - **Categories:** Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $1.00 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## GitHub Repository Intelligence Pull rich, structured metadata about any public GitHub repository — stars, forks, languages, topics, license, README, and full timestamps — either by search query or from a list of repository URLs. ### What this actor does GitHub's REST API exposes a deep well of per-repository metadata, but stitching it together across many repos (metadata + README + language breakdown + topics) is awkward to do by hand. This actor does that stitching for you and returns one clean, denormalised record per repository, ready for a dataset, a CSV export, or a downstream analytics job. It has two modes. In **search mode** you hand it any GitHub search query — `stars:>10000 language:python`, `topic:llm`, `user:apify`, or whatever qualifier expression you like — and it returns up to 1000 matching repos, each fully enriched. In **direct mode** you hand it a list of `https://github.com/owner/repo` URLs and it fetches them in parallel. For each repository the actor returns full metadata (stars, forks, open issues, watchers, size, homepage, default branch, timestamps, archived / template / disabled flags), the README text (base64-decoded), the list of topics, a byte-level language breakdown, and a normalised license block with SPDX identifier. ### Key features - **Two modes** — `search` (any GitHub search query, up to 1000 results) or `direct` (explicit repository URLs) - **Rich per-repo metadata** — identity, owner, stars, forks, open issues, watchers, size, homepage, default branch, timestamps, and status flags - **README** — fetched, decoded, and included inline (truncated at 500 KB) - **Topics** — the tags the repository owner has assigned - **Language breakdown** — byte counts per language - **License block** — SPDX id, full name, and URL - **Optional `githubToken`** — lifts the rate limit from 60 to 5000 requests per hour (roughly 80x), letting you scale up to thousands of repositories per run - **Resilient** — automatic retry on transient errors, rate-limit detection, per-repo error sentinels so one bad repo doesn't kill the batch - **Zero-setup** — no browser, no cookies, no proxy required - Zero-null output — empty fields are omitted from each record ### Input | Field | Type | Default | Description | | --- | --- | --- | --- | | `mode` | enum | `search` | `search` to run a GitHub search query, or `direct` to fetch a list of URLs. | | `searchQuery` | string | — | GitHub search query (used when `mode=search`). Examples: `stars:>10000 language:python`, `topic:llm`, `user:apify`. | | `repositoryUrls` | string[] | — | List of repository URLs (used when `mode=direct`). Example: `https://github.com/apify/crawlee`. | | `sortBy` | enum | `stars` | Search result ordering: `stars`, `forks`, `updated`, or `help-wanted-issues`. | | `maxResults` | integer | `100` | Cap on the number of repositories returned (1-1000). | | `includeReadme` | boolean | `true` | Fetch and include each repository's README. | | `includeTopics` | boolean | `true` | Include the repository's topic tags. | | `includeLanguages` | boolean | `true` | Include the byte-level language breakdown. | | `githubToken` | string (secret) | — | Optional personal access token. Highly recommended for runs that touch more than ~20 repositories. | **Example input — search mode** ```json { "mode": "search", "searchQuery": "topic:llm stars:>500 language:python", "sortBy": "stars", "maxResults": 200, "includeReadme": true, "includeTopics": true, "includeLanguages": true, "githubToken": "ghp_xxx" } ```` **Example input — direct mode** ```json { "mode": "direct", "repositoryUrls": [ "https://github.com/apify/crawlee", "https://github.com/apify/actor-templates" ] } ``` ### Output One record per repository. Fields without data are omitted. ```json { "id": 169257834, "name": "crawlee", "fullName": "apify/crawlee", "owner": { "login": "apify", "id": 24586296, "avatarUrl": "https://avatars.githubusercontent.com/u/24586296?v=4", "htmlUrl": "https://github.com/apify", "type": "Organization" }, "description": "Crawlee — a web scraping and browser automation library...", "htmlUrl": "https://github.com/apify/crawlee", "homepage": "https://crawlee.dev", "primaryLanguage": "TypeScript", "stars": 15000, "forks": 1000, "watchers": 120, "openIssues": 80, "size": 25000, "topics": ["scraping", "crawler", "automation"], "license": { "spdxId": "Apache-2.0", "name": "Apache License 2.0", "url": "https://api.github.com/licenses/apache-2.0" }, "createdAt": "2019-02-05T12:00:00Z", "updatedAt": "2024-05-01T00:00:00Z", "pushedAt": "2024-05-02T00:00:00Z", "isFork": false, "isArchived": false, "isDisabled": false, "isTemplate": false, "defaultBranch": "master", "readme": "## Crawlee\n...", "languages": {"TypeScript": 987654, "JavaScript": 4321}, "scrapedAt": "2026-04-24T12:00:00+00:00" } ``` **Field descriptions** - **Identity** — `id`, `name`, `fullName`, `htmlUrl`, `owner` (login / id / avatar / URL / type) - **Description** — `description`, `homepage`, `primaryLanguage` - **Engagement** — `stars`, `forks`, `watchers`, `openIssues`, `size` (in KB) - **Tags and license** — `topics` array, `license` block with SPDX identifier - **Timestamps** — `createdAt`, `updatedAt`, `pushedAt` - **Status flags** — `isFork`, `isArchived`, `isDisabled`, `isTemplate` - **Branch** — `defaultBranch` - **Content** — `readme` (decoded, truncated to 500 KB), `languages` (bytes per language) - **`scrapedAt`** — ISO-8601 timestamp of this run **Error record** — emitted per repository when the fetch fails: ```json { "type": "github_repo_intelligence_error", "reason": "rate_limit", "message": "GitHub API rate limit exceeded. Supply `githubToken` to lift the limit.", "repoIdentifier": "apify/crawlee", "scrapedAt": "2026-04-24T12:00:00+00:00" } ``` Reason codes: `rate_limit`, `not_found`, `invalid_url`, `search_failed`, `no_results`, `fetch_failed`. ### Use cases - **Ecosystem mapping** — enumerate every repo tagged with `topic:llm` or `topic:web3` to build a competitive map - **Leaderboards and dashboards** — rank a set of repos by stars, forks, or recent activity for internal dashboards - **Due diligence** — gather license, archived status, push cadence, and README for a list of dependencies - **Market research** — study language mix, topic distribution, and growth across a population of related projects - **Downstream code analysis** — seed an ML / static-analysis pipeline with enriched repo metadata plus READMEs ### FAQ **Do I need a GitHub token?** For very small runs (under ~20 repositories an hour, total) you can run without one. For anything larger, supply a personal access token — the token lifts your hourly request budget from 60 to 5000 (roughly 80x). That's because GitHub enforces a low anonymous-request cap (60 requests per IP per hour) and a much higher token-based cap (5000 requests per token per hour). Create a classic token at https://github.com/settings/tokens with the `public_repo` scope. **Why does each repository use multiple API calls?** Fetching full metadata, the README, and the language breakdown is three API calls per repo. If you disable `includeReadme`, `includeTopics`, and `includeLanguages` you drop back to one call per repo — useful for big shallow scans. **Do I need a proxy?** No. GitHub's API isn't rate-limited by IP; it's limited by the anonymous/token budget described above. Apify datacenter IPs are fine. **Can I scrape private repositories?** Yes, as long as the `githubToken` you supply has access to them. A classic token with `repo` scope covers all private repos your account can read. **Why are some fields missing in my output?** The actor omits empty fields to keep records compact and meaningful — a repo without a homepage simply won't have a `homepage` key, instead of reporting `null`. **How large can a README get?** READMEs are truncated at 500 KB of decoded UTF-8 with a `...[truncated]` marker. This keeps dataset rows well-behaved without losing the vast majority of actual README content. **Will search return more than 1000 results?** No — that's GitHub's hard cap on search. For a bigger universe, slice the space with additional qualifiers (e.g. `stars:1000..5000`, then `stars:500..999`, then `stars:100..499`) and run once per slice. **What happens when a single repo fails?** The actor emits a per-repo error record and continues with the rest of the batch. One deleted or private repository never kills the whole run. ### Known limitations - **1000-result search cap.** GitHub itself caps any search query at 1000 results. For larger spaces, slice the query into ranges. - **Anonymous rate limit is tight.** Without a `githubToken` you get about 60 API calls per hour. Each enriched repo is up to 3 calls, so runs over ~20 repositories need a token. - **README truncation at 500 KB.** Very long READMEs (rare) are cut at 500 KB with a marker. - **No commit, issue, or PR data.** This actor focuses on repository-level metadata; commit history, issues, and pull requests are out of scope. - **Private repos need explicit access.** The `githubToken` must have `repo` scope for private repositories; public-only tokens return a `not_found` error record for private URLs. # Actor input Schema ## `mode` (type: `string`): Choose 'search' to query GitHub's search API with a keyword / qualifier expression, or 'direct' to fetch a known list of repository URLs. ## `searchQuery` (type: `string`): GitHub search query (used when mode = search). Supports qualifiers like 'stars:>1000', 'language:python', 'topic:llm', 'user:apify'. See GitHub's search syntax docs for the full grammar. ## `repositoryUrls` (type: `array`): List of full GitHub repository URLs to scrape (used when mode = direct). Accepts https://github.com/owner/repo, ending with .git, trailing slash, or the git@github.com:owner/repo.git SSH form. ## `sortBy` (type: `string`): Sort order for search results. Ignored in direct mode. ## `maxResults` (type: `integer`): Hard cap on the number of repositories to return. GitHub's search API itself is capped at 1000 results; direct mode is only limited by this value. ## `includeReadme` (type: `boolean`): Fetch and include the decoded README text for each repository (capped at 500 KB). ## `includeTopics` (type: `boolean`): Include the repository's topic tags (e.g. 'machine-learning', 'web-scraping'). ## `includeLanguages` (type: `boolean`): Include the per-language byte counts (e.g. {"Python": 123456, "JavaScript": 2345}). ## `githubToken` (type: `string`): Optional GitHub personal access token. Mark this field as Secret when pasting. Increases the API rate limit from 60 requests/hour (unauthenticated) to 5000 requests/hour. A classic token with 'public\_repo' scope is enough for public repositories. ## Actor input object example ```json { "mode": "search", "searchQuery": "language:rust stars:>5000", "repositoryUrls": [ "https://github.com/apify/crawlee" ], "sortBy": "stars", "maxResults": 30, "includeReadme": true, "includeTopics": true, "includeLanguages": true } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "search", "searchQuery": "stars:>10000 language:python", "repositoryUrls": [ "https://github.com/apify/crawlee" ], "maxResults": 30 }; // Run the Actor and wait for it to finish const run = await client.actor("crawlerbros/github-repo-intelligence").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "search", "searchQuery": "stars:>10000 language:python", "repositoryUrls": ["https://github.com/apify/crawlee"], "maxResults": 30, } # Run the Actor and wait for it to finish run = client.actor("crawlerbros/github-repo-intelligence").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "search", "searchQuery": "stars:>10000 language:python", "repositoryUrls": [ "https://github.com/apify/crawlee" ], "maxResults": 30 }' | apify call crawlerbros/github-repo-intelligence --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=crawlerbros/github-repo-intelligence", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "GitHub Repository Intelligence", "description": "Fetch rich metadata (stars, forks, README, languages, topics, license) from GitHub repositories. Search by query or provide direct URLs. Optional GitHub token for 80x higher rate limit.", "version": "0.1", "x-build-id": "ytT3bQ5htN3i2SJjl" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/crawlerbros~github-repo-intelligence/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-crawlerbros-github-repo-intelligence", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/crawlerbros~github-repo-intelligence/runs": { "post": { "operationId": "runs-sync-crawlerbros-github-repo-intelligence", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/crawlerbros~github-repo-intelligence/run-sync": { "post": { "operationId": "run-sync-crawlerbros-github-repo-intelligence", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "mode": { "title": "Mode", "enum": [ "search", "direct" ], "type": "string", "description": "Choose 'search' to query GitHub's search API with a keyword / qualifier expression, or 'direct' to fetch a known list of repository URLs.", "default": "search" }, "searchQuery": { "title": "Search query", "type": "string", "description": "GitHub search query (used when mode = search). Supports qualifiers like 'stars:>1000', 'language:python', 'topic:llm', 'user:apify'. See GitHub's search syntax docs for the full grammar.", "default": "stars:>10000 language:python" }, "repositoryUrls": { "title": "Repository URLs", "type": "array", "description": "List of full GitHub repository URLs to scrape (used when mode = direct). Accepts https://github.com/owner/repo, ending with .git, trailing slash, or the git@github.com:owner/repo.git SSH form.", "items": { "type": "string" } }, "sortBy": { "title": "Sort results by", "enum": [ "stars", "forks", "updated", "help-wanted-issues" ], "type": "string", "description": "Sort order for search results. Ignored in direct mode.", "default": "stars" }, "maxResults": { "title": "Maximum results", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Hard cap on the number of repositories to return. GitHub's search API itself is capped at 1000 results; direct mode is only limited by this value.", "default": 30 }, "includeReadme": { "title": "Include README", "type": "boolean", "description": "Fetch and include the decoded README text for each repository (capped at 500 KB).", "default": true }, "includeTopics": { "title": "Include topics", "type": "boolean", "description": "Include the repository's topic tags (e.g. 'machine-learning', 'web-scraping').", "default": true }, "includeLanguages": { "title": "Include language breakdown", "type": "boolean", "description": "Include the per-language byte counts (e.g. {\"Python\": 123456, \"JavaScript\": 2345}).", "default": true }, "githubToken": { "title": "GitHub personal access token (optional)", "type": "string", "description": "Optional GitHub personal access token. Mark this field as Secret when pasting. Increases the API rate limit from 60 requests/hour (unauthenticated) to 5000 requests/hour. A classic token with 'public_repo' scope is enough for public repositories." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```