# GitHub Scraper API — Repositories, Users & Data Export (`bovi/github-scraper`) Actor Scrape GitHub repositories and user profiles via the official public API. Search by keyword, fetch repos by owner/name, or look up user profiles. No auth required; supply your free GitHub token to raise rate limits to 5000 req/hr. Pay per result. - **URL**: https://apify.com/bovi/github-scraper.md - **Developed by:** [Vitalii Bondarev](https://apify.com/bovi) (community) - **Categories:** Developer tools, Lead generation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $3.90 / 1,000 repositories This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## GitHub Repository & User Scraper API Used by developer-focused recruiters building outreach lists, B2B SaaS teams identifying active open-source contributors in their stack, and AI agents answering "who are the top contributors to this Python project?" Scrape GitHub repositories and user profiles via the **official GitHub REST API**. Search repos by keyword, fetch specific repos by owner/name, or retrieve user and organization profiles. Pay per result. No proxy needed — GitHub's public API requires no authentication for read-only data. **$2.00/1K repos** · **$3.00/1K user/org profiles** (Pay Per Event). First 10 results free. No proxy needed. Official GitHub API. 5000 req/hr with free token. Full GitHub search syntax supported. > **Not affiliated with GitHub, Inc. or Microsoft.** GitHub is a trademark of GitHub, Inc. --- ### What You Can Scrape #### Repositories - Full name (owner/repo), description, homepage URL - Stars, forks, watchers, open issues count - Primary language, topic tags, license (SPDX identifier) - Created / last-updated dates (ISO 8601) - Archived and fork flags #### Users & Organizations - Login, display name, bio, company, location - Follower and following counts, public repo count - Account creation date, blog/website URL - Twitter username, public email (when available) --- ### Rate Limits | Mode | Limit | |---|---| | No token (anonymous) | 60 API requests/hr; 10 search requests/min | | With free GitHub token | **5,000 requests/hr** (no scope required) | **For large runs: generate a free token at [github.com/settings/tokens](https://github.com/settings/tokens) (no scopes needed for public data) and supply it via the `githubToken` input field.** The token never leaves your Apify account's encrypted storage. --- ### Modes #### `searchRepos` (default) Searches GitHub repositories by keyword using the [GitHub Search API](https://docs.github.com/en/rest/search/search#search-repositories). Supports full GitHub search syntax: - `"machine learning language:python stars:>500"` - `"react hooks"` - `"apify scraper"` Returns repos sorted by stars (default), forks, update date, or help-wanted issues. #### `repoFullNames` Fetch one or more specific repos by exact `owner/repo` identifier. Returns the full repo object including topics, license, and all metrics. Useful when you have a curated list. #### `usernames` Look up GitHub user or organization profiles by login. Returns follower count, public repo count, bio, company, location, and account creation date. --- ### Output Schema #### Repo record | Field | Type | Description | |---|---|---| | `type` | string | Always `"repo"` | | `full_name` | string | `"owner/repo"` | | `owner` | string | Owner login | | `name` | string | Repo name | | `description` | string | Repo description | | `homepage` | string | Project homepage URL | | `html_url` | string | GitHub page URL | | `language` | string | Primary programming language | | `topics` | list[str] | Topic tags | | `license` | string | SPDX license ID (e.g. `"MIT"`) | | `stars` | integer | Star count | | `forks` | integer | Fork count | | `watchers` | integer | Watcher count | | `open_issues` | integer | Open issues count | | `created_at` | string | ISO 8601 creation date | | `updated_at` | string | ISO 8601 last-updated date | | `is_archived` | boolean | Whether repo is archived | | `is_fork` | boolean | Whether repo is a fork | | `query` | string | Search query that produced this result | | `scraped_at` | string | ISO 8601 scrape timestamp | | `parse_confidence` | float | Data quality score (1.0 = all fields present) | | `warnings` | list[str] | Missing-field codes for low-confidence records | #### User record | Field | Type | Description | |---|---|---| | `type` | string | `"user"` or `"organization"` | | `login` | string | GitHub login | | `name` | string | Display name | | `bio` | string | Bio text | | `company` | string | Employer (@ prefix stripped) | | `location` | string | Location string | | `blog` | string | Blog/homepage URL | | `email` | string | Public email (often null) | | `twitter_username` | string | Twitter handle | | `followers` | integer | Follower count | | `following` | integer | Following count | | `public_repos` | integer | Public repo count | | `public_gists` | integer | Public gist count | | `created_at` | string | ISO 8601 account creation date | | `updated_at` | string | ISO 8601 profile last-updated date | | `html_url` | string | GitHub profile URL | | `query` | string | Username queried | | `scraped_at` | string | ISO 8601 scrape timestamp | | `parse_confidence` | float | Data quality score | | `warnings` | list[str] | Missing-field codes | --- ### Example Use Cases - **Lead generation**: Find active developers in specific languages or topics - **Tech stack research**: Discover most-starred repos per language - **Competitive intelligence**: Track competitor repos (stars, forks, activity) - **Developer outreach**: Build lists of contributors to specific projects - **Open source analytics**: Monitor ecosystem health by topic tag --- ### vs. Competitors | Feature | This Actor | drobnikj/github-scraper | HTML web scrapers | |---|---|---|---| | Data source | Official GitHub REST API | Older GitHub API | HTML scraping | | Proxy or anti-bot workaround needed | No | No | Sometimes | | Rate limit handling | Yes (429 backoff, token boost) | Basic | Often breaks | | User + org profiles | Yes (separate `gh-user` event) | Limited | Inconsistent | | parse_confidence | Yes | No | No | | Full GitHub search syntax | Yes | Partial | No | | Cost | **$2/1K repos · $3/1K profiles** | ~$1-2/1K | $1-5/1K + proxy | ### Use with AI agents (MCP) An agent calls this tool to **look up GitHub repos and developer profiles mid-conversation** — e.g. "Find top Python ML repos with 1000+ stars", "Get the profile for linus-torvalds", or "What repos does the Apify org maintain?" Point your MCP client at this tool: ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=bovi/github-scraper", "--header", "Authorization: Bearer " ] } } } ```` Minimal agent input (repo search): ```json { "mode": "searchRepos", "searchQueries": ["apify scraper language:python stars:>100"], "maxItems": 20 } ``` ### Pricing **Pay-per-result (PPE):** | Event | Rate | Trigger | |---|---|---| | `gh-item` | **$2.00/1K** | Repository records (`searchRepos` or `repoFullNames` mode) | | `gh-user` | **$3.00/1K** | User and organization profile records (`usernames` mode) | User/org profiles are priced higher because they contain contact signals (public email, Twitter handle, company) useful for developer lead-gen. You pay only for data you receive — no charge for rate-limit retries or failed lookups. **Worked examples:** - 500 repo search results = **$1.00** - 1,000 repo search results = **$2.00** - 200 user/org profiles = **$0.60** - 500 repos + 100 user profiles = **$1.30** ($1.00 repos + $0.30 profiles) GitHub's public API is free — compute is billed to your Apify account based on actual run time. ### FAQ **Do I need a GitHub token?** A token is optional but strongly recommended for runs over ~50 items. Without one, the rate limit is 60 requests/hr; with a free token (no scopes needed) it's 5,000 requests/hr. Generate one at github.com/settings/tokens. **Can I search private repositories?** No. The actor uses GitHub's public REST API — only public repos and public profile data are accessible. **What if my search returns empty?** GitHub Search API caps results at 1,000 per query. For common keywords, narrow your search with qualifiers (language, stars, topics). Empty results are not charged. **What output formats are available?** JSON, CSV, Excel — download from the Apify dataset, or pipe to n8n / Make / Zapier. Every record includes `html_url` linking directly to GitHub. *** ### Tips - Supply a GitHub token (`githubToken` input) for large runs — without it you'll hit the 60 req/hr limit quickly. - For `searchRepos`, GitHub caps results at 1,000 per query. Use multiple specific queries to cover more ground. - `is_fork: true` repos often have low original activity — filter them out if you need organic projects. - `parse_confidence < 1.0` means some optional fields were missing from the API response — the record is still valid. ### Integrations Built for developer-focused recruiters and B2B SaaS teams identifying active contributors and repo signals in their stack — the JSON/dataset output drops into the tools you already run, no glue code: - **n8n / Make / Zapier** — trigger a run or pipe every new dataset item into 500+ apps (Google Sheets, Airtable, Slack, HubSpot, your database) with no code: [n8n](https://docs.apify.com/platform/integrations/n8n), [Make](https://docs.apify.com/platform/integrations/make), [Zapier](https://docs.apify.com/platform/integrations/zapier). - **Webhooks** — fire your own endpoint the moment a run finishes, to push results straight into your pipeline ([docs](https://docs.apify.com/platform/integrations/webhooks)). - **MCP server** — expose this actor as a tool to Claude, Cursor, or any [MCP client](https://mcp.apify.com) so an AI agent can pull this data mid-conversation ([guide](https://blog.apify.com/how-to-use-mcp/)). - **API & SDKs** — fetch the dataset as JSON, CSV, or Excel through the Apify REST API or the Python / JS SDKs. See all [Apify integrations](https://apify.com/integrations). # Actor input Schema ## `mode` (type: `string`): What to scrape: searchRepos = find repos by keyword (GitHub Search API); repoFullNames = fetch specific repos by owner/repo name; usernames = look up user or org profiles by GitHub login. ## `searchQueries` (type: `array`): Keyword search queries for mode=searchRepos (GitHub Search API). Each query runs separately. Supports full GitHub search syntax: 'react language:javascript', 'machine learning stars:>1000', 'apify scraper language:python'. ## `repoFullNames` (type: `array`): Specific repositories to fetch for mode=repoFullNames. Format: 'owner/repo'. Examples: \['torvalds/linux', 'psf/requests', 'apify/crawlee']. Returns full repo details including stars, topics, license. ## `usernames` (type: `array`): GitHub login names for mode=usernames. Works for individual users and organizations. Examples: \['torvalds', 'gvanrossum', 'apify']. Returns follower count, public repos, bio, company, email, location. ## `maxItems` (type: `integer`): Maximum total records to return across all queries. 0 = no limit (up to GitHub API caps: 1000 per search query). Default 100. ## `githubToken` (type: `string`): Optional personal access token or fine-grained PAT. Without a token: 60 API requests/hr and 10 search requests/min per IP. With a free token (generate at github.com/settings/tokens — no scopes needed for public data): 5000 requests/hr. Strongly recommended for large runs. ## `sort` (type: `string`): Sort order for searchRepos mode. stars: most starred first (default). forks: most forked. updated: most recently updated. help-wanted-issues: most help-wanted issues. ## Actor input object example ```json { "mode": "searchRepos", "searchQueries": [ "apify scraper", "python web scraper stars:>500" ], "repoFullNames": [ "psf/requests", "apify/crawlee" ], "usernames": [ "torvalds", "gvanrossum" ], "maxItems": 100, "sort": "stars" } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "searchRepos", "searchQueries": [ "apify scraper", "python web scraper stars:>500" ], "repoFullNames": [ "psf/requests", "apify/crawlee" ], "usernames": [ "torvalds", "gvanrossum" ], "maxItems": 100, "sort": "stars" }; // Run the Actor and wait for it to finish const run = await client.actor("bovi/github-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "searchRepos", "searchQueries": [ "apify scraper", "python web scraper stars:>500", ], "repoFullNames": [ "psf/requests", "apify/crawlee", ], "usernames": [ "torvalds", "gvanrossum", ], "maxItems": 100, "sort": "stars", } # Run the Actor and wait for it to finish run = client.actor("bovi/github-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "searchRepos", "searchQueries": [ "apify scraper", "python web scraper stars:>500" ], "repoFullNames": [ "psf/requests", "apify/crawlee" ], "usernames": [ "torvalds", "gvanrossum" ], "maxItems": 100, "sort": "stars" }' | apify call bovi/github-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=bovi/github-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "GitHub Scraper API — Repositories, Users & Data Export", "description": "Scrape GitHub repositories and user profiles via the official public API. Search by keyword, fetch repos by owner/name, or look up user profiles. No auth required; supply your free GitHub token to raise rate limits to 5000 req/hr. Pay per result.", "version": "0.1", "x-build-id": "grOYm1qilPcXakmQ0" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/bovi~github-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-bovi-github-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/bovi~github-scraper/runs": { "post": { "operationId": "runs-sync-bovi-github-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/bovi~github-scraper/run-sync": { "post": { "operationId": "run-sync-bovi-github-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "mode": { "title": "Mode", "enum": [ "searchRepos", "repoFullNames", "usernames" ], "type": "string", "description": "What to scrape: searchRepos = find repos by keyword (GitHub Search API); repoFullNames = fetch specific repos by owner/repo name; usernames = look up user or org profiles by GitHub login." }, "searchQueries": { "title": "Search queries", "type": "array", "description": "Keyword search queries for mode=searchRepos (GitHub Search API). Each query runs separately. Supports full GitHub search syntax: 'react language:javascript', 'machine learning stars:>1000', 'apify scraper language:python'.", "items": { "type": "string" } }, "repoFullNames": { "title": "Repo full names", "type": "array", "description": "Specific repositories to fetch for mode=repoFullNames. Format: 'owner/repo'. Examples: ['torvalds/linux', 'psf/requests', 'apify/crawlee']. Returns full repo details including stars, topics, license.", "items": { "type": "string" } }, "usernames": { "title": "Usernames or org names", "type": "array", "description": "GitHub login names for mode=usernames. Works for individual users and organizations. Examples: ['torvalds', 'gvanrossum', 'apify']. Returns follower count, public repos, bio, company, email, location.", "items": { "type": "string" } }, "maxItems": { "title": "Max items total", "minimum": 0, "type": "integer", "description": "Maximum total records to return across all queries. 0 = no limit (up to GitHub API caps: 1000 per search query). Default 100." }, "githubToken": { "title": "GitHub token (optional)", "type": "string", "description": "Optional personal access token or fine-grained PAT. Without a token: 60 API requests/hr and 10 search requests/min per IP. With a free token (generate at github.com/settings/tokens — no scopes needed for public data): 5000 requests/hr. Strongly recommended for large runs." }, "sort": { "title": "Search sort order", "enum": [ "stars", "forks", "updated", "help-wanted-issues" ], "type": "string", "description": "Sort order for searchRepos mode. stars: most starred first (default). forks: most forked. updated: most recently updated. help-wanted-issues: most help-wanted issues." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```