# GitHub Scraper (`goat255/github-scraper`) Actor Scrape GitHub repositories, users, organizations, and repository search results without a login. Pull a repo's full metadata by owner/repo, a user's or org's profile and repositories, or repository search results. Walks pagination up to your chosen limit. - **URL**: https://apify.com/goat255/github-scraper.md - **Developed by:** [Goutam Soni](https://apify.com/goat255) (community) - **Categories:** Developer tools, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: 5.00 out of 5 stars ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## GitHub Scraper Extract public GitHub data without a login or API key. Scrape any repository's full metadata, any user's or organization's profile and repositories, and repository search results, with pagination walked automatically up to the limit you set. ### What it does - **Repository scraper.** Pass any `owner/repo` (or a repo link) and get stars, forks, watchers, open issues, primary language, topics, license, repo size, default branch, and created / updated / last-push timestamps. - **User and organization scraper.** Pass a login and get the profile (name, bio, company, location, blog, email, followers, following, public repo and gist counts) and, optionally, every public repository the account owns. - **Repository search.** Run any search query with the standard GitHub search syntax (language, stars, topic, etc.) and collect the matching repositories. - **No login, no API key required.** Runs out of the box. An optional access token can be supplied to raise the request budget on very large runs, but it is never required. - **Automatic pagination.** Multi-page user repository lists and search results are walked page by page until your limit is reached or the source is exhausted. - **Clean, normalized output.** Every row is a flat object with a stable schema and a `type` field (`repo` or `user`), ready for spreadsheets, databases, or downstream automation. ### Use cases - **Developer lead generation.** Build lists of maintainers and organizations in a niche (for example, repositories matching `language:rust stars:>1000`) and enrich them with profile and contact fields. - **Open-source market research.** Track stars, forks, and activity timestamps across a set of competing projects to see which tools are gaining traction. - **Tech recruiting and sourcing.** Pull contributor organizations and their public repositories to find active engineers by language and topic. - **Portfolio and ecosystem monitoring.** Re-run on a schedule to watch how a list of repositories or accounts changes over time. - **Dataset building.** Assemble a structured dataset of repositories by topic or search query for analysis, dashboards, or training data. ### Input | Field | Type | Description | |---|---|---| | `repos` | array | Repositories to fetch full metadata for, as `owner/repo` or a repo link. | | `users` | array | User or organization logins to fetch, with or without `@`, or a profile link. | | `searchQueries` | array | Repository search queries using the standard GitHub search syntax. | | `maxItemsPerSource` | integer | Cap per user (repositories) or per search query. Default 100. Search is capped at 1000 by the source. | | `includeUserRepos` | boolean | When on, each user or organization also emits its public repositories after the profile row. Default true. | | `searchSort` | string | Sort search results: best match (default), `stars`, `forks`, or `updated`. | | `githubToken` | string | Optional access token to raise the request budget for very large runs. Not required. | | `concurrency` | integer | How many sources to process in parallel. Default 5. | | `proxyConfig` | object | Optional proxy configuration. Not required for public data. | #### Example input ```json { "repos": ["apify/crawlee", "example_user/example-repo"], "users": ["example_user"], "searchQueries": ["web scraping language:python stars:>1000"], "maxItemsPerSource": 200, "searchSort": "stars" } ```` ### Output Each result is pushed as one row to the dataset. A repository row: ```json { "type": "repo", "id": 66670819, "fullName": "example_user/example-repo", "name": "example-repo", "owner": "example_user", "ownerType": "Organization", "url": "https://github.com/example_user/example-repo", "homepage": "https://example.com", "stars": 23801, "forks": 1433, "watchers": 131, "openIssues": 172, "sizeKb": 166212, "description": "An example repository.", "language": "TypeScript", "topics": ["example", "scraper"], "license": "Apache-2.0", "defaultBranch": "main", "avatarUrl": "https://avatars.githubusercontent.com/u/000000?v=4", "isFork": false, "isArchived": false, "createdAt": "2016-08-26T18:35:03Z", "updatedAt": "2026-06-17T16:38:29Z", "pushedAt": "2026-06-17T14:49:49Z", "scrapedAt": "2026-06-18T08:15:00Z" } ``` A user or organization row (`type: "user"`) carries `login`, `name`, `accountType`, `url`, `followers`, `following`, `publicRepos`, `publicGists`, `bio`, `company`, `location`, `blog`, `email`, `twitter`, `avatarUrl`, `createdAt`, and `updatedAt`. **Key fields.** Columns are ordered by importance: identity first (`type`, `id`, `fullName`/`login`, `name`, `owner`, `url`), then metrics (`stars`, `forks`, `watchers`, `openIssues`, `followers`), then content (`description`, `language`, `topics`, `license`), then media and timestamps. `watchers` is the true count of accounts watching a repository and is filled in for single-repository lookups. Profile fields such as `company`, `email`, and `twitter` are present only when the account has set them publicly; they are `null` otherwise. ### FAQ **Is it free? How is it priced?** The actor is pay-per-result, so you are billed per row returned. There is no separate per-run start fee. Check the current rate on the actor's pricing tab before you run. **Do I need a GitHub login or API key?** No. It runs without any account, password, or API key. You can optionally provide your own access token to raise the request budget on very large runs, but it is never required. **How many results can I get?** Repository and user-profile lookups return one row each. A user's or organization's repositories and each search query return up to `maxItemsPerSource` rows (search is capped at 1000 per query by the source). Pagination is walked automatically across multiple pages to reach your target. **How fast is it?** Sources are processed in parallel (set by `concurrency`), and each result is streamed to the dataset as it is collected, so partial output is available while a large run is still in progress. **Can I scrape a private repository?** No. Only public repositories, users, and organizations are accessible. Supplying an access token raises the request budget but does not unlock private data. **What input formats are accepted for repos and users?** Repositories accept `owner/repo` or a full repo link. Users and organizations accept a bare login, an `@login`, or a full profile link. Inputs are cleaned automatically. # Actor input Schema ## `repos` (type: `array`): Repositories to fetch full metadata for. Use owner/repo or a full repo link. Example: apify/crawlee, https://github.com/example\_user/example-repo. ## `users` (type: `array`): User or organization logins to fetch profiles for, optionally with their repositories. With or without the @ prefix, or a full profile link. Example: example\_user, @example\_user, https://github.com/example\_user. ## `searchQueries` (type: `array`): Repository search queries. Supports the standard search syntax, for example: web scraping language:python stars:>1000. Each query returns up to 1000 results. ## `maxItemsPerSource` (type: `integer`): Cap on items returned per user (repositories) or per search query. Pagination is walked across multiple pages until this is reached or the source is exhausted. Search is capped at 1000 by the source. ## `includeUserRepos` (type: `boolean`): When on, each user or organization also emits its public repositories (up to the max per source) after its profile row. ## `searchSort` (type: `string`): Sort order for search results. Best match (relevance), or by stars, forks, or recent updates. ## `githubToken` (type: `string`): Optional personal access token. Not required. When provided it raises the request budget for large runs. A read-only token with public access is sufficient. ## `concurrency` (type: `integer`): How many sources to process in parallel. ## `proxyConfig` (type: `object`): Optional proxy. The public read endpoints do not require a proxy, but you may route through one if you wish. ## Actor input object example ```json { "repos": [ "apify/crawlee" ], "users": [], "searchQueries": [], "maxItemsPerSource": 100, "includeUserRepos": true, "searchSort": "", "concurrency": 5, "proxyConfig": { "useApifyProxy": false } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "repos": [ "apify/crawlee" ], "users": [], "searchQueries": [] }; // Run the Actor and wait for it to finish const run = await client.actor("goat255/github-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "repos": ["apify/crawlee"], "users": [], "searchQueries": [], } # Run the Actor and wait for it to finish run = client.actor("goat255/github-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "repos": [ "apify/crawlee" ], "users": [], "searchQueries": [] }' | apify call goat255/github-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=goat255/github-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "GitHub Scraper", "description": "Scrape GitHub repositories, users, organizations, and repository search results without a login. Pull a repo's full metadata by owner/repo, a user's or org's profile and repositories, or repository search results. Walks pagination up to your chosen limit.", "version": "0.1", "x-build-id": "noNDrQWeZ3NXB9rCB" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/goat255~github-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-goat255-github-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/goat255~github-scraper/runs": { "post": { "operationId": "runs-sync-goat255-github-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/goat255~github-scraper/run-sync": { "post": { "operationId": "run-sync-goat255-github-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "repos": { "title": "Repositories (repo mode)", "type": "array", "description": "Repositories to fetch full metadata for. Use owner/repo or a full repo link. Example: apify/crawlee, https://github.com/example_user/example-repo.", "default": [ "apify/crawlee" ], "items": { "type": "string" } }, "users": { "title": "Users or organizations (user mode)", "type": "array", "description": "User or organization logins to fetch profiles for, optionally with their repositories. With or without the @ prefix, or a full profile link. Example: example_user, @example_user, https://github.com/example_user.", "items": { "type": "string" } }, "searchQueries": { "title": "Search queries (search mode)", "type": "array", "description": "Repository search queries. Supports the standard search syntax, for example: web scraping language:python stars:>1000. Each query returns up to 1000 results.", "items": { "type": "string" } }, "maxItemsPerSource": { "title": "Max items per source", "minimum": 1, "maximum": 50000, "type": "integer", "description": "Cap on items returned per user (repositories) or per search query. Pagination is walked across multiple pages until this is reached or the source is exhausted. Search is capped at 1000 by the source.", "default": 100 }, "includeUserRepos": { "title": "Include a user's repositories", "type": "boolean", "description": "When on, each user or organization also emits its public repositories (up to the max per source) after its profile row.", "default": true }, "searchSort": { "title": "Search sort", "enum": [ "", "stars", "forks", "updated" ], "type": "string", "description": "Sort order for search results. Best match (relevance), or by stars, forks, or recent updates.", "default": "" }, "githubToken": { "title": "Access token (optional)", "type": "string", "description": "Optional personal access token. Not required. When provided it raises the request budget for large runs. A read-only token with public access is sufficient." }, "concurrency": { "title": "Concurrency", "minimum": 1, "maximum": 20, "type": "integer", "description": "How many sources to process in parallel.", "default": 5 }, "proxyConfig": { "title": "Proxy configuration", "type": "object", "description": "Optional proxy. The public read endpoints do not require a proxy, but you may route through one if you wish.", "default": { "useApifyProxy": false } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```