# GitHub Repo & Developer Scraper - Repos and Users as Rows (`flash_scraper/github-repo-scraper`) Actor Search the GitHub REST API for repositories or developers and get clean rows: stars, forks, language, topics, license & URLs for repos; login, name, bio, company, location, public email & follower counts for users. Keyless (60 req/hr) or add a token for 5,000/hr. Public data. Export CSV/JSON/Excel. - **URL**: https://apify.com/flash\_scraper/github-repo-scraper.md - **Developed by:** [Flash Scrape](https://apify.com/flash_scraper) (community) - **Categories:** Developer tools, Lead generation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $5.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## GitHub Repo & Developer Scraper — repos and developers as clean rows Turn a **GitHub search** into a clean, exportable table. Search **repositories** (stars, forks, language, topics, license, activity dates) or **developers** (login, name, bio, company, location, public email, follower counts) by keyword, language and sort order — then export straight to CSV, JSON, or Excel. Works **keyless** out of the box; add an optional token for higher rate limits. **$5 per 1,000 results** — pay only for the rows you actually get. ### What does it do? This actor queries the official **GitHub REST API** and flattens the deeply-nested JSON into one row per repository or per developer. For **repositories** you get the popularity signals that matter — stars, forks, watchers, open issues — plus the language, topic tags, license, homepage, and the created / updated / pushed timestamps that tell you whether a project is alive or abandoned. For **developers** you get the profile fields recruiters and BD teams care about — name, bio, company, location, public email (when the user has made it public), public-repo count, and follower / following counts — enriched from each user's full profile. No scraping of private data, no login, no anti-bot bypass. It reads the same public API that powers github.com search, and returns exactly what a logged-out visitor could see. ### Why use it / who's it for - **Developer-tool & DevTool SaaS teams** doing market research: pull the top 300 repos for `language:rust cli`, `vector database`, or `llm agent` and see who the incumbents are, how fast they're growing, and what they're licensed under. - **Open-source maintainers and investors** tracking a trend: sort by *recently updated* to catch fast-moving new projects in a topic, or by *stars* to snapshot the leaders in a category. - **Developer lead-generation & devrel teams** building a list of authors active in a niche (e.g. everyone publishing `terraform` modules) to invite to a beta, a community, or a sponsorship. - **Technical recruiters** sourcing candidates by searching users for a skill keyword and location, then filtering to profiles with a public email or company for outreach. - **Analysts and data teams** who want a reproducible, exportable snapshot of a GitHub category instead of clicking through the web UI. ### How to use it 1. Enter a **search query** (e.g. `machine learning`, `react state management`, or a topic like `kubernetes operator`). GitHub qualifiers such as `stars:>1000` work too. 2. Pick a **search type** — *Repositories* or *Users / developers*. 3. For repositories, optionally set a **language** filter (`python`, `typescript`, `go`) and a **sort** order (stars, forks, or recently updated). 4. Set **max results** (up to 300). 5. Optionally paste a **GitHub token** to lift the rate limit from 60 to 5,000 requests/hour. 6. Run it, then export the dataset to CSV, JSON, or Excel. ### Output fields **Repositories:** | Field | Description | |---|---| | `full_name` | `owner/repo` identifier | | `owner` | Owner login (user or org) | | `description` | Repo description (trimmed) | | `stars` | Stargazer count | | `forks` | Fork count | | `watchers` | Watcher count | | `open_issues` | Open issues + PRs | | `language` | Primary language | | `topics` | List of topic tags | | `license` | SPDX id or license name | | `url` | Repository URL | | `homepage` | Project homepage (if set) | | `created_at`, `updated_at`, `pushed_at` | Timestamps (ISO 8601) | | `is_archived` | Whether the repo is archived | **Users / developers:** | Field | Description | |---|---| | `login` | GitHub username | | `name` | Display name | | `url` | Profile URL | | `type` | `User` or `Organization` | | `bio` | Profile bio | | `company` | Company field | | `location` | Location field | | `email` | Public email (if the user made it public) | | `public_repos` | Public repo count | | `followers`, `following` | Follower / following counts | | `created_at` | Account creation date | #### Example output A real repository row from a live run: | Repository | Owner | Stars | Forks | Language | License | URL | |---|---|---|---|---|---|---| | tensorflow/tensorflow | tensorflow | 196097 | 75336 | C++ | Apache-2.0 | https://github.com/tensorflow/tensorflow | ### Input example ```json { "query": "machine learning", "searchType": "repositories", "language": "python", "sort": "stars", "maxItems": 30 } ```` Only `query` is required — every other field has a sensible default. ### How much does it cost? This actor uses **pay-per-event pricing: $0.005 per result ($5 per 1,000 results)**. You're charged only for rows actually returned — no subscription, no charge for empty runs. New Apify accounts get roughly **1,000 free results per month** on the platform's free tier, so you can pull a full category snapshot and test the output before paying anything. A 5,000-row pull costs $25; a 20,000-row research campaign costs $100. ### FAQ #### Do I need a GitHub token? No. The actor works **keyless**, using GitHub's unauthenticated API which allows **60 requests per hour** — enough for a small pull (the default 30-result run fits easily). If you need larger or more frequent runs, create a free **personal access token** (no scopes required for public data) and paste it into the `githubToken` field to raise the limit to **5,000 requests/hour**. The token is stored as a secret. #### What happens when I hit the rate limit? The actor stops **gracefully**. It pushes whatever it collected so far, sets a clear status message (*"GitHub API rate limit reached — please retry."*), and exits cleanly — it never crashes and never hammers the API. Add a token or wait for the hourly window to reset, then re-run. #### Why do I get at most 1,000 results per query? That's a GitHub API limit, not the actor's: **search returns only the first 1,000 matches** for any query. To go deeper, narrow the query — split by language, by star range (`stars:1000..5000`), or by topic — and run each slice separately. #### Why is `email` often empty for users? GitHub only exposes an email through the API when the user has **chosen to make it public** on their profile. Most developers don't, so treat a populated `email` as a bonus, not a guarantee. All data returned is public. #### Is this legal? Yes. The actor reads only **publicly available data** from GitHub's official REST API — the same information any logged-out visitor can see. It doesn't log in, bypass limits, or access private repositories or profiles. Always follow applicable outreach laws (CAN-SPAM, GDPR, etc.) when you use any contact data. ### Other Flash Scrape tools - [Company & Domain Enricher](https://apify.com/flash_scraper/company-domain-enrichment) — turn a domain list into full company records with tech stack and socials - [Bulk Email Verifier](https://apify.com/flash_scraper/email-verifier) — verify and score every email address before you send a campaign Questions or a field you need added? Email **zakaria@lysi.co** — happy to help. # Actor input Schema ## `query` (type: `string`): Keyword(s) to search GitHub for, e.g. 'machine learning', 'react state management', or a topic like 'kubernetes operator'. GitHub search qualifiers (e.g. 'stars:>1000') also work. ## `searchType` (type: `string`): Search repositories or users/developers. ## `language` (type: `string`): Filter repositories by primary language, e.g. 'python', 'typescript', 'go'. Applies to repository searches only. ## `sort` (type: `string`): Sort order for repository results (users are returned by best match). ## `maxItems` (type: `integer`): Maximum rows to return. GitHub search returns at most the first 1,000 matches for any query; keyless runs are also capped by the 60 requests/hour rate limit. ## `githubToken` (type: `string`): Optional GitHub personal access token for higher rate limits (5000/hr vs 60/hr). ## Actor input object example ```json { "query": "machine learning", "searchType": "repositories", "sort": "stars", "maxItems": 30 } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "query": "machine learning", "searchType": "repositories", "sort": "stars", "maxItems": 30 }; // Run the Actor and wait for it to finish const run = await client.actor("flash_scraper/github-repo-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "query": "machine learning", "searchType": "repositories", "sort": "stars", "maxItems": 30, } # Run the Actor and wait for it to finish run = client.actor("flash_scraper/github-repo-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "query": "machine learning", "searchType": "repositories", "sort": "stars", "maxItems": 30 }' | apify call flash_scraper/github-repo-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=flash_scraper/github-repo-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "GitHub Repo & Developer Scraper - Repos and Users as Rows", "description": "Search the GitHub REST API for repositories or developers and get clean rows: stars, forks, language, topics, license & URLs for repos; login, name, bio, company, location, public email & follower counts for users. Keyless (60 req/hr) or add a token for 5,000/hr. Public data. Export CSV/JSON/Excel.", "version": "0.1", "x-build-id": "FKhinxKkl2jtGs4aM" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/flash_scraper~github-repo-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-flash_scraper-github-repo-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/flash_scraper~github-repo-scraper/runs": { "post": { "operationId": "runs-sync-flash_scraper-github-repo-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/flash_scraper~github-repo-scraper/run-sync": { "post": { "operationId": "run-sync-flash_scraper-github-repo-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "query": { "title": "Search query", "type": "string", "description": "Keyword(s) to search GitHub for, e.g. 'machine learning', 'react state management', or a topic like 'kubernetes operator'. GitHub search qualifiers (e.g. 'stars:>1000') also work." }, "searchType": { "title": "Search type", "enum": [ "repositories", "users" ], "type": "string", "description": "Search repositories or users/developers.", "default": "repositories" }, "language": { "title": "Programming language", "type": "string", "description": "Filter repositories by primary language, e.g. 'python', 'typescript', 'go'. Applies to repository searches only." }, "sort": { "title": "Sort by", "enum": [ "stars", "forks", "updated" ], "type": "string", "description": "Sort order for repository results (users are returned by best match).", "default": "stars" }, "maxItems": { "title": "Max results", "minimum": 1, "maximum": 300, "type": "integer", "description": "Maximum rows to return. GitHub search returns at most the first 1,000 matches for any query; keyless runs are also capped by the 60 requests/hour rate limit.", "default": 50 }, "githubToken": { "title": "GitHub token (optional)", "type": "string", "description": "Optional GitHub personal access token for higher rate limits (5000/hr vs 60/hr)." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```