# GitHub Issues Scraper (`glassventures/github-issues-scraper`) Actor Scrape GitHub issues from repos, orgs, or search queries. Extract titles, labels, assignees, comments, reactions. Export to JSON, CSV, Excel. - **URL**: https://apify.com/glassventures/github-issues-scraper.md - **Developed by:** [Glass Ventures](https://apify.com/glassventures) (community) - **Categories:** Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## GitHub Issues Scraper Scrape GitHub issues from any repository, organization, or search query using the GitHub REST API. Extract titles, bodies, labels, assignees, comments, reactions, milestones, and more. ### What does GitHub Issues Scraper do? GitHub Issues Scraper lets you extract structured issue data from GitHub repositories at scale. Instead of manually browsing through hundreds of issues or writing custom API scripts, this actor handles pagination, rate limiting, and data normalization automatically. It works with individual repositories, entire organizations (scrapes issues from all public repos), and GitHub's powerful issue search. The actor hits the official GitHub REST API directly, so the data is always accurate and complete. Whether you need to track open bugs across a competitor's repos, analyze issue trends for market research, or export issues for project management dashboards, this actor delivers clean, structured data ready for analysis. ### Use Cases - **Open source maintainers** -- Export all issues from your repositories for analysis and triage in spreadsheets or dashboards - **Market researchers** -- Track competitor product issues and feature requests to identify market gaps - **Data analysts** -- Analyze issue trends, response times, and community engagement across repositories - **Developers** -- Monitor libraries and frameworks for bugs and breaking changes - **Project managers** -- Bulk export issues with labels, milestones, and assignees for reporting ### Features - Scrape issues from any public GitHub repository - Scrape all repos in an organization automatically - Search issues across all of GitHub with search queries - Filter by issue state (open, closed, all) - Optionally fetch all comments for each issue - Extract reactions, labels, assignees, milestones - Support for GitHub personal access tokens (5,000 requests/hour vs 60) - Automatic pagination and rate limit handling - Exports to JSON, CSV, Excel, or connect via API ### How much will it cost? GitHub Issues Scraper uses the GitHub REST API, which is very efficient. Most runs complete quickly with minimal compute. | Results | Estimated Cost | |---------|---------------| | 100 | ~$0.01 | | 1,000 | ~$0.05 | | 10,000 | ~$0.25 | | Cost Component | Per 1,000 Results | |----------------|-------------------| | Platform compute | ~$0.03 | | Proxy (optional) | ~$0.00 | | **Total** | **~$0.03** | Note: Fetching comments increases API calls and run time. Without a GitHub token, the rate limit is 60 requests/hour, which limits throughput. ### How to use 1. Go to the GitHub Issues Scraper page on Apify Store 2. Click "Start" or "Try for free" 3. Enter GitHub repository URLs (e.g., `https://github.com/apify/crawlee`) or search terms 4. Optionally set the issue state filter and whether to include comments 5. Set the maximum number of issues to scrape 6. Click "Start" and wait for the results ### Input parameters | Parameter | Type | Description | Default | |-----------|------|-------------|---------| | startUrls | array | GitHub repo or org URLs | - | | searchTerms | array | Search queries for GitHub issues | - | | issueState | string | Filter: all, open, or closed | all | | includeComments | boolean | Fetch comments for each issue | false | | githubToken | string | Personal access token for higher rate limits | - | | maxItems | number | Max issues to return | 100 | | maxConcurrency | number | Parallel API requests | 5 | | proxyConfig | object | Proxy settings | Apify Proxy | ### Output The actor produces a dataset with the following fields: ```json { "url": "https://github.com/apify/crawlee/issues/1234", "issueNumber": 1234, "title": "Bug: CheerioCrawler timeout on large pages", "body": "### Description\nWhen crawling pages larger than 5MB...", "state": "open", "author": "username", "authorUrl": "https://github.com/username", "labels": ["bug", "priority-high"], "assignees": ["maintainer1"], "commentsCount": 5, "reactionsCount": 3, "reactions": { "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 }, "milestone": "v3.5.0", "isPullRequest": false, "repository": "apify/crawlee", "createdAt": "2024-01-15T10:30:00Z", "updatedAt": "2024-01-20T14:00:00Z", "closedAt": null, "comments": null, "scrapedAt": "2024-01-25T12:00:00.000Z" } ```` | Field | Type | Description | |-------|------|-------------| | url | string | Issue URL on GitHub | | issueNumber | number | Issue number in the repository | | title | string | Issue title | | body | string | Issue body content (Markdown) | | state | string | open or closed | | author | string | Username of issue creator | | authorUrl | string | Profile URL of issue creator | | labels | array | List of label names | | assignees | array | List of assigned usernames | | commentsCount | number | Number of comments | | reactionsCount | number | Total reaction count | | reactions | object | Reaction breakdown by type | | milestone | string | Milestone name | | isPullRequest | boolean | Whether entry is a pull request | | repository | string | Repository full name (owner/repo) | | createdAt | string | ISO 8601 creation date | | updatedAt | string | ISO 8601 last update date | | closedAt | string | ISO 8601 close date | | comments | array | Full comment data (when includeComments is true) | | scrapedAt | string | ISO 8601 scrape timestamp | ### Integrations Connect GitHub Issues Scraper with other tools: - **Apify API** -- REST API for programmatic access - **Webhooks** -- get notified when a run finishes - **Zapier / Make** -- connect to 5,000+ apps - **Google Sheets** -- export directly to spreadsheets #### API Example (Node.js) ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: 'YOUR_TOKEN' }); const run = await client.actor('YOUR_USERNAME/github-issues-scraper').call({ startUrls: [{ url: 'https://github.com/apify/crawlee' }], maxItems: 100, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); ``` #### API Example (Python) ```python from apify_client import ApifyClient client = ApifyClient('YOUR_TOKEN') run = client.actor('YOUR_USERNAME/github-issues-scraper').call(run_input={ 'startUrls': [{'url': 'https://github.com/apify/crawlee'}], 'maxItems': 100, }) items = client.dataset(run['defaultDatasetId']).list_items().items ``` #### API Example (cURL) ```bash curl "https://api.apify.com/v2/acts/YOUR_USERNAME~github-issues-scraper/runs" \ -X POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer YOUR_TOKEN" \ -d '{"startUrls": [{"url": "https://github.com/apify/crawlee"}], "maxItems": 100}' ``` ### Tips and tricks - Start with a small `maxItems` (10-20) to test before running large scrapes - Add a GitHub personal access token to increase rate limits from 60 to 5,000 requests/hour - Use `issueState: "open"` to only get active issues and reduce data volume - Enable `includeComments` only when you need comment data, as it significantly increases API calls - For organization URLs, all public repos will be scraped -- combine with `maxItems` to control volume - GitHub search is limited to 1,000 results per query -- use specific search terms for best results ### FAQ **Q: Does this actor require login credentials?** A: No. GitHub's REST API is publicly accessible. However, adding a personal access token increases the rate limit from 60 to 5,000 requests per hour. **Q: How fast is the scraping?** A: Without a token: ~50-60 issues per hour (rate limited). With a token: ~5,000-10,000 issues per hour depending on whether comments are included. **Q: What should I do if I get rate limited?** A: Add a GitHub personal access token in the Authentication section. You can create one at github.com/settings/tokens with no special permissions needed for public repos. **Q: Does it scrape pull requests too?** A: GitHub's issues API includes pull requests. Each item has an `isPullRequest` field so you can filter them out if needed. **Q: Can I scrape private repositories?** A: Yes, if you provide a GitHub personal access token that has access to the private repository. ### Is it legal to scrape GitHub? This actor uses the official GitHub REST API, which is a public API designed for programmatic access. It respects rate limits and follows GitHub's API usage guidelines. GitHub's API Terms of Service permit accessing public data. Always review GitHub's Terms of Service for your specific use case. For more information, see [Apify's blog on web scraping legality](https://blog.apify.com/is-web-scraping-legal/). ### Limitations - Without a GitHub token, rate limit is 60 API requests per hour - GitHub search API returns a maximum of 1,000 results per query - Only public repositories are accessible without authentication - Pull requests are included in the issues API (filterable via `isPullRequest` field) ### Changelog - **v0.1** (2026-04-23) -- Initial release # Actor input Schema ## `startUrls` (type: `array`): GitHub repository URLs to scrape issues from. Supports repo URLs (github.com/owner/repo) and org URLs (github.com/orgs/orgname). ## `searchTerms` (type: `array`): Search queries to find issues across all of GitHub. Uses GitHub's issue search syntax. ## `issueState` (type: `string`): Filter issues by state. ## `includeComments` (type: `boolean`): Fetch and include all comments for each issue. Increases API usage and run time. ## `githubToken` (type: `string`): Optional GitHub personal access token. Without it, rate limit is 60 requests/hour. With a token, it's 5,000 requests/hour. ## `maxItems` (type: `integer`): Maximum number of issues to scrape. Use 0 or leave empty for unlimited. ## `maxConcurrency` (type: `integer`): Maximum number of API requests processed in parallel. ## `debugMode` (type: `boolean`): Enables verbose logging for troubleshooting. ## `extendOutputFunction` (type: `string`): A JavaScript function to customize each output item. Receives { data, request }. ## `proxyConfig` (type: `object`): Select proxies to be used. Usually not needed for GitHub API. ## Actor input object example ```json { "startUrls": [ { "url": "https://github.com/apify/crawlee" } ], "searchTerms": [ "apify actor bug" ], "issueState": "all", "includeComments": false, "maxItems": 100, "maxConcurrency": 5, "debugMode": false, "extendOutputFunction": "async ({ data }) => {\n return data;\n}", "proxyConfig": { "useApifyProxy": true } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://github.com/apify/crawlee" } ], "searchTerms": [ "apify actor bug" ], "extendOutputFunction": async ({ data }) => { return data; }, "proxyConfig": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("glassventures/github-issues-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://github.com/apify/crawlee" }], "searchTerms": ["apify actor bug"], "extendOutputFunction": """async ({ data }) => { return data; }""", "proxyConfig": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("glassventures/github-issues-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://github.com/apify/crawlee" } ], "searchTerms": [ "apify actor bug" ], "extendOutputFunction": "async ({ data }) => {\\n return data;\\n}", "proxyConfig": { "useApifyProxy": true } }' | apify call glassventures/github-issues-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=glassventures/github-issues-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "GitHub Issues Scraper", "description": "Scrape GitHub issues from repos, orgs, or search queries. Extract titles, labels, assignees, comments, reactions. Export to JSON, CSV, Excel.", "version": "0.1", "x-build-id": "QTSQxwoPsDQuRIwUY" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/glassventures~github-issues-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-glassventures-github-issues-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/glassventures~github-issues-scraper/runs": { "post": { "operationId": "runs-sync-glassventures-github-issues-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/glassventures~github-issues-scraper/run-sync": { "post": { "operationId": "run-sync-glassventures-github-issues-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "startUrls": { "title": "Repository URLs", "type": "array", "description": "GitHub repository URLs to scrape issues from. Supports repo URLs (github.com/owner/repo) and org URLs (github.com/orgs/orgname).", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "searchTerms": { "title": "Search Terms", "type": "array", "description": "Search queries to find issues across all of GitHub. Uses GitHub's issue search syntax.", "items": { "type": "string" } }, "issueState": { "title": "Issue State", "enum": [ "all", "open", "closed" ], "type": "string", "description": "Filter issues by state.", "default": "all" }, "includeComments": { "title": "Include Comments", "type": "boolean", "description": "Fetch and include all comments for each issue. Increases API usage and run time.", "default": false }, "githubToken": { "title": "GitHub Personal Access Token", "type": "string", "description": "Optional GitHub personal access token. Without it, rate limit is 60 requests/hour. With a token, it's 5,000 requests/hour." }, "maxItems": { "title": "Max Items", "minimum": 0, "type": "integer", "description": "Maximum number of issues to scrape. Use 0 or leave empty for unlimited.", "default": 100 }, "maxConcurrency": { "title": "Max Concurrency", "minimum": 1, "maximum": 20, "type": "integer", "description": "Maximum number of API requests processed in parallel.", "default": 5 }, "debugMode": { "title": "Debug Mode", "type": "boolean", "description": "Enables verbose logging for troubleshooting.", "default": false }, "extendOutputFunction": { "title": "Extend Output Function", "type": "string", "description": "A JavaScript function to customize each output item. Receives { data, request }." }, "proxyConfig": { "title": "Proxy Configuration", "type": "object", "description": "Select proxies to be used. Usually not needed for GitHub API." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```