# GitHub Repo Scraper: Stars, Releases, Licenses, Dependencies (`nightwave-owner/github-repo-intelligence`) Actor Returns one row per GitHub repository with stars, forks, topics, license, languages with share, latest commit, latest release and release count, and package.json, pyproject.toml or go.mod details. Repository lists or GitHub search, optional token, onlyNew for release monitoring. - **URL**: https://apify.com/nightwave-owner/github-repo-intelligence.md - **Developed by:** [Viktor Wiberg](https://apify.com/nightwave-owner) (community) - **Categories:** Developer tools, Automation, AI - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing $4.00 / 1,000 repositories This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## GitHub Repo Scraper: Stars, Releases, Licenses, Dependencies A GitHub scraper that returns one row per repository with the facts you would otherwise collect by hand from several pages: GitHub stars, forks, topics, license, the languages and their share of the code, the latest commit on the default branch, the latest release and how many releases there are, and the package name, version and dependency count from `package.json`, `pyproject.toml` or `go.mod`. Give it a list of repositories, a GitHub search such as `topic:web-scraping stars:>500`, or both. It reads the official GitHub REST API and works without a login. A GitHub token is optional and only raises the rate limit. The actor returns repository data only. It does not read user profiles, email addresses, follower lists or the names of commit authors and contributors. ### Example from a real run Run `MBr1GDZhdckCRZ8bB` on Apify on 5 October 2026, without a token, with this input: ```json { "repos": ["cli/cli", "apify/crawlee-python", "microsoft/playwright", "psf/requests", "denoland/deno"], "includeReleases": true, "includeManifest": true } ``` It returned 5 rows in 5 seconds of run time and used 20 of the 60 GitHub requests allowed per hour without a token. The first row, with `languages` cut to two entries and `source`, `sourceLicense` and `retrievedAt` left out here to keep it short: ```json { "fullName": "cli/cli", "owner": "cli", "name": "cli", "url": "https://github.com/cli/cli", "status": "ok", "error": null, "description": "GitHub’s official command line tool", "homepage": "https://cli.github.com", "topics": ["cli", "git", "github-api-v4", "golang"], "license": "MIT", "licenseName": "MIT License", "stars": 46544, "forks": 9121, "watchers": 1140, "openIssues": 1116, "defaultBranch": "trunk", "primaryLanguage": "Go", "languages": [ { "language": "Go", "bytes": 7892539, "share": 99.4 }, { "language": "Shell", "bytes": 41794, "share": 0.5 } ], "isArchived": false, "isFork": false, "isTemplate": false, "sizeKb": 81873, "createdAt": "2019-10-03T15:24:53Z", "updatedAt": "2026-10-05T02:41:52Z", "pushedAt": "2026-10-02T23:13:52Z", "lastCommitSha": "6fc1c29d5477bfe71da7af290eb481c0df7811f1", "lastCommitDate": "2026-10-02T16:05:07Z", "latestReleaseTag": "v2.102.0", "latestReleaseName": "GitHub CLI 2.102.0", "latestReleaseDate": "2026-09-30T02:40:02Z", "latestReleasePrerelease": false, "latestReleaseUrl": "https://github.com/cli/cli/releases/tag/v2.102.0", "releaseCount": 205, "manifests": [ { "file": "go.mod", "ecosystem": "go", "packageName": "github.com/cli/cli/v2", "version": null, "dependencies": 61, "devDependencies": 118 } ] } ``` The other four rows in the same run: `apify/crawlee-python` (Apache-2.0, release v1.10.3, `pyproject.toml` with 13 dependencies), `microsoft/playwright` (Apache-2.0, release v1.63.0, `package.json`), `psf/requests` (Apache-2.0, release v2.34.2, 19 releases) and `denoland/deno` (MIT, release v2.9.7, 390 releases, no manifest in the root since it is a Rust project). A search run on the same day (run `DgYbbzawDgrRmDqlb`, input `{"search": "topic:web-scraping stars:>500", "sort": "stars", "maxResults": 10}`) found 133 matching repositories and returned the 10 with the most stars, starting with `firecrawl/firecrawl`, `D4Vinci/Scrapling` and `unclecode/crawl4ai`. With empty input the actor reads `apify/crawlee` and `apify/apify-sdk-js` as an example (run `yDFLkbdeyDRY2XEXW`, 2 rows, 8 GitHub requests). ### Input | Field | Type | Default | Description | |---|---|---|---| | `repos` | array | | Repositories as `owner/name` or a github.com link, for example `apify/crawlee` or `https://github.com/cli/cli`. | | `search` | string | | A GitHub repository search with the same syntax as on github.com, for example `topic:web-scraping stars:>500` or `language:go stars:>10000 pushed:>2026-09-01`. | | `sort` | string | `stars` | Order of the search results: `stars`, `forks`, `updated`, `help-wanted-issues` or `best-match`. | | `githubToken` | string | | Optional. Raises the limit from 60 to 5 000 GitHub requests per hour. See "GitHub token". | | `includeReleases` | boolean | `true` | Latest release and number of releases. One extra request per repository. | | `includeManifest` | boolean | `true` | Package name, version and dependency counts from `package.json`, `pyproject.toml` and `go.mod` in the root of the default branch. | | `maxResults` | integer | `50` | Maximum number of repositories. Named repositories count first, then search results. 1 to 1 000. | | `onlyNew` | boolean | `false` | Return only repositories with a new push or a new release since the last run with the same input. See "Monitoring and scheduling". | When both `repos` and `search` are set, the named repositories come first and the search fills up to `maxResults`. Without either, the two example repositories are read. ### Output | Field | Description | |---|---| | `fullName`, `owner`, `name`, `url` | The repository and its link | | `status`, `error` | `ok`, or `error` with the reason (not found, private, rate limit reached). Error rows are not charged | | `description`, `homepage`, `topics` | As set by the maintainers | | `license`, `licenseName` | SPDX id and name as detected by GitHub, for example `MIT`. `NOASSERTION` means GitHub found a license file it could not classify. `null` means no license file | | `stars`, `forks`, `watchers` | Stargazers, forks and subscribers. `watchers` is `null` for rows from a search, since the search API does not return it | | `openIssues` | Open issues and pull requests together, as GitHub counts them | | `defaultBranch`, `primaryLanguage` | Default branch and the language GitHub shows for the repository | | `languages` | Every language with bytes of code and percent of the total, largest first | | `isArchived`, `isFork`, `isTemplate` | Repository flags | | `sizeKb`, `createdAt`, `updatedAt`, `pushedAt` | Size of the repository and its dates, UTC | | `lastCommitSha`, `lastCommitDate` | The newest commit on the default branch and its commit date. No author name or email | | `latestReleaseTag`, `latestReleaseName`, `latestReleaseDate`, `latestReleasePrerelease`, `latestReleaseUrl` | The most recently created release, which can be a prerelease (see the flag) | | `releaseCount` | Number of published releases. Tags without a release are not counted | | `manifests` | One entry per manifest found: `file`, `ecosystem` (`npm`, `python`, `go`), `packageName`, `version`, `dependencies`, `devDependencies` | | `source`, `sourceLicense`, `retrievedAt` | Attribution and the time of the run | How dependencies are counted: in `package.json`, `dependencies` and `devDependencies`. In `pyproject.toml`, `[project] dependencies`, and as `devDependencies` the optional dependencies plus dependency groups (for Poetry projects, `[tool.poetry.dependencies]` without `python`, and the dev groups). In `go.mod`, direct requirements as `dependencies` and `// indirect` ones as `devDependencies`. A version written as `dynamic` in `pyproject.toml` comes back as `null`. ### GitHub token The actor works without a token. GitHub then allows 60 API requests per hour per IP address, and Apify's IP addresses are shared, so part of that hour may already be used by others when your run starts. Requests per repository: 1 for the repository record (0 when it comes from a search), 1 for languages, 1 for the latest commit and 1 for releases when `includeReleases` is on. Manifests are read from `raw.githubusercontent.com` and do not count. So without a token: - 50 named repositories need 200 requests, and a run reads about 14 of them before the limit. - A search for 50 repositories needs 151 requests (1 search plus 3 per repository) and reads about 19. - With `includeReleases` off, each repository needs one request less. When the limit is reached, the actor stops calmly: every repository it did not read gets a row with `status` `error` and the time the limit resets, the run finishes as succeeded and those rows are not charged. A short secondary limit (a `Retry-After` of up to 60 seconds) is waited out and retried. With a token the limit is 5 000 requests per hour, enough for about 1 250 repositories. To create one that can only read public data: 1. On GitHub, open Settings, Developer settings, Personal access tokens, **Fine-grained tokens**, and choose Generate new token. 2. Give it a name and an expiry date. Under Repository access, choose **Public repositories (read-only)**. Leave all permissions at No access. 3. Generate it and paste it into `githubToken`. The field is marked secret, so Apify stores it encrypted and the actor never writes it to the log or the dataset. A token cannot read anything private here: the actor skips any repository that is not public. Use your own token only. GitHub's terms do not allow sharing tokens to get around its rate limits. ### Monitoring and scheduling Set `onlyNew` to `true` to watch a list of repositories on a schedule. The actor remembers what it has delivered for that input in a named key-value store in your Apify account (`nightwave-state-github-repo-intelligence`, one record per input). A repository is delivered again when its `pushedAt` time or its latest release tag changes, so you get a row after a new push or a new release, and nothing for repositories that have not changed. Only delivered rows are charged. The first run returns everything. `onlyNew`, `maxResults` and `githubToken` are not part of the remembered input, so you can change them without losing the state. Changing any other field starts a fresh state. To start over, delete the record in the key-value store. In a test on 5 October 2026, the input `{"repos": ["apify/crawlee", "cli/cli", "psf/requests"], "onlyNew": true}` returned 3 rows on the first run (run `ZoikDDQWPorHclsPg`) and 0 rows on a second run right after (run `YXheiYG7rD2cuFqL6`), since nothing had changed in between. A second run like that, with `onlyNew` and the same input, is not charged. Example: every morning at 07:00 Swedish time, check the dependencies you care about for new releases. In Apify Console, open **Schedules**, create a schedule with the cron expression `0 7 * * *` and add this actor with the input below. ```json { "repos": ["nodejs/node", "microsoft/playwright", "apify/crawlee", "psf/requests"], "onlyNew": true } ``` The same schedule through the Apify API: ```sh curl -X POST "https://api.apify.com/v2/schedules?token=" \ -H "Content-Type: application/json" \ -d '{"name": "daily-release-check", "cronExpression": "0 7 * * *", "timezone": "Europe/Stockholm", "isEnabled": true, "isExclusive": true, "actions": [{"type": "RUN_ACTOR", "actorId": "nightwave-owner~github-repo-intelligence", "runInput": {"contentType": "application/json; charset=utf-8", "body": ""}}]}' ``` Add an Apify integration (email, Slack or a webhook) on the schedule to be told when a run has rows. A push to any branch counts as a change, so for busy repositories you will get a row most days. Compare `latestReleaseTag` with the previous row if you only care about releases. ### Use cases - **Release monitoring of dependencies.** Run the repositories your product depends on every day with `onlyNew` and get a row only when one of them has pushed or published a new release. - **Due diligence on open source.** Before you adopt a project, check its maintenance activity (last commit, last release, release count, archived or not), its size and who stands behind it. - **License inventory.** List the license of every repository in a dependency list or an organization, and find projects without a license or with one your policy does not allow. - **Market research for developer tools.** Pull the top repositories for a topic or language with stars, forks, release cadence and package names. ### Limitations - GitHub's search returns at most 1 000 repositories per query. Split a large query, for example by `created:` ranges, to cover more. - `releaseCount` counts releases, not git tags. Some projects only tag and never publish releases on GitHub; they have `releaseCount` 0. - The latest release is the most recently created one and can be a prerelease. Check `latestReleasePrerelease`. - Manifests are read only from the root of the default branch. Monorepos often have a private root `package.json` (for example `@crawlee/root` with no version), and projects with their manifest in a subfolder return an empty list. - `openIssues` includes open pull requests, since that is how GitHub counts it. - Requests are retried three times on network and server errors. ### Source and license - **Source:** the GitHub REST API (`https://api.github.com`, version `2022-11-28`) and, for manifests, `https://raw.githubusercontent.com`. Every row carries `source` and `sourceLicense`. - **GitHub's terms:** the API is used under Section H (API Terms) of the GitHub Terms of Service, which forbids excessive request rates and sharing tokens to exceed the rate limits. The actor stays within the limits, identifies itself with a User-Agent and stops when the limit is reached. GitHub's Acceptable Use Policies (Section 7, Information Usage Restrictions) forbid using information from GitHub for spamming or for selling personal information, for example to recruiters. This actor returns no personal information. You are responsible for using the data in line with GitHub's terms and the GitHub Privacy Statement. - **The repositories:** metadata such as stars and release dates is facts about a repository. The code and files in each repository are under that repository's own license, shown in the `license` field. The actor reads manifests only to count and name packages; it does not copy code. This actor is not affiliated with or endorsed by GitHub. ### Pricing Pay per event: 0.004 USD per repository returned (event `repository`), which is 4.00 USD per 1 000 repositories. Apify platform usage comes on top and is small, since the actor only makes API requests: the test runs above used between 0.0003 and 0.0005 USD each. Error rows, unchanged repositories skipped by `onlyNew` and runs without results are not charged. `maxResults` caps how many repositories, and therefore how much, a run can charge. Rows are delivered only after they have been charged. If you set a maximum cost per run (maxTotalChargeUsd), the run stops there and its status message says how many rows were delivered. ### Contact Built and maintained by Nightwave AB. Questions, bugs and feature requests: kontakt@nightwave.se ### På svenska Actorn hämtar fakta om GitHub-repon via GitHubs officiella API och ger en rad per repo: stjärnor, forks, ämnen, licens, språk med andel, senaste commit, senaste release och antal releaser, samt paketnamn, version och antal beroenden från `package.json`, `pyproject.toml` eller `go.mod`. - Indata: en lista med repon (`owner/name` eller länk), en GitHub-sökning (till exempel `topic:web-scraping stars:>500`) eller båda. Standard är 50 repon. Tom input läser `apify/crawlee` och `apify/apify-sdk-js` som exempel. - Inga personuppgifter: actorn läser inga användarprofiler, e-postadresser eller namn på dem som gjort commits. - Token är valfri. Utan token tillåter GitHub 60 anrop i timmen, vilket räcker till cirka 14 namngivna repon eller 19 från en sökning per körning. När gränsen nås avslutas körningen lugnt med en felrad per repo som inte lästes, och felrader debiteras inte. En fine-grained token med bara läsrätt till publika repon höjer gränsen till 5 000 anrop i timmen. - Bevakning: med `onlyNew` kommer actorn ihåg vad den har levererat (key-value store `nightwave-state-github-repo-intelligence`) och levererar bara repon med en ny push eller en ny release sedan förra körningen. Lägg den på ett dagligt schema i Apify under Schedules, se avsnittet "Monitoring and scheduling". - Källa: GitHub REST API enligt GitHubs användarvillkor (avsnitt H, API Terms). Koden i varje repo har sin egen licens, som står i fältet `license`. - Pris: 0,004 USD per repo (4,00 USD per 1 000), plus Apifys plattformsanvändning. - Kontakt: kontakt@nightwave.se # Actor input Schema ## `repos` (type: `array`): Repositories to read, as owner/name or a github.com link, for example \["apify/crawlee"]. Empty together with an empty search reads apify/crawlee and apify/apify-sdk-js as an example. ## `search` (type: `string`): A GitHub repository search, with the same syntax as on github.com, for example "topic:web-scraping stars:>500". Matching repositories are read after the ones in Repositories. ## `sort` (type: `string`): Order of the search results: stars, forks, updated, help-wanted-issues or best-match. Defaults to stars. Applies only to the search query. ## `githubToken` (type: `string`): Optional. A GitHub personal access token, for example a fine-grained token with read access to public repositories only. Raises GitHub's limit from 60 to 5 000 requests per hour. Without it, a run reads about 14 named or 19 searched repositories. Stored encrypted and never written to the output or the log. No default, because it is a secret. ## `includeReleases` (type: `boolean`): Read the latest release (tag, name, date, prerelease) and the number of releases, for example true. Costs one GitHub request per repository. Defaults to true. ## `includeManifest` (type: `boolean`): Read package.json, pyproject.toml and go.mod in the default branch and return package name, version and number of dependencies, for example true. These files are read from raw.githubusercontent.com and do not count against the API limit. Defaults to true. ## `maxResults` (type: `integer`): Maximum number of repositories to read, for example 50. Named repositories count first, then search results. 1 to 1 000, defaults to 50. ## `onlyNew` (type: `boolean`): Return only repositories that have a new push or a new latest release since the last run with the same input, for example true for a daily schedule. The first run returns all. Defaults to false. ## Actor input object example ```json { "repos": [ "apify/crawlee", "microsoft/playwright", "golang/go" ], "search": "topic:web-scraping stars:>500", "sort": "updated", "githubToken": "github_pat_...", "includeReleases": true, "includeManifest": true, "maxResults": 50, "onlyNew": true } ``` # Actor output Schema ## `results` (type: `string`): One row per repository, as JSON. Open in Apify Console or download via the dataset API. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "repos": [ "apify/crawlee", "apify/apify-sdk-js" ], "sort": "stars", "includeReleases": true, "includeManifest": true, "maxResults": 50, "onlyNew": false }; // Run the Actor and wait for it to finish const run = await client.actor("nightwave-owner/github-repo-intelligence").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "repos": [ "apify/crawlee", "apify/apify-sdk-js", ], "sort": "stars", "includeReleases": True, "includeManifest": True, "maxResults": 50, "onlyNew": False, } # Run the Actor and wait for it to finish run = client.actor("nightwave-owner/github-repo-intelligence").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "repos": [ "apify/crawlee", "apify/apify-sdk-js" ], "sort": "stars", "includeReleases": true, "includeManifest": true, "maxResults": 50, "onlyNew": false }' | apify call nightwave-owner/github-repo-intelligence --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,nightwave-owner/github-repo-intelligence" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/RNMfRBOOW1G4Ik95A/builds/DXbdyfHXtfu7W6JfK/openapi.json