# GitLab Scraper · Projects, Stars, Issues & Namespaces (`reapx/gitlab-scraper`) Actor Scrape public GitLab projects, repositories, star counts, topics, programming languages, and namespaces without authentication. Returns clean structured metadata with total coverage. - **URL**: https://apify.com/reapx/gitlab-scraper.md - **Developed by:** [Tarek Etman](https://apify.com/reapx) (community) - **Categories:** Developer tools, Business, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.85 / 1,000 gitlab project record scrapeds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ![reapX — public sources in, addressable records out](https://reapx.dev/reapx.gif) ## GitLab Scraper · Projects, Stars, Issues & Namespaces Scrape public GitLab projects, repositories, star counts, topics, programming languages, and namespaces without authentication. Extract clean, structured project metadata directly from GitLab's open REST API. > Maintained by **reapX**. Every row cites the Apify run that produced it — nothing is inferred, modelled or filled in, and a field absent from the source is absent from the row. The extracted archive for this source is browsable at [reapx.dev/data/gitlab-scraper/](https://reapx.dev/data/gitlab-scraper/) and mirrored as an open dataset on [Hugging Face](https://huggingface.co/datasets/reapxdev/gitlab-scraper) and [Kaggle](https://www.kaggle.com/datasets/reapxdev/gitlab-scraper). Questions: reapxdev@proton.me *** ### Overview The **GitLab Scraper** actor extracts structured public repository data from [GitLab.com](https://gitlab.com). Whether you are analyzing open-source software trends, benchmarking developer activity, monitoring organizational namespaces, or harvesting code metadata for machine learning models, this scraper delivers total coverage with addressable entity slugs. Key capabilities include: - **Keyword & Search Queries**: Search project titles, descriptions, and namespace paths for specific keywords. - **Topic Tag Filtering**: Target specific technology stacks and tags like `kubernetes`, `machine-learning`, `devops`, or `docker`. - **Language Filtering**: Filter repositories by primary programming language (`Python`, `TypeScript`, `Rust`, `Go`, `C++`, etc.). - **Star Range Filtering**: Define minimum and maximum star count bounds to target high-visibility projects or emerging codebases. - **Flexible Ordering**: Sort by star count, last activity timestamp, creation date, update date, name, or project ID in descending or ascending sequence. - **Zero Authentication Required**: Access public GitLab data without personal access tokens or API keys. *** ### ⬇️ Input The actor accepts structured JSON configuration allowing precise filtering and pagination parameters. #### Input Parameters | Parameter | Type | Required | Default | Description | | :--- | :--- | :--- | :--- | :--- | | `search` | String | No | `"devops"` (prefill) | Search query term to filter GitLab projects by name, description, or namespace path. | | `topic` | String | No | — | Filter projects tagged with a specific GitLab topic (e.g., `kubernetes`, `react`). | | `language` | String | No | — | Filter projects written in a primary programming language (e.g., `Python`, `Rust`). | | `minStars` | Integer | No | — | Minimum number of stars a project must have received. | | `maxStars` | Integer | No | — | Maximum number of stars a project may have received. | | `visibility` | String | No | `public` | Visibility level (`public`, `internal`, `private`). | | `orderBy` | String | No | `star_count` | Sort field (`star_count`, `last_activity_at`, `created_at`, `updated_at`, `name`, `id`). | | `sort` | String | No | `desc` | Sort direction (`desc` for descending, `asc` for ascending). | | `maxItems` | Integer | No | `50` | Maximum number of project records to fetch and push (1 to 5000). | #### Example Input JSON ```json { "search": "devops", "topic": "kubernetes", "language": "Go", "minStars": 100, "orderBy": "star_count", "sort": "desc", "maxItems": 100 } ``` *** ### ⬆️ Output Every project record is emitted directly to the run's default dataset in standardized camelCase JSON format. Each item carries total identifier coverage (`slug = path_with_namespace`). #### Output Fields Reference Table | Field Name | Type | Example | Description | | :--- | :--- | :--- | :--- | | `slug` | String | `"gitlab-org/gitlab"` | Canonical addressable identifier matching `path_with_namespace`. | | `projectId` | Integer | `278964` | Unique numeric project ID assigned by GitLab. | | `name` | String | `"GitLab"` | Short human-readable project name. | | `nameWithNamespace` | String | `"GitLab / GitLab"` | Full display name including parent namespace or group. | | `description` | String / Null | `"GitLab DevOps platform"` | Project description or summary. | | `createdAt` | String / Null | `"2015-05-20T10:15:30.000Z"` | ISO 8601 UTC timestamp of repository creation. | | `defaultBranch` | String / Null | `"master"` | Primary default git branch (e.g., `master` or `main`). | | `topics` | Array\[String] | `["devops", "git", "ci-cd"]` | List of topic tags associated with the project. | | `starCount` | Integer | `7166` | Total stars awarded to the repository. | | `forksCount` | Integer | `1200` | Total number of repository forks. | | `lastActivityAt` | String / Null | `"2026-08-03T18:00:00.000Z"` | Timestamp of the most recent commit or activity. | | `visibility` | String | `"public"` | Visibility level (`public`, `internal`, `private`). | | `webUrl` | String / Null | `"https://gitlab.com/gitlab-org/gitlab"` | Direct web page URL on GitLab. | | `readmeUrl` | String / Null | `"https://gitlab.com/gitlab-org/gitlab/-/blob/master/README.md"` | Direct URL to project README file. | | `avatarUrl` | String / Null | `"https://gitlab.com/uploads/-/system/project/avatar/278964/logo.png"` | Direct URL to project logo or avatar. | | `sshUrl` | String / Null | `"git@gitlab.com:gitlab-org/gitlab.git"` | Git SSH clone URL. | | `httpUrl` | String / Null | `"https://gitlab.com/gitlab-org/gitlab.git"` | Git HTTP clone URL. | | `namespacePath` | String / Null | `"gitlab-org"` | Full path of the owner namespace or organization group. | | `namespaceKind` | String / Null | `"group"` | Kind of namespace (`group` or `user`). | #### Example Output Record ```json { "slug": "gitlab-org/gitlab", "projectId": 278964, "name": "GitLab", "nameWithNamespace": "GitLab / GitLab", "description": "GitLab DevOps platform", "createdAt": "2015-05-20T10:15:30.000Z", "defaultBranch": "master", "topics": [ "devops", "git", "ci-cd" ], "starCount": 7166, "forksCount": 1200, "lastActivityAt": "2026-08-03T18:00:00.000Z", "visibility": "public", "webUrl": "https://gitlab.com/gitlab-org/gitlab", "readmeUrl": "https://gitlab.com/gitlab-org/gitlab/-/blob/master/README.md", "avatarUrl": "https://gitlab.com/uploads/-/system/project/avatar/278964/logo-extra-large.png", "sshUrl": "git@gitlab.com:gitlab-org/gitlab.git", "httpUrl": "https://gitlab.com/gitlab-org/gitlab.git", "namespacePath": "gitlab-org", "namespaceKind": "group" } ``` *** ### How it works The actor connects directly to GitLab's official v4 REST API (`https://gitlab.com/api/v4/projects`). 1. **Parameter Construction**: The actor formats user filters (`search`, `topic`, `language`, `minStars`, `maxStars`, `visibility`, `orderBy`, `sort`) into standard URL query strings. 2. **Paginated Requests**: HTTP requests fetch 100 items per page over secure connection. 3. **Resilient Rate Handling**: Includes automatic exponential backoff retry logic to seamlessly handle temporary 429 rate limits or 5xx server errors without losing data. 4. **Data Normalization**: Raw API payloads are transformed into structured camelCase schemas with total `slug` addressability. 5. **Item-by-Item Stream & Billing**: Each extracted record is pushed directly to the default dataset while triggering event billing ($0.001 per record), ensuring no memory loss during long runs. *** ### Use Cases & Integration - **Technology Intelligence & Market Research**: Track the growth of specific open-source tools, topics, and programming languages across GitLab codebases. - **Lead Generation & Namespace Discovery**: Discover organizations, companies, and active maintainers hosting open-source software on GitLab. - **Machine Learning & Datasets**: Collect repository metadata for code analysis, recommendation engines, and training data pipelines. - **Competitive Benchmarking**: Monitor star counts, fork activity, and commit freshness across competing developer projects. - **Automated Data Pipelines**: Integrate dataset outputs directly into analytics tools via REST API, Python SDK, or Apify Integrations (Slack, Google Sheets, PostgreSQL, Webhooks). *** ### Performance & Rate Limits - **HTTP Native Speed**: Built without heavy browser overhead. Executes hundreds of record extractions in seconds using lightweight HTTP requests. - **Rate Limit Respect**: Built-in exponential backoff pauses execution if GitLab responds with HTTP 429 backoff signals. - **Budget Control**: Enforces maximum run cost limits to ensure execution never exceeds configured financial thresholds. *** ### ❓ FAQ #### 1. Do I need a GitLab API token to run this actor? No. The actor scrapes public GitLab project data over the unauthenticated REST API endpoint (`https://gitlab.com/api/v4/projects`). #### 2. What field is used as the unique record identifier? The actor emits `slug = path_with_namespace` (e.g. `gitlab-org/gitlab`), ensuring every single record can be uniquely addressed. The numeric project ID is preserved in `projectId`. #### 3. How does pricing work? This actor runs under Pay-Per-Event pricing. You are charged $0.001 per scraped project record pushed to your dataset. Empty runs that return zero rows cost $0 for event charges. #### 4. Can I scrape private GitLab repositories? No. Unauthenticated REST requests can only access public projects (`visibility=public`). #### 5. What format can I export the dataset into? You can download dataset items in JSON, CSV, XML, Excel (XLSX), or NDJSON formats directly from the Console or via Apify API. *** ### 💬 Your feedback We actively maintain our scrapers and update them whenever upstream APIs change. If you encounter bugs, need additional fields, or want to request custom features, contact us at `reapxdev@proton.me`. *** *Disclaimer: Unofficial - not affiliated with GitLab. Collects public data only. reapx. Contact reapxdev@proton.me.* ### 🧪 Example input A real, runnable configuration — this is an actual input this Actor has run with. ```json { "language": "python", "minStars": 10, "orderBy": "star_count", "sort": "desc", "maxItems": 50 } ``` ### 📄 Sample output One real row from a real run of this Actor, unedited. ```json { "slug": "lytor/devops-screening-assignment-rush-deletion_scheduled-85071369", "projectId": 85071369, "name": "devops-screening-assignment-rush-deletion_scheduled-85071369", "nameWithNamespace": "lytor / devops-screening-assignment-rush-deletion_scheduled-85071369", "description": null, "createdAt": "2026-08-03T13:17:36.376Z", "defaultBranch": "main", "topics": [], "starCount": 0, "forksCount": 0, "lastActivityAt": "2026-08-03T15:36:55.972Z", "visibility": "public", "webUrl": "https://gitlab.com/lytor/devops-screening-assignment-rush-deletion_scheduled-85071369", "readmeUrl": "https://gitlab.com/lytor/devops-screening-assignment-rush-deletion_scheduled-85071369/-/blob/main/README.md", "avatarUrl": null, "sshUrl": "git@gitlab.com:lytor/devops-screening-assignment-rush-deletion_scheduled-85071369.git", "httpUrl": "https://gitlab.com/lytor/devops-screening-assignment-rush-deletion_scheduled-85071369.git", "namespacePath": "lytor", "namespaceKind": "user" } ``` ### ⚠️ Run outcomes and error handling This Actor reports what happened in the run's **status message**, and it always keeps whatever it collected. These are the outcomes you can get and what each one means. | Outcome | What it means | |---|---| | **Success** | Rows were returned and you were charged `project-scraped` at $0.001 per row. | | **No matches** | The source returned nothing for your filters. **Nothing is charged.** Widen the date window or drop a filter. | | **Partial - source refused** | The source rate-limited or refused some requests. The affected items are skipped and named in the log, and **everything already collected is still pushed**. A block never discards a run's work. | | **Rejected filter** | The source itself rejected the filter combination. The run fails fast with the source's own reason and **nothing is charged.** | #### What is guaranteed either way - **Every row is pushed as it is built**, not buffered to the end of the run. Anything that buffers output loses everything to a timeout, a block or a migration; this does not. - **A field absent from the source is absent from the row.** Nothing is inferred, modelled or filled in to make a row look complete. # Actor input Schema ## `search` (type: `string`): Search query term to filter GitLab projects by name, description, or namespace path (e.g., devops or cli).

Consequence: Broader searches return more project records, taking longer and consuming more budget.

If empty: Returns public projects without text search constraints.

Leave this field empty to skip this filter. ## `topic` (type: `string`): Filter projects by a specific GitLab topic tag (e.g., kubernetes or machine-learning).

Consequence: Specific topic filters narrow results to targeted technology domains.

If empty: Does not filter projects by topic tag.

Leave this field empty to skip this filter. ## `language` (type: `string`): Filter projects written in a primary programming language (e.g., Python, TypeScript, or Rust).

Consequence: Filters out projects written in other programming languages.

If empty: Includes projects across all programming languages.

Leave this field empty to skip this filter. ## `minStars` (type: `integer`): Minimum number of stars a GitLab project must have received (e.g., 50 or 500).

Consequence: Higher minimum star counts restrict results to established popular repositories.

If empty: No minimum star count constraint is enforced. ## `maxStars` (type: `integer`): Maximum number of stars a GitLab project may have received (e.g., 5000).

Consequence: Restricts results to mid-tier or emerging repositories.

If empty: No maximum star count ceiling is enforced.

The higher the number, the longer the run takes and the more you are charged, because you pay per row returned. ## `visibility` (type: `string`): Visibility level of the project. Options include public, internal, or private.

Consequence: Unauthenticated REST calls access public projects.

If empty: Defaults to fetching public projects. ## `orderBy` (type: `string`): Order projects by a specific field such as star\_count, last\_activity\_at, or created\_at.

Consequence: Controls the sequence of scraped project records.

If empty: Sorts by star\_count by default. ## `sort` (type: `string`): Sort direction for ordered results: desc for descending (highest first) or asc for ascending.

Consequence: Determines whether highest or lowest values appear first.

If empty: Sorts in descending order. ## `maxItems` (type: `integer`): Maximum number of project records to fetch and push to the dataset (between 1 and 5000).

Consequence: Higher limits take longer to execute and consume more event budget.

If empty: Defaults to 50 projects for fast execution.

The higher the number, the longer the run takes and the more you are charged, because you pay per row returned. ## Actor input object example ```json { "search": "devops", "maxItems": 50 } ``` # Actor output Schema ## `dataset` (type: `string`): Default dataset containing scraped GitLab project records # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "search": "devops" }; // Run the Actor and wait for it to finish const run = await client.actor("reapx/gitlab-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "search": "devops" } # Run the Actor and wait for it to finish run = client.actor("reapx/gitlab-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "search": "devops" }' | apify call reapx/gitlab-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,reapx/gitlab-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/7bgtZq7UuTKJH5Pgb/builds/aTrBIP0wwfQDCSPyP/openapi.json