GitLab Scraper — Projects, Issues, Merge Requests & User Repos

Scrape public GitLab repositories, issues, merge requests, and user project lists at scale — no API key, no login, no credentials required.

What does GitLab Scraper do?

GitLab Scraper hits the official GitLab REST API v4 at https://gitlab.com/api/v4/ using only public, unauthenticated endpoints. It supports four modes:

• projects — search public projects by keyword, sorted by star count, last activity, or creation date. Paginates through hundreds to thousands of results per run.
• issues — fetch open issues for one or more project IDs or paths, with full metadata including author, labels, and timestamps.
• mergeRequests — fetch open merge requests for given projects, including state, author, and merge timestamps.
• userProjects — list all public repositories for a specific GitLab username.

The actor uses got-scraping for resilient HTTP requests with automatic retries, respects GitLab's pagination headers, and gracefully handles rate limits and transient errors. Each run can return 200–5,000 rows depending on your maxResults setting.

Who is it for?

• Developers and engineers researching the GitLab ecosystem for libraries, frameworks, or tooling by keyword and star count.
• Data scientists and analysts building datasets of open-source projects for trend analysis, NLP, or academic research.
• DevRel and marketing teams monitoring GitLab activity, tracking competitor repos, or identifying top contributors.
• Recruiters and talent scouts finding active open-source developers on GitLab by username and project activity.
• Security researchers scanning public repositories for specific technologies, frameworks, or known-vulnerable patterns at scale.

Use cases

• Extract the top 1,000 most-starred public GitLab projects tagged "kubernetes" for a competitive analysis dashboard.
• Fetch all open issues from a list of infrastructure repos to build a centralized bug tracker feed.
• Monitor merge request activity on GitLab's own open-source repos to track release cycles.
• List all public repos for a specific GitLab user as part of a developer profile enrichment pipeline.
• Build a dataset of GitLab projects by topic for a machine learning classifier that predicts project quality.

Why use GitLab Scraper?

• No API key needed: works entirely with GitLab's public unauthenticated API — zero setup, zero cost for credentials.
• 14 output fields per project: id, name, path, namespace, description, stars, forks, open issues, last activity, URL, topics, visibility, created date, default branch.
• Bulk pagination: fetches up to 5,000 results per run by iterating through all API pages automatically.
• 4 modes in one actor: projects search, issues, merge requests, and user projects — no need for separate scrapers.
• Export to CSV, JSON, Excel: all output formats available natively in Apify datasets.
• Pay-per-result: only pay for what you extract — Apify's compute-unit pricing scales linearly with data volume.

What data can you extract?

The actor outputs structured records for projects, issues, and merge requests. Project records include:

Field	Type	Description
type	string	Record type: project, issue, or merge_request
id	string	GitLab numeric ID of the project
name	string	Project name
path	string	Full path with namespace (e.g. gitlab-org/gitlab)
namespace	string	Owner namespace (user or group path)
description	string	Project description text
starCount	number	Number of stars
forksCount	number	Number of forks
openIssuesCount	number	Count of open issues
lastActivityAt	string	ISO 8601 timestamp of last activity
webUrl	string	Full URL to the project on GitLab.com
topics	string	Comma-separated list of topics/tags
visibility	string	public, internal, or private
createdAt	string	ISO 8601 project creation timestamp
defaultBranch	string	Default branch name (e.g. main, master)

Issue and merge request records add:

Field	Type	Description
projectId	string	Parent project's numeric ID
projectPath	string	Parent project path identifier
number	number	Internal issue/MR number (iid)
title	string	Issue or MR title
state	string	opened, closed, merged
author	string	GitLab username of the author
labels	string	Comma-separated labels
createdAtItem	string	ISO 8601 creation timestamp
updatedAt	string	ISO 8601 last update timestamp
closedAt	string	ISO 8601 close/merge timestamp

Sample output (project):

{
  "type": "project",
  "id": "278964",
  "name": "GitLab",
  "path": "gitlab-org/gitlab",
  "namespace": "gitlab-org",
  "description": "GitLab is an open source end-to-end software development platform with built-in version control, issue tracking, code review, CI/CD, and more.",
  "starCount": 23400,
  "forksCount": 6890,
  "openIssuesCount": 42015,
  "lastActivityAt": "2026-07-08T08:00:00.000Z",
  "webUrl": "https://gitlab.com/gitlab-org/gitlab",
  "topics": "DevOps, CI/CD, Ruby, Go",
  "visibility": "public",
  "createdAt": "2017-01-01T09:00:00.000Z",
  "defaultBranch": "master"
}

How to use

Option A — Search projects by keyword

Use mode: "projects" with a search query to find relevant public repositories sorted by star count.

1. Open the actor on Apify and go to Input.
2. Set mode to projects.
3. Enter a query like kubernetes, rust web framework, or machine learning.
4. Set maxResults (default 200, up to 5000).
5. Click Start and wait for results.

{
  "mode": "projects",
  "query": "kubernetes",
  "orderBy": "star_count",
  "maxResults": 500,
  "proxyConfiguration": { "useApifyProxy": true }
}

Option B — Fetch issues or MRs by project ID

Use mode: "issues" or mode: "mergeRequests" with a list of project IDs or paths to fetch activity data.

1. Find the project ID from the GitLab UI (Settings > General) or use the path like gitlab-org/gitlab.
2. Set projectIds to a list of IDs or paths.
3. Set maxResults to control total items per run.

{
  "mode": "issues",
  "projectIds": ["gitlab-org/gitlab", "278964"],
  "maxResults": 1000,
  "proxyConfiguration": { "useApifyProxy": true }
}

Option C — List all projects for a user

Use mode: "userProjects" to fetch all public repos for a GitLab username.

{
  "mode": "userProjects",
  "username": "torvalds",
  "maxResults": 200,
  "proxyConfiguration": { "useApifyProxy": true }
}

Input parameters

Parameter	Type	Default	Description
mode	string	projects	Scraping mode: projects, issues, mergeRequests, userProjects
query	string	—	Keyword search for projects mode
projectIds	array	—	List of project IDs or paths for issues/MR modes
username	string	—	GitLab username for userProjects mode
maxResults	integer	200	Max items to return (1–5000)
orderBy	string	star_count	Sort field for projects: star_count, last_activity_at, created_at, id, name
proxyConfiguration	object	Apify datacenter	Proxy settings

Full input JSON:

{
  "mode": "projects",
  "query": "react",
  "projectIds": [],
  "username": "",
  "maxResults": 300,
  "orderBy": "star_count",
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [] }
}

Output example

{
  "type": "project",
  "id": "12584701",
  "name": "StackGres",
  "path": "ongresinc/stackgres",
  "namespace": "ongresinc",
  "description": "StackGres Operator, Full Stack PostgreSQL on Kubernetes",
  "starCount": 140,
  "forksCount": 63,
  "openIssuesCount": 45,
  "lastActivityAt": "2026-07-07T21:19:05.216Z",
  "webUrl": "https://gitlab.com/ongresinc/stackgres",
  "topics": "PgBouncer, envoy, java, k8s, kubernetes, operator, patroni, postgresql",
  "visibility": "public",
  "createdAt": "2019-05-29T11:46:15.548Z",
  "defaultBranch": "main"
}

Tips for best results

• Use specific queries: "kubernetes operator" returns more targeted results than "k8s".
• Paginate large datasets: set maxResults to 2000–5000 for comprehensive coverage of a topic.
• Combine modes: run projects first to get IDs, then use issues mode with those IDs in a follow-up run.
• Sort by last_activity_at to find the most recently updated repos instead of the most popular.
• Use path strings: you can pass "gitlab-org/gitlab" directly in projectIds — the actor handles URL encoding.
• Export to Google Sheets: use Apify's built-in integration to pipe results into a live spreadsheet.
• Schedule regularly: set up a weekly schedule for competitive monitoring of rival projects' issue velocity.
• Filter post-export: download as JSON and filter by starCount > 500 or topics containing specific frameworks.
• Rate limits: GitLab public API is rate-limited to ~60 req/min unauthenticated. For very large runs (>2000 results), the actor handles this gracefully with retries.
• Proxy not required: GitLab's public API doesn't block datacenter IPs — proxy is optional but helps stability.

Integrations

Google Sheets: Connect Apify to Google Sheets via the native integration. New project records appear automatically as rows — ideal for team collaboration dashboards.

Slack: Use Apify webhooks to send a Slack message whenever a run completes. Combine with a schedule for daily repo monitoring alerts.

Zapier / Make: Trigger Zapier or Make workflows from Apify dataset events. For example: new GitLab project scraped → add to CRM → notify sales team.

Webhooks: POST run results to any endpoint via Apify's webhook system. Use this to pipe data into Elasticsearch, BigQuery, or your own database.

Schedule: Run on a daily or weekly cron in Apify Scheduler. Monitor new repositories in your niche or track issue counts over time without manual intervention.

API usage

cURL:

curl -X POST \
  "https://api.apify.com/v2/acts/logiover~gitlab-scraper/runs?token=YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode":"projects","query":"kubernetes","maxResults":200}'

Node.js (Apify client):

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('logiover/gitlab-scraper').call({
  mode: 'projects',
  query: 'rust web framework',
  maxResults: 500,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python:

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")

run = client.actor("logiover/gitlab-scraper").call(run_input={
    "mode": "projects",
    "query": "machine learning",
    "maxResults": 300,
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["name"], item["starCount"])

Use with AI agents (MCP)

GitLab Scraper is compatible with the Apify MCP (Model Context Protocol) server, which lets AI assistants like Claude or GPT-4 call this actor directly as a tool. Ask your AI agent: "Use the GitLab Scraper to find the top 100 most-starred public projects tagged 'security' and summarize the most active ones" — the agent calls the actor, retrieves structured data, and processes it inline.

FAQ

Does GitLab Scraper require an API key?

No. It uses only public GitLab API endpoints at https://gitlab.com/api/v4/ which are accessible without authentication. You do not need a GitLab account or Personal Access Token.

What is the coverage? Does it include private repos?

Only public repositories are accessible. GitLab's unauthenticated API returns all projects with visibility: public. Private and internal projects are not included.

What is the maximum number of results I can get?

You can set maxResults up to 5,000 per run. For projects mode, GitLab's API has deep pagination support. Practical limits depend on search result availability — a broad query like "react" may have thousands of matching repos.

Why am I getting zero results?

• Check your query for typos or overly specific terms with no matches.
• For issues/mergeRequests mode, verify your projectIds are valid public project IDs or paths.
• For userProjects mode, ensure the username is correct and the user has public repos.

Some fields are null — is that expected?

Yes. Fields like openIssuesCount may be null if a project has issues tracking disabled. topics is null if the project has no tags. These are natural data gaps in GitLab's API response.

How do I export the data to Excel?

In the Apify dataset view, click Export and choose XLSX. For large datasets, use JSON or CSV which handle more rows without Excel's row limit.

How fast does it run?

The actor fetches 100 records per API call. For 1,000 records, expect roughly 10–20 seconds of runtime. For 5,000, around 1–2 minutes depending on API response times and proxy latency.

Is this legal? Can I scrape GitLab?

Yes. The actor only accesses GitLab's official public REST API at documented endpoints. This is equivalent to any developer querying the API in their own application. No login bypasses, no scraping of private data. See GitLab's ToS for usage policies at https://about.gitlab.com/terms/.

What other scrapers are related?

• GitHub Repository Scraper — scrape GitHub repos, issues, and contributors.
• Docker Hub Scraper — scrape Docker Hub images and tags.
• npm Package Intelligence Scraper — scrape npm for package metadata, downloads, and dependencies.

How often is the data updated?

The actor fetches live data from GitLab's API on every run. There is no caching — each run returns the current state of projects and issues at the time of execution.

Can I fetch merge requests that are already merged or closed?

Currently the actor fetches state=opened items by default. You can modify the actor's baseUrl if you deploy a custom version, or request a new input parameter via the actor's GitHub issues.

Is it legal?

This actor accesses only publicly available data through GitLab's official documented API. No authentication bypass, no private data, no terms-of-service violation. All extracted data is the same data accessible to any user visiting GitLab.com or making direct API calls. You are responsible for how you use the extracted data. Always review the platform's Terms of Service at https://about.gitlab.com/terms/ and applicable privacy laws (GDPR, CCPA) before using scraped data for commercial purposes.

Related scrapers

• GitHub Repository Scraper — mirror tool for GitHub.com with stars, forks, contributors, and repo metadata.
• npm Package Intelligence Scraper — extract npm registry data including weekly downloads, dependencies, and maintainer info.
• Docker Hub Scraper — scrape Docker Hub for images, tags, pull counts, and descriptions.
• Crates.io Scraper — Rust package registry data including crate metadata, versions, and download stats.