# GitLab Public Projects Scraper | Stars, Forks, Topics (`parseforge/gitlab-public-projects-scraper`) Actor
Harvest records from multiple Gitlab sources in a single run and get a unified, normalized result set. Pull names, identifiers, dates, descriptions, status flags and source links per record. Perfect for research, lead generation and intelligence pipelines.
- **URL**: https://apify.com/parseforge/gitlab-public-projects-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks
- **User rating**: No ratings yet
## Pricing
Pay per usage
This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## ๐ฆ GitLab Public Projects Scraper
> ๐ **Pull public GitLab projects with stars, forks, topics, and owners in seconds.** Built on the official GitLab REST API.
> ๐ **Last updated:** 2026-05-27 ยท **๐ 25 fields** per record ยท **All public gitlab.com projects** ยท **Search, sort, and topic filters**
The GitLab Public Projects Scraper queries the official `gitlab.com/api/v4/projects` endpoint and returns one normalized record per public project. Useful for tracking open-source DevOps tooling, discovering self-hosted alternatives to GitHub repos, monitoring topic communities (kubernetes, gitops, AI), or building competitive intelligence on the open-source ecosystem.
Coverage: every public project on gitlab.com. Filters by search query, topic, and sort order (stars, last activity, created date). Up to 1,000,000 records per run on the paid plan.
| ๐ฏ Target Audience | ๐ก Primary Use Cases |
|---|---|
| Developer relations | Map ecosystem of related OSS projects |
| Security teams | Find packages by topic or maintainer |
| Recruiters | Spot top GitLab contributors |
| Researchers | Study OSS contribution patterns |
### ๐ What the GitLab Public Projects Scraper does
- Queries the public GitLab REST API directly
- Returns 25 normalized fields per project (name, path, stars, forks, topics, license, owner...)
- Supports search, sort, topic, and ascending/descending order
- Outputs to multiple table outputs via Apify dataset
- Auto-limits to 10 items on the free plan; up to 1,000,000 on paid
> ๐ก **Why it matters:** GitLab hosts millions of projects but no public search UI exposes them at scale. This Actor turns that catalog into a queryable dataset.
### ๐ฌ Full Demo (_๐ง Coming soon_)
### โ๏ธ Input
Field
Type
Description
search
string
Search term, e.g. react
maxItems
integer
Cap on rows (free: 10)
orderBy
enum
star_count, last_activity_at, created_at, etc.
sort
enum
asc / desc
topic
string
Filter by topic, e.g. kubernetes
```json
{ "search": "react", "maxItems": 100, "orderBy": "star_count", "sort": "desc" }
````
```json
{ "topic": "kubernetes", "orderBy": "last_activity_at", "maxItems": 200 }
```
> โ ๏ธ **Good to Know:** Without a token, GitLab's API rate-limits to ~10 requests per second per IP. The Actor paces requests automatically.
### ๐ Output
| Field | Type | Description |
|---|---|---|
| ๐ผ imageUrl | string | Project avatar URL |
| ๐ name | string | Project name |
| ๐ nameWithNamespace | string | Full namespaced name |
| ๐ url | string | gitlab.com web URL |
| ๐ id | integer | GitLab project ID |
| ๐ path | string | URL slug |
| ๐ pathWithNamespace | string | Full path with namespace |
| ๐ description | string | Project description |
| ๐ฟ defaultBranch | string | Default branch name |
| ๐ visibility | string | public / internal |
| โญ starCount | number | Stars |
| ๐ด forksCount | number | Forks |
| ๐ openIssuesCount | number | Open issues |
| ๐ท topics | array | Topics list |
| ๐ท tagList | array | Tags |
| ๐ license | string | License name |
| ๐ createdAt | string | Created ISO timestamp |
| ๐ lastActivityAt | string | Last activity ISO timestamp |
| ๐ readmeUrl | string | README URL |
| ๐ sshUrl | string | SSH clone URL |
| ๐ httpUrl | string | HTTPS clone URL |
| ๐ค owner | string | Namespace name |
| ๐ค ownerPath | string | Namespace path |
| ๐ค ownerKind | string | user / group |
| ๐ scrapedAt | string | ISO timestamp |
| โ error | string | null | Error message if extraction failed |
### โจ Why choose this Actor
- Direct GitLab REST API, no third-party caching
- Real-time data, never stale
- Pay-per-result pricing; only charged for what you keep
- Works with Make, Zapier, n8n, Airbyte, GitHub Actions, Google Sheets
### ๐ How it compares to alternatives
| Approach | Cost | Maintenance | Coverage |
|---|---|---|---|
| GitLab UI search | Free | Manual | One page at a time |
| Self-built API client | Dev cost | High | Custom |
| This actor | Pay per result | None | Full search + filters |
### ๐ How to use
1. [Create a free Apify account w/ $5 credit](https://console.apify.com/sign-up?fpr=vmoqkp)
2. Open the GitLab Public Projects Scraper actor page
3. Set `search`, `orderBy`, optional `topic`, and `maxItems`
4. Click **Start** and wait for the run to finish
5. Use the dataset as multiple table outputs
### ๐ผ Business use cases
#### Developer relations
| Need | How this Actor helps |
|---|---|
| Ecosystem mapping | Pull all projects by topic |
| Influencer tracking | Sort by stars to find top owners |
#### Security
| Need | How this Actor helps |
|---|---|
| Supply chain audit | Scan public projects in your topic |
| License compliance | Filter by license field |
#### Recruiting
| Need | How this Actor helps |
|---|---|
| Find OSS contributors | Sort by activity, drill into owner |
| Skill mapping | Filter by topic, language |
#### Research
| Need | How this Actor helps |
|---|---|
| OSS trends | Bulk pull over time |
| Comparative analysis | GitLab vs GitHub trends |
### ๐ Automating GitLab Public Projects Scraper
Run on a schedule, forward results to Make, Zapier, n8n, Slack, Airbyte, GitHub Actions, or Google Drive. Push new high-star projects into a Slack channel daily.
### ๐ Beyond business use cases
#### Research
Build a longitudinal study of OSS topic growth over months/years.
#### Personal
Discover obscure self-hosted alternatives to your favourite SaaS tools.
#### Non-profit
Track open civic-tech projects by topic (e.g. `civic-tech`, `accessibility`).
#### Experimentation
Build a "GitLab radar" Slack bot for newly active projects in your space.
### ๐ค Ask an AI assistant about this scraper
- [ChatGPT](https://chatgpt.com/)
- [Claude](https://claude.ai/)
- [Perplexity](https://www.perplexity.ai/)
- [Microsoft Copilot](https://copilot.microsoft.com/)
### โ Frequently Asked Questions
**Q: Does it need a GitLab token?** No. Public endpoints work anonymously.
**Q: Can I scrape private projects?** No - only `visibility: public` is returned.
**Q: How fresh is the data?** Live - every request hits gitlab.com.
**Q: Can I filter by language?** Use the `topic` field - many projects tag languages (e.g. `python`, `rust`).
**Q: What's the rate limit?** ~10 RPS anonymous; the Actor paces well under it.
**Q: Can I scrape self-hosted GitLab?** This Actor targets gitlab.com specifically. Self-hosted instances would need a forked Actor.
**Q: Does it include archived projects?** Yes - check `archived` field if needed.
**Q: How do I find a topic slug?** Browse `gitlab.com/explore/projects/topics`.
**Q: Can I sort by forks?** `orderBy` supports `id`, `name`, `path`, `created_at`, `updated_at`, `last_activity_at`, `star_count`. Forks not supported by upstream API.
**Q: Does it return README contents?** No - just `readmeUrl`. Fetch separately if needed.
### ๐ Integrate with any app
Slack, Discord, Sheets, Airtable, BigQuery, S3, Snowflake, and 100+ more via Apify webhooks.
### ๐ Recommended Actors
| Actor | What it does |
|---|---|
| GitHub Trending Scraper | Daily trending repos |
| Hacker News Scraper | Top tech stories |
| npm Packages Scraper | npm metadata |
| Mastodon Trends Scraper | Fediverse trends |
> ๐ก **Pro Tip:** browse the complete [ParseForge collection](https://apify.com/parseforge).
**๐ Need Help?** [Open our contact form](https://tally.so/r/BzdKgA)
> **โ ๏ธ Disclaimer:** independent tool, not affiliated with GitLab Inc. Only publicly available data is collected.
# Actor input Schema
## `search` (type: `string`):
Search term (project name / description). Leave blank to list newest public projects.
## `maxItems` (type: `integer`):
Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000
## `orderBy` (type: `string`):
Field to sort results by.
## `sort` (type: `string`):
Sort direction (asc / desc).
## `topic` (type: `string`):
Filter projects by topic (e.g. 'kubernetes'). Optional.
## Actor input object example
```json
{
"search": "react",
"maxItems": 10,
"orderBy": "star_count",
"sort": "desc",
"topic": ""
}
```
# Actor output Schema
## `results` (type: `string`):
No description
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"maxItems": 10
};
// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/gitlab-public-projects-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`๐พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// ๐ Want to learn more ๐? Go to โ https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = { "maxItems": 10 }
# Run the Actor and wait for it to finish
run = client.actor("parseforge/gitlab-public-projects-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("๐พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# ๐ Want to learn more ๐? Go to โ https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"maxItems": 10
}' |
apify call parseforge/gitlab-public-projects-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=parseforge/gitlab-public-projects-scraper",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "GitLab Public Projects Scraper | Stars, Forks, Topics",
"description": "Harvest records from multiple Gitlab sources in a single run and get a unified, normalized result set. Pull names, identifiers, dates, descriptions, status flags and source links per record. Perfect for research, lead generation and intelligence pipelines.",
"version": "0.1",
"x-build-id": "K6w8x6Wfw3KpIell6"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/parseforge~gitlab-public-projects-scraper/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-parseforge-gitlab-public-projects-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/parseforge~gitlab-public-projects-scraper/runs": {
"post": {
"operationId": "runs-sync-parseforge-gitlab-public-projects-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/parseforge~gitlab-public-projects-scraper/run-sync": {
"post": {
"operationId": "run-sync-parseforge-gitlab-public-projects-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"properties": {
"search": {
"title": "Search Query",
"type": "string",
"description": "Search term (project name / description). Leave blank to list newest public projects.",
"default": "react"
},
"maxItems": {
"title": "Max Items",
"minimum": 1,
"maximum": 1000000,
"type": "integer",
"description": "Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000"
},
"orderBy": {
"title": "Order By",
"enum": [
"id",
"name",
"path",
"created_at",
"updated_at",
"last_activity_at",
"star_count"
],
"type": "string",
"description": "Field to sort results by.",
"default": "star_count"
},
"sort": {
"title": "Sort Direction",
"enum": [
"asc",
"desc"
],
"type": "string",
"description": "Sort direction (asc / desc).",
"default": "desc"
},
"topic": {
"title": "Topic Filter",
"type": "string",
"description": "Filter projects by topic (e.g. 'kubernetes'). Optional.",
"default": ""
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```