GitHub Repositories Search Scraper

Search and export public GitHub repositories by keyword, language, topic, stars, forks, and recent activity.

Turn GitHub search into a clean dataset for market research, developer relations, open-source intelligence, competitive tracking, and technical lead generation. Results can be downloaded as CSV, JSON, Excel, XML, RSS, or used through the Apify Dataset API.

At a glance

• Repository discovery: Search by product, company, framework, language, topic, or keyword and export repository URLs, names, descriptions, stars, forks, and topics.
• Market mapping: Compare ecosystems, categories, or developer tools using language, license, owner type, and activity fields.
• Competitive tracking: Monitor new or recently updated repositories around a competitor, package, or market category.
• Developer relations: Find projects that mention your SDK, API, package, or integration, then review owner and homepage context.
• API export: Send flat GitHub repository records to spreadsheets, BI tools, enrichment pipelines, or agents.

Ready-to-run examples

Open an example, review the input, and run it as-is or adjust the query:

• Search GitHub Repositories by Keyword: Start with a simple keyword search and inspect the output table.
• Find Popular Python GitHub Repositories: Build a language-specific list sorted by stars.
• Find Recent AI Agent GitHub Repositories: Track recently updated repositories in an active category.
• Monitor Recent GitHub Repositories by Topic: Combine a query with a GitHub topic filter for cleaner monitoring.
• Find React Component Library Repositories: Compare a technical ecosystem by popularity signals.

What can it do?

GitHub Repositories Search Scraper uses public GitHub repository search and returns structured rows you can sort, filter, export, and reuse from the Apify API.

• Search repositories: Enter one or more search queries such as ai agent, web scraping, or language:python stars:>1000.
• Filter results: Add a language and optional GitHub topics when you need a narrower list.
• Sort repositories: Choose best match, stars, forks, or recently updated repositories.
• Export data: Download from Apify as JSON, CSV, Excel, XML, RSS, or stream through the Dataset API.
• Enrich owners: Turn on owner details when public profile context matters for research or outreach.

Common workflows

• Developer relations: Search your-sdk-name or your-api-name to find projects that mention your product, SDK, API, or integration.
• Market research: Search topic:ai stars:>500 or another category query to find high-signal repositories sorted by popularity or freshness.
• Competitive intelligence: Search a competitor package name or framework to map public projects around that tool or market.
• Open-source intelligence: Search a vendor, package, or vulnerability-related keyword to build a monitoring list for follow-up analysis.
• Technical lead generation: Search language:rust stars:>100 or another stack-specific query to discover maintainers, companies, and active projects.
• Content research: Search mcp, agent, automation, or another trend keyword to build lists for newsletters, reports, and trend tracking.

What data can you extract?

The actor returns one dataset row per repository.

Field	Description
query	The input query that produced the repository
fullName	Owner and repository name
htmlUrl	Public GitHub repository URL
description	Repository description
ownerLogin	Repository owner login
ownerType	Owner type, such as User or Organization
stars	Stargazer count
forks	Fork count
watchers	Watcher count
openIssues	Open issue count
language	Primary language
topics	Public GitHub topics
licenseName	Repository license name when available
createdAt	Repository creation date
updatedAt	Repository update date
pushedAt	Last push date
archived	Whether the repository is archived
fork	Whether the repository is a fork
homepage	Public homepage URL when set
cloneUrl	HTTPS clone URL

Optional owner details

Turn on includeOwnerDetails only when owner profile context is worth the extra GitHub API requests.

• Default search: Keep includeOwnerDetails set to false for faster repository search, dashboards, ranking lists, and trend monitoring.
• Owner enrichment: Set includeOwnerDetails to true for developer relations, lead research, owner segmentation, or outreach preparation.
• Extra public owner fields: Enrichment can add public owner name, company, blog, location, public repo count, followers, following, and profile timestamps.
• Larger enriched runs: Add a githubToken secret when owner details or larger searches need higher GitHub API limits.

Pricing

This Actor uses Apify pay-per-event pricing. The prices below come from the current Actor pricing configuration. Apify public plans map to Store discount tiers, so the table shows both the user-facing plan context and the pricing tier name. The final price shown in Apify depends on the user account plan and any custom agreement.

Event	What is charged	Price
start	One-time fee per run	$0.005

Event	What is charged	Free / no discount	Starter / Bronze	Scale / Silver	Business / Gold	Custom / Platinum	Custom / Diamond
result	Per public GitHub repository saved to the dataset	$0.23 / 1,000	$0.2 / 1,000	$0.156 / 1,000	$0.12 / 1,000	$0.08 / 1,000	$0.056 / 1,000

Apify may also charge platform usage for compute, storage, proxies, or data transfer outside this Actor pricing. Check the Actor run and the Apify Pricing tab for the exact cost shown to your account.

Input configuration

Setting	JSON key	Use it for	Example
Search queries	queries	Required list of GitHub search terms or qualifiers.	["ai agent", "language:python stars:>1000"]
Programming language	language	Restrict results to one primary language.	javascript
Topics	topics	Require one or more GitHub topics.	["web-scraping"]
Sort by	sort	Choose best match, stars, forks, or updated repositories.	stars
Sort order	order	Descending or ascending order.	desc
Maximum results	maxResults	Cap saved repository rows and spend.	50
Owner details	includeOwnerDetails	Fetch extra public owner profile fields.	false
GitHub token	githubToken	Optional secret token for higher GitHub API limits.	ghp_...

Example input

{
  "queries": ["apify", "web scraping"],
  "language": "javascript",
  "topics": ["automation"],
  "sort": "stars",
  "order": "desc",
  "maxResults": 50,
  "includeOwnerDetails": false
}

Example output

{
  "query": "apify",
  "name": "API-mega-list",
  "fullName": "cporter202/API-mega-list",
  "htmlUrl": "https://github.com/cporter202/API-mega-list",
  "description": "A public API collection and developer resource list.",
  "ownerLogin": "cporter202",
  "ownerType": "User",
  "stars": 6733,
  "forks": 1294,
  "watchers": 6733,
  "openIssues": 15,
  "language": "JavaScript",
  "topics": ["api", "automation", "web-scraping"],
  "licenseName": null,
  "updatedAt": "2026-06-22T01:14:16Z",
  "pushedAt": "2026-01-27T18:40:57Z",
  "archived": false,
  "fork": false,
  "homepage": null,
  "cloneUrl": "https://github.com/cporter202/API-mega-list.git"
}

How to run it

1. Open the actor on Apify.
2. Add one or more repository search queries.
3. Optionally choose a language and topics.
4. Set the maximum number of repositories.
5. Start the run.
6. Download the dataset or use the API.

Search tips

• Start broad: Try a short keyword such as mcp, ai agent, web scraping, or a product name.
• Narrow by language: Use the language setting or a query qualifier such as language:python.
• Narrow by topic: Add a topic such as web-scraping, llm, automation, or react when you need a cleaner category list.
• Find popular projects: Sort by stars and start with a small maxResults value.
• Find recent activity: Sort by updated or use GitHub qualifiers such as pushed:>2026-01-01.
• Compare competitors: Run multiple queries in one input, then compare results by query, stars, forks, and pushedAt.
• Control spend: Start with maxResults: 25, inspect the table, then scale up.

Handling GitHub limits

• Small public search: Run without a token.
• Larger search: Keep maxResults bounded and split broad searches into focused queries.
• Owner enrichment: Add a GitHub token because owner details require extra API calls.
• Rate limit message: Retry with a smaller run or add a token.
• Token handling: The token is used only for GitHub API authentication during your run.

Integrations

You can connect the dataset to downstream tools.

• Send CSV exports to spreadsheets for market analysis.
• Use Make or Zapier to trigger follow-up enrichment.
• Load JSON results into a warehouse or database.
• Feed repository lists into developer-relations workflows.
• Combine repository URLs with other Apify actors for broader research.

API usage

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('fetch_cat/github-repositories-search-scraper').call({
  queries: ['apify'],
  language: 'javascript',
  maxResults: 25
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient
import os

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('fetch_cat/github-repositories-search-scraper').call(run_input={
    'queries': ['apify'],
    'language': 'javascript',
    'maxResults': 25,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

cURL

curl -X POST 'https://api.apify.com/v2/acts/fetch_cat~github-repositories-search-scraper/runs?token=YOUR_APIFY_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{"queries":["apify"],"language":"javascript","maxResults":25}'

MCP and AI agents

Use this Actor from MCP-compatible tools through the official Apify MCP server. The default server can search and run Actors; the focused URL exposes only this Actor.

MCP URL:

https://mcp.apify.com?tools=fetch_cat/github-repositories-search-scraper

Claude Code setup:

$claude mcp add apify-github-repositories "https://mcp.apify.com?tools=fetch_cat/github-repositories-search-scraper"

Claude Desktop JSON config:

{
  "mcpServers": {
    "apify-github-repositories": {
      "url": "https://mcp.apify.com?tools=fetch_cat/github-repositories-search-scraper"
    }
  }
}

Example prompts:

• "Find 25 JavaScript repositories about Apify and summarize the top owners."
• "Export popular Python repositories about web scraping to a table."
• "Search GitHub repositories for MCP tools and rank them by stars."

Data quality notes

Repository counts and metadata come from public GitHub repository records.

Stars, forks, issues, and timestamps can change over time.

Some repositories do not have a license, homepage, topics, or recent push timestamp.

Owner email is often unavailable because many GitHub users do not publish an email on their profile.

FAQ and troubleshooting

Why did my run return fewer repositories than requested?

The source may have fewer matching public repositories for your exact query, language, and topic filters.

Try removing a topic filter or using a broader query.

Why did I hit a rate limit?

Anonymous GitHub API traffic has stricter limits.

Use a smaller maxResults value or add a GitHub token for higher limits.

Why are some fields empty?

Some public repositories do not provide homepage, license, topics, or owner profile details.

The actor keeps those fields empty instead of guessing.

Legality and responsible use

This actor extracts publicly available GitHub repository data.

Use the data responsibly and follow GitHub's terms, Apify's terms, and applicable laws.

Do not use exported data for spam, harassment, credential collection, or other abusive activity.

If you use owner details for outreach, comply with privacy and anti-spam rules in your jurisdiction.

Related actors

Explore related actors from the same Apify account:

• Stack Exchange Questions Scraper
• Reddit Scraper
• Web Page to Markdown Extractor
• Website Contact Finder
• Google News Scraper

Changelog

• 0.1 - Initial version for public GitHub repository search exports.

Need help?

If a query behaves differently than expected, start with a small maxResults value and inspect the dataset.

Then adjust language, topic, sort, and order settings.

For larger enriched runs, use a GitHub token and keep your first test small.

Privacy and data handling

This Actor only requests the permissions needed to run the input you provide. It uses your input (such as URLs, search terms, identifiers, filters, and limits) only to fetch the requested public data from the relevant source site or API for this Actor, then writes results to your Apify dataset/key-value store.

Data may pass through Apify platform services and Apify Proxy during the run, and requests are sent only to the target site or public data provider required for this Actor's results. FetchCat does not send your inputs or outputs to advertising networks, data brokers, or model-training services, and does not retain run data outside Apify storage after the run except when you explicitly share run details for transient support debugging.

You are responsible for using this Actor lawfully, respecting the target site's terms, and avoiding unnecessary personal or sensitive data in inputs. Review the output before storing, sharing, or combining it with other data.