# GitHub Projects Scraper (`zippy_vector/github-projects-scraper`) Actor Discover, score, and rank GitHub repositories by quality, stack fit, and reusable feature value. Top Benefits Stack-aware discovery: filters repos by your real stack (React, PostgreSQL, TypeScript, etc.) Actionable ranking: each repo gets score + compatibility + feature intelligence - **URL**: https://apify.com/zippy\_vector/github-projects-scraper.md - **Developed by:** [Algirdas Kolesnikovas](https://apify.com/zippy_vector) (community) - **Categories:** Lead generation, Agents, Automation - **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing from $0.01 / actor start This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### GitHub Projects Scraper — Apify Actor **Purpose:** This actor searches GitHub repositories using the GitHub Search API, enriches each repository with additional metadata (README, topics, languages, contributors, activity), scores build potential (0–100), and saves the best candidates to an Apify dataset. #### Publish Description (Apify Store) Discover high-potential GitHub repositories you can build on. This Actor searches GitHub by your criteria, enriches each repo with README/topic/language/activity metadata, scores it, and highlights: - `stackCompatibility` (how well it matches your tech stack) - `stealableFeatures` (concrete features worth reusing) - `topFeature` (highest-value detected capability) Use it for startup validation, boilerplate discovery, competitor research, and feature inspiration. #### Credentials / Security - **Required:** `githubToken` (GitHub Personal Access Token) - The Actor does **not** work without a valid token. - Never hardcode tokens in code, README, or committed files. - If a token was ever shared in chat/screenshots/logs, revoke it and create a new one. #### Input The input schema is defined in `.actor/input_schema.json`. Key fields: - **githubToken** (string, required): GitHub Personal Access Token with repo read access. - **searchQueries** (string[]): Search queries, e.g. `["ai agent", "saas boilerplate"]`. - **languages** (string[]): Language filters, e.g. `["TypeScript", "Python"]`. - **minStars** (integer): Minimum stars (default: 50). - **maxStars** (integer): Maximum stars (default: 50000). - **minForks** (integer): Minimum forks (default: 5). - **pushedAfter** (string, date): Only repos with last commit after this date (`YYYY-MM-DD`). Default: 1 year ago. - **excludeTopics** (string[]): Topics to exclude, e.g. `["deprecated", "abandoned"]`. - **includeTopics** (string[]): Topics that slightly boost score, e.g. `["boilerplate", "starter", "template", "saas"]`. - **maxResults** (integer): Maximum number of projects to save (default: 100). - **scoreThreshold** (number): Minimum total score (0–100) (default: 10). - **techStack** (string[]): Your technologies for compatibility scoring. - **stackMatchThreshold** (integer): Minimum stack compatibility score (0–100, default: 30). - **outputDataset** (boolean): Save results to dataset (default: true). #### Running locally ```bash npm install apify run --input='{ "githubToken": "YOUR_TOKEN", "searchQueries": ["saas boilerplate", "ai agent"], "languages": ["TypeScript", "Python"], "minStars": 50, "maxStars": 50000, "maxResults": 50, "scoreThreshold": 60 }' ```` #### Output dataset Each item in the dataset has (at least) the following structure: - `id`, `name`, `fullName`, `url`, `description` - `stars`, `forks`, `watchers`, `openIssues`, `language`, `languages` - `topics`, `license`, `createdAt`, `updatedAt`, `pushedAt` - `score` (0–100) and `scoreBreakdown` with per-criterion scores - `readme` metadata (exists, wordCount, hasSections, preview) - `contributors` (top 5) - `buildPotential`: `"HIGH" | "MEDIUM" | "LOW"` - `suggestedUseCases`: string\[] - `stackCompatibility`: `{ score, compatibility, matched, missing }` - `stealableFeatures`: ranked feature list with confidence/signals/steal tips - `topFeature`: highest value extracted feature or null #### Notes - All GitHub API calls use a small retry + exponential backoff strategy. - Rate limiting errors (403/429) are retried a few times before failing the individual request. - Invalid tokens (`401`) stop the run with a clear error. - Missing repos (`404`) are skipped but logged in `debugging.md` as needed. # Actor input Schema ## `githubToken` (type: `string`): GitHub PAT with at least repo read permissions. Used for authenticated GitHub REST/Search API calls. ## `searchQueries` (type: `array`): GitHub search queries, e.g. \['ai agent', 'saas boilerplate']. Combined with language and stars filters. ## `languages` (type: `array`): Optional language filters, e.g. \['TypeScript', 'Python']. If empty, all languages are allowed. ## `minStars` (type: `integer`): Minimum GitHub stars to include a repository. ## `maxStars` (type: `integer`): Maximum GitHub stars (to avoid overly popular repos). ## `minForks` (type: `integer`): Minimum number of forks. ## `pushedAfter` (type: `string`): Only include repositories with the last commit after this date. Format: YYYY-MM-DD. Default is one year ago. ## `excludeTopics` (type: `array`): GitHub topics to avoid, e.g. \['abandoned', 'deprecated']. ## `includeTopics` (type: `array`): Topics that increase a project's score, e.g. \['boilerplate', 'starter', 'template', 'saas']. ## `maxResults` (type: `integer`): How many projects to collect in total across all queries. ## `scoreThreshold` (type: `number`): Only include projects with a total score >= this threshold (0–100). ## `deepAnalysis` (type: `boolean`): If true, fetches extra GitHub stats (commits, releases, issues, contributors). If false, collects only basic repo + README + topics + languages for faster runs. ## `maxSearchPagesPerQuery` (type: `integer`): Limit how many GitHub Search pages to process per query/language (1 page = 30 repos). Lower number = faster run. ## `techStack` (type: `array`): Specify technologies you use. Example: \['React', 'PostgreSQL', 'TypeScript', 'Docker'] ## `stackMatchThreshold` (type: `integer`): Minimum compatibility percentage to keep the repo (0-100) ## `outputDataset` (type: `boolean`): If true, results are stored to the default Apify dataset. ## `llmProvider` (type: `string`): Select LLM provider for analysis. 'none' = no AI analysis. ## `llmApiKey` (type: `string`): API key for the selected provider ## `llmModel` (type: `string`): e.g. gpt-4o, claude-opus-4-5, gemini-1.5-pro. If empty, a default model is used. ## `llmAnalysisDepth` (type: `string`): quick = only summary, standard = full analysis, deep = multi-LLM consensus ## `secondaryLlmProvider` (type: `string`): Used only when llmAnalysisDepth='deep' ## `secondaryLlmApiKey` (type: `string`): Used only when llmAnalysisDepth='deep' ## `secondaryLlmModel` (type: `string`): e.g. claude-opus-4-5, gemini-1.5-pro. If empty, a default model is used. ## `userContext` (type: `string`): Describe yourself: technologies, budget, experience. Example: 'Solo developer, I know TypeScript and React, have $500 for marketing, looking for a SaaS project I can launch in 3 months.' ## `generatePdfReport` (type: `boolean`): Generate an HTML → PDF report with all insights ## `reportEmail` (type: `string`): If provided, the PDF report will be sent to this email address ## `userProductIdea` (type: `string`): Describe what you want to build. Example: 'A SaaS tool that automatically generates SEO copy from a product description.' ## Actor input object example ```json { "searchQueries": [ "saas boilerplate", "starter template" ], "languages": [ "TypeScript", "Python" ], "minStars": 50, "maxStars": 50000, "minForks": 5, "pushedAfter": "", "excludeTopics": [ "deprecated", "abandoned" ], "includeTopics": [ "boilerplate", "starter", "template", "saas" ], "maxResults": 100, "scoreThreshold": 10, "deepAnalysis": true, "maxSearchPagesPerQuery": 3, "techStack": [], "stackMatchThreshold": 30, "outputDataset": true, "llmProvider": "none", "llmAnalysisDepth": "standard", "secondaryLlmProvider": "none", "generatePdfReport": false } ``` # Actor output Schema ## `datasetItem` (type: `string`): One enriched and scored GitHub repository record pushed to the default dataset. ## `RUN_SUMMARY` (type: `string`): Key-value output with run counters, skip/error diagnostics, and effective input parameters. ## `EXECUTIVE_SUMMARY` (type: `string`): LLM generated executive summary text. ## `REPORT_HTML` (type: `string`): HTML report string (also stored as REPORT\_HTML in KeyValueStore). ## `REPORT_PDF_URL` (type: `string`): URL to the generated PDF report. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("zippy_vector/github-projects-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("zippy_vector/github-projects-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call zippy_vector/github-projects-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=zippy_vector/github-projects-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "GitHub Projects Scraper", "description": "Discover, score, and rank GitHub repositories by quality, stack fit, and reusable feature value.\nTop Benefits\nStack-aware discovery: filters repos by your real stack (React, PostgreSQL, TypeScript, etc.)\nActionable ranking: each repo gets score + compatibility + feature intelligence", "version": "1.0", "x-build-id": "eTiPL0Mcg4MPUU9VM" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/zippy_vector~github-projects-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-zippy_vector-github-projects-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/zippy_vector~github-projects-scraper/runs": { "post": { "operationId": "runs-sync-zippy_vector-github-projects-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/zippy_vector~github-projects-scraper/run-sync": { "post": { "operationId": "run-sync-zippy_vector-github-projects-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "githubToken" ], "properties": { "githubToken": { "title": "GitHub Personal Access Token", "minLength": 1, "type": "string", "description": "GitHub PAT with at least repo read permissions. Used for authenticated GitHub REST/Search API calls." }, "searchQueries": { "title": "Search queries", "type": "array", "description": "GitHub search queries, e.g. ['ai agent', 'saas boilerplate']. Combined with language and stars filters.", "items": { "type": "string" }, "default": [ "saas boilerplate", "starter template" ] }, "languages": { "title": "Languages filter", "type": "array", "description": "Optional language filters, e.g. ['TypeScript', 'Python']. If empty, all languages are allowed.", "items": { "type": "string" }, "default": [ "TypeScript", "Python" ] }, "minStars": { "title": "Minimum stars", "type": "integer", "description": "Minimum GitHub stars to include a repository.", "default": 50 }, "maxStars": { "title": "Maximum stars", "type": "integer", "description": "Maximum GitHub stars (to avoid overly popular repos).", "default": 50000 }, "minForks": { "title": "Minimum forks", "type": "integer", "description": "Minimum number of forks.", "default": 5 }, "pushedAfter": { "title": "Pushed after (date)", "type": "string", "description": "Only include repositories with the last commit after this date. Format: YYYY-MM-DD. Default is one year ago.", "default": "" }, "excludeTopics": { "title": "Exclude topics", "type": "array", "description": "GitHub topics to avoid, e.g. ['abandoned', 'deprecated'].", "items": { "type": "string" }, "default": [ "deprecated", "abandoned" ] }, "includeTopics": { "title": "Preferred topics", "type": "array", "description": "Topics that increase a project's score, e.g. ['boilerplate', 'starter', 'template', 'saas'].", "items": { "type": "string" }, "default": [ "boilerplate", "starter", "template", "saas" ] }, "maxResults": { "title": "Maximum projects", "type": "integer", "description": "How many projects to collect in total across all queries.", "default": 100 }, "scoreThreshold": { "title": "Minimum score threshold", "minimum": 0, "maximum": 100, "type": "number", "description": "Only include projects with a total score >= this threshold (0–100).", "default": 10 }, "deepAnalysis": { "title": "Deep analysis (slower)", "type": "boolean", "description": "If true, fetches extra GitHub stats (commits, releases, issues, contributors). If false, collects only basic repo + README + topics + languages for faster runs.", "default": true }, "maxSearchPagesPerQuery": { "title": "Max search pages per query", "minimum": 1, "maximum": 34, "type": "integer", "description": "Limit how many GitHub Search pages to process per query/language (1 page = 30 repos). Lower number = faster run.", "default": 3 }, "techStack": { "title": "Your Tech Stack", "type": "array", "description": "Specify technologies you use. Example: ['React', 'PostgreSQL', 'TypeScript', 'Docker']", "items": { "type": "string" }, "default": [] }, "stackMatchThreshold": { "title": "Min stack match %", "minimum": 0, "maximum": 100, "type": "integer", "description": "Minimum compatibility percentage to keep the repo (0-100)", "default": 30 }, "outputDataset": { "title": "Save to dataset", "type": "boolean", "description": "If true, results are stored to the default Apify dataset.", "default": true }, "llmProvider": { "title": "LLM Provider", "enum": [ "openai", "anthropic", "gemini", "none" ], "type": "string", "description": "Select LLM provider for analysis. 'none' = no AI analysis.", "default": "none" }, "llmApiKey": { "title": "LLM API Key", "type": "string", "description": "API key for the selected provider" }, "llmModel": { "title": "LLM Model (optional)", "type": "string", "description": "e.g. gpt-4o, claude-opus-4-5, gemini-1.5-pro. If empty, a default model is used." }, "llmAnalysisDepth": { "title": "Analysis depth", "enum": [ "quick", "standard", "deep" ], "type": "string", "description": "quick = only summary, standard = full analysis, deep = multi-LLM consensus", "default": "standard" }, "secondaryLlmProvider": { "title": "Secondary LLM Provider (Multi-consensus)", "enum": [ "openai", "anthropic", "gemini", "none" ], "type": "string", "description": "Used only when llmAnalysisDepth='deep'", "default": "none" }, "secondaryLlmApiKey": { "title": "Secondary LLM API Key (optional)", "type": "string", "description": "Used only when llmAnalysisDepth='deep'" }, "secondaryLlmModel": { "title": "Secondary LLM Model (optional)", "type": "string", "description": "e.g. claude-opus-4-5, gemini-1.5-pro. If empty, a default model is used." }, "userContext": { "title": "Your context (for personalized analysis)", "type": "string", "description": "Describe yourself: technologies, budget, experience. Example: 'Solo developer, I know TypeScript and React, have $500 for marketing, looking for a SaaS project I can launch in 3 months.'" }, "generatePdfReport": { "title": "Generate PDF report", "type": "boolean", "description": "Generate an HTML → PDF report with all insights", "default": false }, "reportEmail": { "title": "Report email address", "type": "string", "description": "If provided, the PDF report will be sent to this email address" }, "userProductIdea": { "title": "Your product idea (for competitor analysis)", "type": "string", "description": "Describe what you want to build. Example: 'A SaaS tool that automatically generates SEO copy from a product description.'" } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```