GitHub Repository → Markdown Converter

This Apify actor converts multiple GitHub repositories into clean, structured Markdown optimized for use with large language models (LLMs). It fetches files from GitHub repositories (optionally filtered by branch, extensions, or glob patterns), processes the content, and outputs Markdown suitable for embeddings, fine-tuning, or context augmentation.

Use this actor to transform codebases into LLM-ready documentation, research corpora, or preparation material for model pretraining or retrieval augmentation. Process single repositories or batch multiple repositories efficiently in one run.

Input Parameters

The actor accepts the following input parameters as a JSON object:

Parameter	Type	Default	Description
repositories	Array	Required	Array of repository objects to process. Must contain at least one repository.

Repository Object Properties

Each repository object in the repositories array supports the following properties:

Parameter	Type	Default	Description
source	String	Required	The GitHub repository URL to convert (e.g. https://github.com/facebook/react).
branch	String|Null	null	Optional branch or tag name to process. Defaults to the repository's default branch.
extensions	Array|Null	null	File extensions to include when converting to Markdown (e.g. [".js", ".ts"]).
maxTokens	Integer|Null	null	Optional maximum token limit for the generated Markdown. Useful for chunking or limiting output.
maxFiles	Integer|Null	null	Maximum number of files to process within the repo.
includeFiles	Array|Null	null	Glob patterns specifying files to include (e.g. ["src/**"]).
excludeFiles	Array|Null	null	Glob patterns specifying files to exclude (e.g. ["**/*.test.js"]).

Example Input

Multiple Repositories

{
    "repositories": [
        {
            "source": "https://github.com/facebook/react",
            "branch": "main",
            "extensions": [".js", ".jsx", ".ts", ".tsx"],
            "maxTokens": 100000,
            "maxFiles": 250,
            "includeFiles": ["packages/react/src/**"],
            "excludeFiles": ["**/*.test.js", "**/*.md"]
        },
        {
            "source": "https://github.com/vercel/next.js",
            "branch": "canary",
            "extensions": [".js", ".ts", ".tsx"],
            "maxTokens": 150000,
            "maxFiles": 300,
            "includeFiles": ["packages/next/src/**"],
            "excludeFiles": ["**/*.test.js", "**/*.spec.js"]
        }
    ]
}

Single Repository

{
    "repositories": [
        {
            "source": "https://github.com/facebook/react",
            "branch": "main",
            "extensions": [".js", ".jsx", ".ts", ".tsx"],
            "maxTokens": 200000
        }
    ]
}

Example Output

{
	"repositoryIndex": 0,
	"repositoryUrl": "https://github.com/facebook/react",
	"result": "<MARKDOWN CONTENT>"
}

Use Cases

The GitHub Repo → Markdown Converter can be used in multiple scenarios, such as:

• LLM Training Preparation
  Convert multiple repositories into token-friendly Markdown for fine-tuning or embeddings.

• Documentation Generation
  Produce readable markdown documents from source code across multiple projects.

• Research & Analysis
  Analyze and compare multiple repositories in LLM workflows by converting them into structured text.

• Knowledge Base Construction
  Build RAG (Retrieval-Augmented Generation) datasets from multiple live repositories in a single run.

• Codebase Summarization & Understanding
  Provide LLMs with high-quality, normalized code inputs from multiple projects for better comparative model reasoning.

• Batch Processing
  Process multiple related repositories (e.g., microservices, related libraries) efficiently in a single Actor run.

Related Actors

• GitHub Profile Scraper - Extract comprehensive GitHub user profile information
• GitHub Repository Scraper - Scrape detailed repository metadata and statistics

🌟 Explore More Actors

✨ Need more scraping solutions? Discover additional actors on Apify for comprehensive web automation and data extraction. Explore our full range of tools at 🌐 Explore More Actors on Apify.

📧 For inquiries or custom development, reach out at apify@vulnv.com.