# Codebase Intelligence Pack (`wheat_tourist/codebase-intelligence-pack`) Actor

Analyze any GitHub repo or ZIP archive and produce 11 AI-ready intelligence files: architecture, API routes, auth patterns, DB schema, env vars, unused files, tech stack, and more.

- **URL**: https://apify.com/wheat\_tourist/codebase-intelligence-pack.md
- **Developed by:** [Varun Chopra](https://apify.com/wheat_tourist) (community)
- **Categories:** AI, Developer tools, Agents
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $20.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Codebase Intelligence Pack

**Stop wasting tokens exploring repositories.**

This Apify Actor analyzes any GitHub repo, ZIP archive, or uploaded codebase and produces **11 structured intelligence files** that let AI coding assistants (Claude Code, Codex, Cursor, Copilot) instantly understand a project — without reading 20,000 files.

***

### ⚡ Quick Start

```bash
## Run via Apify CLI
apify run -i '{"sourceType":"github","githubUrl":"https://github.com/vercel/next.js"}'
```

Or configure in the [Apify Console](https://console.apify.com) with the visual input form.

***

### 📥 Input Options

#### Option 1 — GitHub Repository URL (recommended)

```json
{
  "sourceType": "github",
  "githubUrl": "https://github.com/owner/repo",
  "githubToken": "ghp_...",
  "branch": "main"
}
```

> **GitHub Token:** Supports both classic Personal Access Tokens (PAT) and fine-grained organization tokens with `repo:read` scope. Required only for private repositories.

#### Option 2 — ZIP Archive URL

```json
{
  "sourceType": "zipUrl",
  "zipUrl": "https://example.com/myproject.zip"
}
```

#### Option 3 — Pre-uploaded ZIP (for local repos)

**Step 1:** Upload your ZIP to the Actor's KV store

```bash
## Get your store ID from https://console.apify.com/storage/key-value-stores
STORE_ID="your-kv-store-id"
APIFY_TOKEN="your-apify-token"

curl -X PUT \
  "https://api.apify.com/v2/key-value-stores/${STORE_ID}/records/MY_REPO_ZIP" \
  -H "Content-Type: application/zip" \
  --data-binary @/path/to/your/repo.zip \
  -G --data-urlencode "token=${APIFY_TOKEN}"
```

**Step 2:** Run the Actor pointing at that key

```json
{
  "sourceType": "uploadedZip",
  "uploadedZipKey": "MY_REPO_ZIP"
}
```

> **How to export a local repository as ZIP:**
>
> ```bash
> # Git archive (clean — no node_modules, no .git)
> git archive --format=zip HEAD -o repo.zip
>
> # Or using zip directly
> zip -r repo.zip . -x "node_modules/*" ".git/*" "__pycache__/*"
> ```

***

### 📤 Outputs

All 11 files are stored in the **default Key-Value Store** of the Actor run.
Access them via the Apify API:

```
https://api.apify.com/v2/key-value-stores/{storeId}/records/{filename}
```

| File | Description |
|------|-------------|
| `architecture.json` | High-level architecture: layers, communication patterns, summary |
| `entrypoints.json` | Main entry files, npm scripts, Dockerfiles, CI configs |
| `dependencies.json` | Runtime/dev deps, internal import graph, circular deps |
| `api-routes.json` | All HTTP routes with method, path, file, line, framework |
| `database.json` | ORM, schema files, models, migration dir, connection pattern |
| `authentication.json` | Auth strategies, providers, protected routes, token location |
| `unused-files.json` | Files not reachable from any entry point (BFS graph analysis) |
| `tech-stack.json` | Languages, frameworks, bundler, testing, DevOps tools |
| `environment.json` | All env vars: documented vs undocumented, required vs optional |
| `folder-summary.md` | Per-directory descriptions with key files and metadata |
| `code-map.md` | Annotated ASCII file tree with entry/route/auth/DB markers |

A **summary row** is also pushed to the Dataset with key metrics.

***

### 🤖 Optional LLM Enrichment

By default, this Actor uses **pure static analysis** (no API keys required).

To enable AI-generated natural language descriptions in `folder-summary.md` and `code-map.md`:

```json
{
  "sourceType": "github",
  "githubUrl": "https://github.com/owner/repo",
  "llmProvider": "openai",
  "llmApiKey": "sk-...",
  "llmModel": "gpt-4o-mini"
}
```

**Supported LLM providers:**

| Provider | Default Model |
|----------|---------------|
| `openai` | `gpt-4o-mini` |
| `google` | `gemini-1.5-flash` |
| `anthropic` | `claude-3-haiku-20240307` |

***

### 🔍 Supported Frameworks

#### API Route Detection

Express, Fastify, Koa, FastAPI, Flask, Django, NestJS, Next.js (App Router + Pages),
Gin, Echo, Fiber, Laravel, Spring Boot, Rails

#### Auth Pattern Detection

JWT, OAuth2, Passport.js, NextAuth, Firebase Auth, Supabase Auth, Sessions, bcrypt/argon2, API Keys

#### ORM / Database Detection

Prisma, TypeORM, Sequelize, Mongoose, SQLAlchemy, Django ORM, GORM, ActiveRecord,
Hibernate, Drizzle, Knex + drivers (pg, mysql2, sqlite3, redis, MongoDB)

#### Package Managers

npm, yarn, pnpm, bun, pip, poetry, cargo, go mod

***

### 🏗️ Project Structure

```
src/
├── __main__.py              # Actor entry point
├── downloader/
│   ├── github_downloader.py # GitHub ZIP download via GitHub API
│   ├── zip_downloader.py    # Generic ZIP URL download
│   └── upload_reader.py     # KV store ZIP reader
├── walker/
│   ├── file_walker.py       # File tree traversal + FileTree object
│   └── ignore_patterns.py   # Always-ignored dirs/extensions/files
├── analyzers/
│   ├── tech_stack.py        # Language + framework detection
│   ├── entry_points.py      # Main entry files, scripts, CI
│   ├── api_routes.py        # HTTP route extraction (12+ frameworks)
│   ├── authentication.py    # Auth strategy detection
│   ├── database.py          # ORM, schema, model detection
│   ├── environment.py       # Env var collection + documentation check
│   ├── dependencies.py      # Package deps + internal import graph
│   ├── unused_files.py      # BFS reachability analysis
│   └── architecture.py      # Architecture synthesis
└── reporters/
    ├── folder_summary.py    # folder-summary.md generator
    ├── code_map.py          # code-map.md generator
    └── output_writer.py     # KV store writer + LLM client factory
```

***

### 🛠️ Local Development

```bash
## Install dependencies
pip install -r requirements.txt

## Run locally with Apify CLI
apify run -i input.json

## Example input.json
{
  "sourceType": "github",
  "githubUrl": "https://github.com/tiangolo/fastapi"
}
```

***

### ⚙️ Advanced Input Options

| Field | Default | Description |
|-------|---------|-------------|
| `maxFileSizeKb` | `500` | Skip files larger than N KB |
| `maxFiles` | `10000` | Process at most N files |
| `includePatterns` | `[]` | Only include files matching globs |
| `excludePatterns` | `[]` | Skip files matching globs |
| `outputFormat` | `"all"` | `"all"` / `"json-only"` / `"markdown-only"` |

***

### 📄 License

MIT

# Actor input Schema

## `sourceType` (type: `string`):

Where to load the repository from.

## `githubUrl` (type: `string`):

Full URL to the GitHub repository. E.g. https://github.com/expressjs/express

## `githubToken` (type: `string`):

Personal Access Token (classic) or fine-grained org token with `repo:read` scope. Required for private repositories. Both personal PAT and organization-level tokens are supported.

## `branch` (type: `string`):

Branch, tag, or commit SHA to analyze. Defaults to the repository's default branch.

## `zipUrl` (type: `string`):

Direct URL to a ZIP archive of the repository. Must be publicly accessible.

## `uploadedZipKey` (type: `string`):

Key in the Actor's default Key-Value Store where the ZIP was pre-uploaded. See README for upload instructions.

## `maxFileSizeKb` (type: `integer`):

Files larger than this will be skipped. Default: 500 KB.

## `maxFiles` (type: `integer`):

Cap on total number of files to process. Default: 10000.

## `includePatterns` (type: `array`):

Only analyze files matching these glob patterns. If empty, all files are included.

## `excludePatterns` (type: `array`):

Skip files matching these patterns. Common directories (node\_modules, .git, **pycache**) are always excluded.

## `outputFormat` (type: `string`):

Which output files to generate.

## `llmProvider` (type: `string`):

When set, folder-summary.md and code-map.md include AI-generated natural language descriptions. Leave empty for pure static analysis (no API key needed).

## `llmApiKey` (type: `string`):

API key for the chosen LLM provider. Only required when an LLM provider is selected.

## `llmModel` (type: `string`):

Specific model to use. Defaults: OpenAI=gpt-4o-mini, Google=gemini-1.5-flash, Anthropic=claude-3-haiku-20240307.

## Actor input object example

```json
{
  "sourceType": "github",
  "githubUrl": "https://github.com/expressjs/express",
  "branch": "main",
  "zipUrl": "https://example.com/myrepo.zip",
  "uploadedZipKey": "MY_REPO_ZIP",
  "maxFileSizeKb": 500,
  "maxFiles": 10000,
  "includePatterns": [
    "src/**",
    "*.py"
  ],
  "excludePatterns": [
    "tests/**",
    "docs/**"
  ],
  "outputFormat": "all",
  "llmProvider": "",
  "llmModel": "gpt-4o"
}
```

# Actor output Schema

## `summaryDataset` (type: `string`):

Dataset containing high-level summary metrics of the analyzed repository

## `folderSummary` (type: `string`):

Human-readable Markdown summary of repository structure and folder roles

## `codeMap` (type: `string`):

Annotated ASCII file tree with entry points, routes, auth, and database tags

## `architecture` (type: `string`):

Architectural breakdown, layers, patterns, and communication protocols

## `entrypoints` (type: `string`):

Main entry files, server startup scripts, Dockerfiles, and CI workflows

## `apiRoutes` (type: `string`):

Detected HTTP routes, endpoints, methods, and line numbers

## `database` (type: `string`):

Database engines, ORMs, schemas, models, and connection string patterns

## `authentication` (type: `string`):

Detected auth strategies, providers, protected routes, and session handlers

## `dependencies` (type: `string`):

External runtime/dev dependencies and internal module import graph

## `unusedFiles` (type: `string`):

Unreachable source files, dead modules, and orphaned media assets

## `techStack` (type: `string`):

Comprehensive list of languages, frameworks, bundlers, and testing tools

## `environment` (type: `string`):

Referenced environment variables, templates, and undocumented vars

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("wheat_tourist/codebase-intelligence-pack").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("wheat_tourist/codebase-intelligence-pack").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call wheat_tourist/codebase-intelligence-pack --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,wheat_tourist/codebase-intelligence-pack"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/rOfLZyGQeQfVYQkBO/builds/DwICh2qGad4yeu4IU/openapi.json