# MCP Reliability Inspector (`vamsi-krishna/mcp-reliability-inspector`) Actor Lint a public TypeScript or JavaScript MCP server on GitHub for tool-definition defects, invalid input schemas, and baseline regressions. Static analysis only: the Actor reads source files and returns findings, a full report, and a baseline for the next run. - **URL**: https://apify.com/vamsi-krishna/mcp-reliability-inspector.md - **Developed by:** [Vamsi Krishna](https://apify.com/vamsi-krishna) (community) - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.00005 / actor start This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README MCP Reliability Inspector is an MCP server linter for public GitHub repositories. It reads a TypeScript or JavaScript Model Context Protocol server, checks each tool definition and `inputSchema`, and compares the contract with a saved baseline. The scan is static analysis. Babel walks the source as a syntax tree. Tool handlers, install scripts, and dependencies stay unexecuted. A finished run gives you a findings table, a full JSON report, and a baseline file for the next scan. You can run it on Apify, then call the same input from the Apify API, a schedule, or an integration. ### What does MCP Reliability Inspector do? - **Tool definitions.** Names must be present, unique, and within the MCP naming rules. Descriptions must exist and be long enough for an agent to choose the tool. - **Input schemas.** Each `inputSchema` is checked as JSON Schema. Required fields must exist in `properties`. Schemas built by a function call or a variable are reported as dynamic. - **Baseline diff.** A previous inventory flags removed tools, schema changes, description changes, and new tools. - **Coverage notes.** Repositories with no recognized MCP registration, and detections without a line number, are reported separately from defects. Confirmed defects are deterministic findings at high or medium severity. Info results and coverage notes describe the scan. They are quality or coverage signals. ### Use cases - **Before you publish an MCP server.** Catch empty tool names, duplicate names, missing descriptions, and broken input schemas on the commit you are about to tag. - **CI regression check.** Save the `BASELINE` record from a good commit and pass it on the next run. Removed tools and schema changes show up as baseline findings. - **Agent-readiness review.** MCP clients pick tools from the name, description, and schema. This scan lists the definitions an agent would have to guess. ### How to scan a GitHub MCP server 1. Open the Actor and start a run. The repository field is prefilled with `modelcontextprotocol/servers`. Replace it with the `owner/repo` you want. 2. Leave the default checks on, or add **Baseline diff** and paste a previous baseline. 3. Run the Actor. Apify stores the log, and you can schedule the same input or call it from the Apify API. 4. Open the **Findings** view for one row per issue. Open the **Summary** view for status and severity counts. 5. Save the `BASELINE` record from the run output if the next scan should detect contract changes. ### Input ```json { "repository": "modelcontextprotocol/servers", "ref": "main", "checks": ["tool-definitions", "input-schemas"], "includeSourceExcerpts": true, "limits": { "maxFiles": 5000, "maxFileBytes": 500000, "maxTotalSourceBytes": 25000000 } } ``` | Field | Required | Default | Purpose | | --- | --- | --- | --- | | `repository` | Yes | Prefill `modelcontextprotocol/servers` | Public GitHub repo as `owner/repo` or a `https://github.com/owner/repo` URL. | | `ref` | No | Default branch | Branch name, tag, or full commit SHA. The report records the commit that was scanned. | | `checks` | No | `tool-definitions`, `input-schemas` | `tool-definitions` checks names and descriptions. `input-schemas` checks JSON Schema. `baseline-diff` compares with `baseline`. | | `baseline` | No | Empty | Baseline inventory from an earlier `BASELINE` record. On Apify, also select **Baseline diff** in Checks. | | `includeSourceExcerpts` | No | `true` | Include a few source lines on each finding. | | `limits.maxFiles` | No | `5000` | Maximum candidate files (1–10000). | | `limits.maxFileBytes` | No | `500000` | Maximum bytes per file. | | `limits.maxTotalSourceBytes` | No | `25000000` | Maximum source bytes across the scan. | #### Baseline inventory Paste the JSON stored under the `BASELINE` key. ```json { "scannerVersion": "0.1.0", "commitSha": "0123456789abcdef0123456789abcdef01234567", "createdAt": "2026-10-23T12:00:12.000Z", "tools": [ { "inventoryKey": "0123456789abcdef0123456789abcdef", "name": "get_weather", "description": "Fetch the current weather for a city.", "schemaHash": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "schemaNormalized": "{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}}}", "path": "src/index.ts" } ] } ``` `description`, `schemaHash`, and `schemaNormalized` are `null` when the tool has no description or no static schema. ### Output A run writes three results. The run's `output` object links to each one. - **Findings dataset.** One `summary` record, then one `finding` record per issue. Filter and export this table. - **`OUTPUT` key.** The full report: coverage, summary, every finding, and the inventory. A failed run stores an error report here instead. - **`BASELINE` key.** The inventory to pass as `baseline` on a later run. Present after a successful scan. #### Example report `OUTPUT` after a scan that found a missing tool description: ```json { "schemaVersion": "1.0.0", "scannerVersion": "0.1.0", "repository": { "fullName": "modelcontextprotocol/servers", "ref": "main", "commitSha": "0123456789abcdef0123456789abcdef01234567", "source": "github-public" }, "status": "completed_with_findings", "startedAt": "2026-10-23T12:00:00.000Z", "completedAt": "2026-10-23T12:00:12.000Z", "coverage": { "filesConsidered": 42, "filesParsed": 40, "filesSkipped": 2, "toolDefinitionsDetected": 6, "unsupportedPatterns": 0, "limitReached": false }, "summary": { "high": 1, "medium": 0, "low": 0, "info": 0, "confirmedDefects": 1, "heuristics": 0, "coverageNotes": 0 }, "findings": [ { "id": "tool-004-a1b2c3d4e5f67890", "ruleId": "TOOL-004", "severity": "high", "confidence": "high", "kind": "deterministic", "toolName": "get_weather", "location": { "path": "src/index.ts", "startLine": 12, "endLine": 16 }, "title": "Tool description is missing or empty", "evidence": { "message": "Tool \"get_weather\" has no description." }, "recommendation": "Add a description that explains what the tool does and when to call it.", "references": ["https://modelcontextprotocol.io/specification/2025-11-25/server/tools"] } ] } ``` `status` is `completed`, `completed_with_findings`, `partial`, `unsupported`, or `failed`. `completed_with_findings` means at least one high or medium deterministic finding. `partial` means a file cap stopped the scan early. `unsupported` means no recognized tool registration was found. The same object includes `inventory` when the scan finishes. That object is also stored on its own under `BASELINE`. Dataset records use the same finding object, wrapped for the table: ```json { "type": "summary", "report": { "status": "completed_with_findings", "summary": { "high": 1 } } } ``` ```json { "type": "finding", "finding": { "ruleId": "TOOL-004", "severity": "high", "toolName": "get_weather", "title": "Tool description is missing or empty" } } ``` #### Example failure A failed run stores this shape in `OUTPUT` and pushes one `summary` dataset record. `error.code` is `INPUT_INVALID`, `REPO_NOT_FOUND`, `GITHUB_RATE_LIMIT`, `LIMIT_EXCEEDED`, `UNSUPPORTED_PATTERN`, or `INTERNAL_ERROR`. ```json { "schemaVersion": "1.0.0", "scannerVersion": "0.1.0", "status": "failed", "error": { "code": "LIMIT_EXCEEDED", "message": "The scan exceeded a configured limit.", "diagnosticId": "a1b2c3d4e5f6" }, "startedAt": "2026-10-23T12:00:00.000Z", "completedAt": "2026-10-23T12:00:04.000Z" } ``` #### For AI agents Call the Actor with `repository` set to `owner/repo`. The default checks are `tool-definitions` and `input-schemas`. Add `baseline-diff` and a `baseline` object only when you have a previous `BASELINE` record. Read issues from the findings URL on the run `output`. Read status, coverage, and the inventory from the `OUTPUT` URL. Pass the `BASELINE` URL's JSON back as `baseline` on the next call. Treat `status: "failed"` together with `error.code` as the failure contract. Count a defect when `kind` is `deterministic` and `severity` is `high` or `medium`. Info findings and `kind: "coverage"` are coverage or quality notes. `summary.confirmedDefects` counts deterministic findings. `summary.coverageNotes` counts coverage notes. ### Rules Info severity and coverage notes are separate from confirmed defects. #### Tool definitions | Rule | Severity | What it reports | | --- | --- | --- | | `TOOL-001` | high | Tool name is missing or empty. | | `TOOL-002` | medium | Tool name is outside the MCP rules: 1–64 characters, only letters, digits, `_`, and `-`. | | `TOOL-003` | high | Two tools in the same server share a name. | | `TOOL-004` | high | Description is missing or blank. | | `TOOL-005` | low | Description is under 10 characters. This is a heuristic. | #### Input schemas | Rule | Severity | What it reports | | --- | --- | --- | | `SCHEMA-001` | high | `inputSchema` is missing or null. | | `SCHEMA-002` | high | The schema object fails JSON Schema validation. | | `SCHEMA-003` | medium | A `required` entry is missing from `properties`. | | `SCHEMA-004` | info | The schema is a variable, call, or spread, so it cannot be checked statically. | #### Baseline diff | Rule | Severity | What it reports | | --- | --- | --- | | `BASELINE-001` | high | A tool from the baseline is gone. | | `BASELINE-002` | medium | The tool's input schema changed. | | `BASELINE-003` | low | The tool's description changed. | | `BASELINE-004` | info | A tool was added. | #### Coverage | Rule | Severity | What it reports | | --- | --- | --- | | `REPO-001` | info | No recognized MCP tool registration in the files that were scanned. | | `SOURCE-001` | info | A tool was detected and the parser could not attach a line number. | ### Supported MCP patterns The scanner recognizes registrations from `@modelcontextprotocol/sdk` and static tool arrays in TypeScript or JavaScript. `McpServer.tool()` positional form: ```ts server.tool("get_weather", "Fetch weather", { city: z.string() }, handler); server.tool("ping", handler); ``` `McpServer.tool()` object form: ```ts server.tool({ name: "query", description: "Run query", inputSchema: { type: "object", properties: {} } }); ``` Low-level `Server.setRequestHandler()`: ```ts server.setRequestHandler(ListToolsRequestSchema, async () => ({ tools: [{ name: "my_tool", description: "Do the thing", inputSchema: { type: "object", properties: {} } }], })); ``` A static tool array: ```ts const tools: Tool[] = [ { name: "calculator", description: "Math operations", inputSchema: { type: "object", properties: {} } }, ]; ``` ### Limitations - The repository must be public on GitHub. The scan reads the git tree and file contents through the GitHub API. - Source files must be TypeScript or JavaScript. - Recognized registrations are the SDK patterns above and static tool arrays. Other frameworks show up as `REPO-001` when nothing matches. - A schema built at runtime is reported as `SCHEMA-004`. The scan records that the schema was dynamic. - Defaults cap the scan at 5000 files, 500000 bytes per file, and 25000000 source bytes in total. Past a cap, `coverage.limitReached` is true and `status` is `partial`. ### Pricing A run has two event charges, and platform usage is billed as well. The Pricing tab on the Actor page lists the current rates. - **Actor start:** $0.00005 when the run starts. One event covers up to 1 GB of memory, and each extra GB adds one event. The default memory is 1 GB, so a normal scan is one Actor start event. The compute cost of the first 5 seconds is waived. - **Result:** $0.00001 for each row in the default dataset, which is $0.01 per 1,000 rows. A scan writes one summary row and one row per finding. - **Platform usage:** compute for the rest of the run, at the rate of your Apify plan. Higher plans have a lower compute rate. ### FAQ #### Does this run the MCP server? The Actor parses source text. It uses the Babel parser and never calls the tool handlers, `eval`, `child_process`, or `npm install`. #### Which repos qualify? Public GitHub repositories that contain a TypeScript or JavaScript MCP server. Pass `owner/repo` or the canonical GitHub URL. Private repositories are outside this Actor. #### What is a baseline? A baseline is the tool inventory from an earlier successful scan: tool name, description, schema hash, and file path, pinned to a commit SHA. Save the `BASELINE` key and pass that JSON as `baseline` with the `baseline-diff` check. The next run reports added, removed, and changed tools. #### Why REPO-001? `REPO-001` means the scanned files had no registration this Actor recognizes. The repository can still contain an MCP server written in another language, or tools built only through dynamic code. Treat it as a coverage note. #### How should an API or AI agent read the result? Pass `repository`. Read the findings dataset URL from the run `output` for one record per issue. Read `OUTPUT` for `status`, `summary`, and `error.code` when the run failed. Use `summary.confirmedDefects` for deterministic findings. Leave info and coverage records out of a failure threshold. Schedule the same input, or call the run endpoint, when this check should repeat. # Changelog This Actor's version history is a separate document: https://apify.com/vamsi-krishna/mcp-reliability-inspector/changelog.md # Actor input Schema ## `repository` (type: `string`): Public GitHub repository to scan. Accepts owner/repo or a canonical github.com URL (for example modelcontextprotocol/servers or https://github.com/modelcontextprotocol/servers). ## `ref` (type: `string`): Git ref to scan. Accepts a branch name, a tag, or a full commit SHA. Leave empty to use the repository's default branch. The report records the commit that was actually read. ## `checks` (type: `array`): Which check categories to enable. Tool names and descriptions (tool-definitions) checks MCP tool names and descriptions. JSON Schema contracts (input-schemas) validates each inputSchema. Baseline diff (baseline-diff) compares the scan with the baseline object. On Apify, select Baseline diff when you paste a baseline. The form sends this list, so a baseline alone does not turn the diff on. ## `includeSourceExcerpts` (type: `boolean`): Include a few source lines around each finding so the result points at the tool definition. ## `baseline` (type: `object`): Optional inventory from a previous run's BASELINE key. Select Baseline diff in Checks when you set this. Shape: { scannerVersion, commitSha, createdAt, tools: \[{ inventoryKey, name, description, schemaHash, schemaNormalized, path }] }. description, schemaHash, and schemaNormalized may be null. See Baseline inventory in the README. ## `limits` (type: `object`): Caps for the GitHub scan. When a cap is hit, coverage.limitReached is true and the report status is partial. ## Actor input object example ```json { "repository": "modelcontextprotocol/servers", "ref": "", "checks": [ "tool-definitions", "input-schemas" ], "includeSourceExcerpts": true, "baseline": null, "limits": { "maxFiles": 5000, "maxFileBytes": 500000, "maxTotalSourceBytes": 25000000 } } ``` # Actor output Schema ## `findings` (type: `string`): Default dataset items. The first item is a summary record (type "summary"). Each following item is one finding (type "finding") with the rule, severity, tool name, and source location. ## `report` (type: `string`): Key-value store record OUTPUT. On success this is the InspectorReport (schemaVersion 1.0.0) including coverage, summary, findings, and inventory. On failure this is an ErrorReport with status "failed" and error.code. ## `baseline` (type: `string`): Key-value store record BASELINE. JSON inventory of the tools found in this scan. Pass this object as the baseline input, with the baseline-diff check, on a later run. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "repository": "modelcontextprotocol/servers" }; // Run the Actor and wait for it to finish const run = await client.actor("vamsi-krishna/mcp-reliability-inspector").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "repository": "modelcontextprotocol/servers" } # Run the Actor and wait for it to finish run = client.actor("vamsi-krishna/mcp-reliability-inspector").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "repository": "modelcontextprotocol/servers" }' | apify call vamsi-krishna/mcp-reliability-inspector --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,vamsi-krishna/mcp-reliability-inspector" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/8U2b7p0kui7Mttdbq/builds/d9ozdljdhPHUFvJTM/openapi.json