AI PR Reviewer

Reviews any public GitHub pull request and returns a structured Markdown review with summary, risks, suggestions, and a confidence score. Cites file:line for every finding. Optional one-click "post the review as a PR comment" mode (idempotent — same bot comment is edited on re-runs, never spammed).

What it does in one run

1. Takes a GitHub PR URL as input
2. Fetches the PR metadata + full diff (paginated)
3. Runs deterministic heuristics against each added line:
   • 🚨 hardcoded secrets (AWS / OpenAI / GitHub token literals)
   • ⚠️ unsafe patterns (eval(), shell=True, dangerouslySetInnerHTML, DROP TABLE, DELETE FROM without WHERE)
   • 🟡 size / testing (huge diffs, source changes without test changes)
   • 🔵 debug leftovers (console.log, print(), debugger)
   • ℹ️ TODO/FIXME markers, empty PR descriptions
4. (Optional) Hands the heuristic findings + diff to Claude for a polished narrative summary
5. Computes a principled confidence score (size penalty + critical/high deductions + empty-description penalty)
6. Pushes a structured record to the default dataset
7. (Optional) Upserts the rendered Markdown as a single bot comment on the PR — same comment edited in place on every re-run, no comment spam

Input

{
  "prUrl": "https://github.com/owner/repo/pull/123",     // required
  "mode": "hybrid",                                       // hybrid | heuristic | llm
  "anthropicApiKey": "sk-ant-...",                        // optional, required for hybrid/llm
  "model": "claude-sonnet-4-20250514",                    // sonnet | opus | haiku
  "githubToken": "ghp_...",                               // optional, raises rate limits and enables postComment
  "postComment": false                                    // upsert review as a PR comment
}

Mode comparison:

Mode	API key needed	Cost	Output style
heuristic	None	Free (Apify compute only)	Deterministic, file:line cited findings, principled confidence
hybrid (default)	Anthropic	Cheap (prompt-cached system prompt)	Heuristic findings + LLM narrative summary
llm	Anthropic	Slightly higher	Pure LLM (skip heuristics)

If you pick hybrid without an API key the Actor falls back to heuristic automatically — never fails.

Output

Dataset record:

{
  "pr": "https://github.com/owner/repo/pull/123",
  "title": "...",
  "author": "...",
  "stats": { "files": 4, "additions": 666, "deletions": 0 },
  "mode": "hybrid",
  "confidence": { "score": 0.65, "label": "Medium" },
  "findings": [
    { "severity": "critical", "category": "security", "message": "...", "file": "src/x.ts", "line": 42 }
  ],
  "markdown": "<!-- claude-review-bot:do-not-remove -->\n## AI PR Review — ..."
}

The markdown field is the rendered review you'd paste into Slack / Discord / wherever. Use postComment: true to skip the copy-paste.

Try it free first

The default input (https://github.com/psf/requests/pull/7401) is a small, real merged PR. Click Run with default settings and the heuristic-mode review completes in <10 seconds and is free (Apify compute is included in your free tier).

Pricing

This Actor uses Pay-Per-Event pricing — you pay only when a review completes. Specifics are listed on the Actor's detail page on the Apify Store.

(For the developer of this Actor: the SDK fires Actor.charge('review-completed') exactly once per completed review. Configure the unit price for the review-completed event in Console → Actor → Publication → Pricing.)

Bring your own API keys

Both the GitHub token and the Anthropic key are passed via the input, not stored by the Actor. They live only in the run logs and are masked in the UI. For long-term automation, use the Actor's "Schedule" feature and set the keys once as Actor secrets so they don't appear in every run config.

Common questions

Does this train on my code? No. The Anthropic API call is made from your run only, with no logging beyond Apify's standard run output. Toggle mode: heuristic to avoid any LLM call.

Will it work on private PRs? Yes if you supply a githubToken with repo scope on the host repository.

Can I run it on a schedule? Yes. Use Apify's Schedules feature and point it at a PR URL pattern, or run it from a webhook trigger when PRs open.

What languages does the heuristic mode cover? Python, TypeScript, JavaScript, Go, Rust, Java, Ruby, PHP, C# for source/test detection. Regex patterns (secrets, SQL, dangerous calls) are language-agnostic.

Roadmap

Open to feedback — message me via the Apify Store contact. Likely next additions:

• GitLab / Bitbucket support
• Custom rule packs (your team's "we don't do X" rules as YAML)
• Inline GitHub review comments (line-by-line) instead of a single summary comment
• OpenAI / Mistral backends as alternatives to Claude