Self-hosted AI code reviewer for GitHub pull requests

The idea

A webhook-driven workflow that runs a local LLM over every GitHub pull request diff before a human reviewer opens the tab. When a PR is opened or updated, GitHub fires a webhook to your n8n instance. A workflow fetches the changed files, sends each one to an Ollama model, and assembles the responses into a single review comment posted back to the PR via the GitHub API.

No external AI API. No monthly usage bill. The review runs on hardware you control.

Why build this

AI-assisted code review is useful for small teams and solo developers who want a second pass before merging, but the commercial options are expensive at scale. GitHub Copilot Code Review, CodeRabbit, and Sourcegraph Cody all make sense for teams with a budget; for a solo developer or a small team running self-hosted infrastructure, paying per-user per-month for a review bot is hard to justify.

The local alternative works. Modern code models — DeepSeek-Coder-V2 at 16B parameters, Qwen2.5-Coder at 7B — catch real bugs, flag missing error handling, and spot suspicious patterns in most languages. Latency is higher than a cloud API but acceptable: a 200-line diff reviewed in under 30 seconds fits the feedback loop a PR review serves.

The workflow requires nothing exotic. n8n's webhook trigger and HTTP Request nodes handle orchestration. The Ollama API is a single POST endpoint. Anyone already running the n8n + Ollama stack can ship a working version in an afternoon.

Stack sketch

• Workflow engine: n8n (self-hosted via Docker Compose); a Webhook trigger node receives the GitHub PR event payload
• LLM inference: Ollama running qwen2.5-coder:7b — fits in 8 GB VRAM; swap in deepseek-coder-v2:16b if you have 16 GB
• Diff fetching: n8n HTTP Request node calling GET /repos/{owner}/{repo}/pulls/{pull_number}/files with a GitHub personal access token in the Authorization header
• Prompt assembly: n8n Code node (JavaScript) that iterates changed files, skips binary and oversized files, and builds a prompt: file name, extension as a language hint, and the raw unified diff
• Comment posting: n8n HTTP Request node calling POST /repos/{owner}/{repo}/issues/{pull_number}/comments with the assembled review in Markdown
• Auth: a fine-grained GitHub personal access token with pull_requests:write and contents:read scopes, stored as an n8n credential

Scope for v1

• Trigger on PR opened and PR synchronized events; ignore all other event types
• Fetch changed files via the GitHub REST API; skip binary files, files over 300 lines, and any path matching a configurable skip list
• Send each in-scope file as a separate Ollama request with a fixed system prompt: "You are a careful code reviewer. Point out bugs, missing error handling, and security issues. Be concise."
• Concatenate per-file responses into one comment prefixed with a bot disclaimer; post to the PR via the Issues comment API
• Skip list defaults: *.lock, *.min.js, dist/, vendor/, migrations/
• Deliberately out of scope for v1: inline diff-anchored comments (requires diff-position math and the Reviews API), approve/request-changes decisions, re-review triggered by a reply, tracking which files were already reviewed on a previous push

Where it could go

Inline comments anchored to specific diff lines are far more actionable than a wall of text at the bottom of the thread. The GitHub Reviews API supports them via POST /repos/{owner}/{repo}/pulls/{pull_number}/reviews with a comments array keyed to diff positions. Adding this requires parsing the unified diff to compute position offsets — straightforward with a small JavaScript parser — and is the single change most likely to make the output feel like a real reviewer rather than a summary dump.

The second expansion worth pursuing is a feedback loop. A thumbs-up or thumbs-down reaction on the bot's comment triggers another n8n webhook, which appends the interaction to a local log with the diff excerpt and the model's output. After a few hundred PRs you have a labeled dataset of useful vs. noisy reviews. Even without fine-tuning, the hit-rate log gives you concrete evidence for evaluating whether switching models — or tightening the system prompt — improves quality over time.

Watch out for

Diffs that include generated files, vendored dependencies, or schema migrations will send thousands of tokens to Ollama per review and produce noise-heavy output. The skip list is critical, not optional — an unconfigured reviewer that reviews package-lock.json on every PR burns through goodwill faster than it builds it. Default the skip list to sensible values and document it clearly at the top of the workflow.