7 min
AI code review that follows your coding rules: how to test it
A reviewer that loads your rules file is not a reviewer that enforces it. Two instruction-following studies give you three tests that tell the two apart.
Category · 16 of 37 articles
Guides
How to run an evaluation, a trial or a rollout without wasting a quarter on it.
7 min
Martian's Code Review Bench: the "#1" claim rots in months
Two vendors posted "#1 on Martian's Code Review Bench." Both still improved their scores. Both lost first place. How to read the live leaderboard without getting burned.
6 min
Self-Hosted AI Code Review Tools: What to Verify Before You Deploy
Self-hosting an AI code reviewer moves the data, not the risk. Here is what to verify in the harness before you point it at a real repo, with the receipts.
6 min
AI code review's absence blindness: measure what it misses
Teams reviewing a growing volume of AI-generated code need to score what a reviewer fails to notice, not only what it flags. A measurable protocol.
6 min
Verify your AI reviewer actually follows your team's rules
A canary test, block tests, and independent judging to prove an AI code reviewer actually follows your team's coding standards, not just claims to.
5 min
Reduce Pull Request Review Time With AI: A Reproducible Protocol
Ask the right question: AI review tools cut how long PRs WAIT, not much how long they take to READ. Plus a two-week PR-slice protocol.
2 min
Your AI reviewer is judging its own output. That's a blind spot
Teams drowning in AI-generated code often let an LLM review the LLM's own patches. Amazon's judge-correlation work shows why that misses real defects.
3 min
Reduce PR review time with AI: what the 45% claim leaves out
Atlassian says Rovo cut PR cycle time 45%. The number is real but self-attested. Here's how to measure whether AI actually reduces your review time.
2 min
Who reviews the AI-generated code before the human does?
The volume of AI-generated code is rising faster than review capacity. The fix starts in evaluation design: don't let the model that wrote the code also judge it.
2 min
Reviewing the surge in AI-generated code: what scales and what breaks
AI is producing more code than teams can review. First-party data on why the old loop breaks (Salesforce, DORA) and what actually scales.
3 min
Reviewing the volume of AI-generated code: the problem is routing, not speed
AI made PRs smaller but much more numerous. Reviewing the volume isn't a per-PR speed problem, it's a routing problem. Here's how teams actually triage AI-generated code.
2 min
How to Actually Evaluate an AI Code Review Tool
The failure mode that matters in AI review is not missing a bug, it is fluent output that is structurally wrong and easy to trust. How to benchmark for it.
2 min
AI code review benchmarks: offline vs online evals
How Martian's Code Review Bench separates reproducible fixed-dataset evals from streaming real-world evals, and the tradeoffs hidden in each.
13 min
How to Evaluate AI Code Review Tools (2026): A Playbook
A practical playbook for how to evaluate AI code review tools: a 9-standard scoring rubric, red flags, a 2-week trial protocol, and vendor questions.
12 min
Open Source AI Code Review: The Real Options (2026)
Open source AI code review tools compared: Kodus (AGPL), PR-Agent (MIT), and more — real licenses, BYOK costs, and how they stack up against closed SaaS.
13 min
Self-Hosted AI Code Review: Options & Trade-Offs (2026)
Self-hosted AI code review explained: full-stack vs BYOK vs on-prem runners, verified vendor options, and what deployment really costs in 2026.