The blog · 37 articles
Everything we learn testing AI code reviewers against the 9 standards — written for the engineer who has to make the call, not the person signing the invoice.
A reviewer that loads your rules file is not a reviewer that enforces it. Two instruction-following studies give you three tests that tell the two apart.
Read the articleTwo vendors posted "#1 on Martian's Code Review Bench." Both still improved their scores. Both lost first place. How to read the live leaderboard without getting burned.
Self-hosting an AI code reviewer moves the data, not the risk. Here is what to verify in the harness before you point it at a real repo, with the receipts.
An eval-grounded comparison of AI code review tools that actually work on Azure Repos: Copilot code review, CodeRabbit, Qodo/PR-Agent, and Kodus, plus the PAT footgun that silently kills reviews.
Teams reviewing a growing volume of AI-generated code need to score what a reviewer fails to notice, not only what it flags. A measurable protocol.
A canary test, block tests, and independent judging to prove an AI code reviewer actually follows your team's coding standards, not just claims to.
Trail of Bits spent six months having agents build an LSP, decompiler, static analyzer, and Lean proof model before the Miden review even started. That is where the leverage landed.
Anthropic made claude.ai 3x faster in two weeks by putting a model inside a measure-build-steer loop. An eval-grounded look at what the sprint proves about agentic workflows.
A hands-on look at what "free" really means across AI code review tools — metered credits, trials, BYO-LLM, and how to evaluate them.
Pullfrog is a BYOK harness over Claude Code and Codex, not a first-party reviewer like CodeRabbit. Review quality is model-attributed, not harness-attributed. Here's what actually separates them.
An eval-grounded look at Pullfrog vs CodeRabbit: when a reviewer wraps Claude Code or Codex, review quality is model-attributed, not harness-attributed. The real difference is the credential boundary.
Comparing Pullfrog and CodeRabbit properly means separating the reviewer harness from the underlying model, not just lining up features.
Ask the right question: AI review tools cut how long PRs WAIT, not much how long they take to READ. Plus a two-week PR-slice protocol.
Qodo folded PR-Agent/Merge into one platform. For teams comparing alternatives, the real question is which model + harness you're actually betting on.
Eval-grounded guide to comparing AI code review tools for large multi-repo teams, on context-fetching, verification, and permission boundary.
Teams drowning in AI-generated code often let an LLM review the LLM's own patches. Amazon's judge-correlation work shows why that misses real defects.
Stop comparing marketing claims. An eval-based protocol for picking AI code review tools across many repositories: cross-repo context, verification, permission boundaries.
Atlassian says Rovo cut PR cycle time 45%. The number is real but self-attested. Here's how to measure whether AI actually reduces your review time.
Eval-grounded comparison of AI code review tools for large teams with many repositories: cross-repo context, verification, permission boundary.
The volume of AI-generated code is rising faster than review capacity. The fix starts in evaluation design: don't let the model that wrote the code also judge it.
When AI writes 30% of your lines, patch-text review hits a ceiling. The fix is an execution layer, not a bigger reviewer.
AI is producing more code than teams can review. First-party data on why the old loop breaks (Salesforce, DORA) and what actually scales.
AI made PRs smaller but much more numerous. Reviewing the volume isn't a per-PR speed problem, it's a routing problem. Here's how teams actually triage AI-generated code.
The failure mode that matters in AI review is not missing a bug, it is fluent output that is structurally wrong and easy to trust. How to benchmark for it.
How Martian's Code Review Bench separates reproducible fixed-dataset evals from streaming real-world evals, and the tradeoffs hidden in each.
I ran zizmor 1.29.0 against the exact Snowflake GitHub Actions workflow. A deterministic static rule flagged the injection at High confidence while AI review cleared it.
Netlify ran the same build prompt across 11 AI models using their open-source AXIS evaluator. Here is what the results tell us about model selection for code generation.
AI code review statistics for 2026: adoption, trust, review turnaround, AI code volume, and bug-catch benchmarks — every stat linked to a primary source.
AI code review vs static analysis compared: determinism vs reasoning, false positives, SAST coverage, cost, and why mature teams run both.
The best AI code review tools 2026 offers, compared honestly: Kodus, CodeRabbit, Greptile, Copilot and more — context depth, pricing, self-hosting.
CodeRabbit vs Greptile head-to-head: context models, review quality, pricing, self-hosting, and when to pick each. Verified August 2026.
Why teams leave CodeRabbit and 7 alternatives compared — Kodus, Greptile, Qodo, BugBot, Copilot, Graphite, Panto. Pricing verified August 2026.
Cursor BugBot vs CodeRabbit: review philosophy, pricing, platform support, and self-hosting compared — plus when neither fits. Verified August 2026.
A practical playbook for how to evaluate AI code review tools: a 9-standard scoring rubric, red flags, a 2-week trial protocol, and vendor questions.
Open source AI code review tools compared: Kodus (AGPL), PR-Agent (MIT), and more — real licenses, BYOK costs, and how they stack up against closed SaaS.
Self-hosted AI code review explained: full-stack vs BYOK vs on-prem runners, verified vendor options, and what deployment really costs in 2026.
AI code review explained: how LLM reviewers work, what they catch and miss, how they differ from linters and static analysis, plus sourced adoption data.
Want the data rather than the argument?
All 27 tools, scored against the same 9 standards, with a source for every claim.