Topic · 24 articles
GitLab changes what an AI reviewer can actually do. The provider matrix, the GitLab-capable tools scored against the same nine standards, and seven checks to run on your own merge requests.
Two vendors posted "#1 on Martian's Code Review Bench." Both still improved their scores. Both lost first place. How to read the live leaderboard without getting burned.
Self-hosting an AI code reviewer moves the data, not the risk. Here is what to verify in the harness before you point it at a real repo, with the receipts.
An eval-grounded comparison of AI code review tools that actually work on Azure Repos: Copilot code review, CodeRabbit, Qodo/PR-Agent, and Kodus, plus the PAT footgun that silently kills reviews.
Teams reviewing a growing volume of AI-generated code need to score what a reviewer fails to notice, not only what it flags. A measurable protocol.
A canary test, block tests, and independent judging to prove an AI code reviewer actually follows your team's coding standards, not just claims to.
Trail of Bits spent six months having agents build an LSP, decompiler, static analyzer, and Lean proof model before the Miden review even started. That is where the leverage landed.
A hands-on look at what "free" really means across AI code review tools — metered credits, trials, BYO-LLM, and how to evaluate them.
Pullfrog is a BYOK harness over Claude Code and Codex, not a first-party reviewer like CodeRabbit. Review quality is model-attributed, not harness-attributed. Here's what actually separates them.
An eval-grounded look at Pullfrog vs CodeRabbit: when a reviewer wraps Claude Code or Codex, review quality is model-attributed, not harness-attributed. The real difference is the credential boundary.
Comparing Pullfrog and CodeRabbit properly means separating the reviewer harness from the underlying model, not just lining up features.
Ask the right question: AI review tools cut how long PRs WAIT, not much how long they take to READ. Plus a two-week PR-slice protocol.
Qodo folded PR-Agent/Merge into one platform. For teams comparing alternatives, the real question is which model + harness you're actually betting on.
Eval-grounded guide to comparing AI code review tools for large multi-repo teams, on context-fetching, verification, and permission boundary.
Teams drowning in AI-generated code often let an LLM review the LLM's own patches. Amazon's judge-correlation work shows why that misses real defects.
Stop comparing marketing claims. An eval-based protocol for picking AI code review tools across many repositories: cross-repo context, verification, permission boundaries.
Eval-grounded comparison of AI code review tools for large teams with many repositories: cross-repo context, verification, permission boundary.
The volume of AI-generated code is rising faster than review capacity. The fix starts in evaluation design: don't let the model that wrote the code also judge it.
When AI writes 30% of your lines, patch-text review hits a ceiling. The fix is an execution layer, not a bigger reviewer.
AI is producing more code than teams can review. First-party data on why the old loop breaks (Salesforce, DORA) and what actually scales.
AI made PRs smaller but much more numerous. Reviewing the volume isn't a per-PR speed problem, it's a routing problem. Here's how teams actually triage AI-generated code.
The failure mode that matters in AI review is not missing a bug, it is fluent output that is structurally wrong and easy to trust. How to benchmark for it.
How Martian's Code Review Bench separates reproducible fixed-dataset evals from streaming real-world evals, and the tradeoffs hidden in each.
I ran zizmor 1.29.0 against the exact Snowflake GitHub Actions workflow. A deterministic static rule flagged the injection at High confidence while AI review cleared it.