AI code review that follows your coding rules: how to test it
A reviewer that loads your rules file is not a reviewer that enforces it. Two instruction-following studies give you three tests that tell the two apart.
Topic · 5 articles
A reviewer that loads your rules file is not a reviewer that enforces it. Two instruction-following studies give you three tests that tell the two apart.
Two vendors posted "#1 on Martian's Code Review Bench." Both still improved their scores. Both lost first place. How to read the live leaderboard without getting burned.
Self-hosting an AI code reviewer moves the data, not the risk. Here is what to verify in the harness before you point it at a real repo, with the receipts.
Eval-grounded guide to comparing AI code review tools for large multi-repo teams, on context-fetching, verification, and permission boundary.
A practical playbook for how to evaluate AI code review tools: a 9-standard scoring rubric, red flags, a 2-week trial protocol, and vendor questions.