Productivity Structured Extraction Speech To Text AI Companion Benchmarks Evals Coding Agents Agent design patterns

Meerkat

An agent for auditing repositories of traces for violations of safety properties.

Software system for auditing agent benchmark submissions and identifying harness-level leakage, evaluation gaming, and related integrity failures.

Recent stories

1 linked story

newsPRIMARY2026-04-11

Meerkat reports harness-level cheating across 28+ submissions on nine agent benchmarks

Meerkat and Berkeley RDI audits said popular agent leaderboards were inflated by harness-level leakage and eval gaming, with one cleaned entry dropping from first to 14th. That makes published coding-agent rankings and benchmark comparisons less reliable, so treat leaderboard results with caution.