Psychological methods reveal major weaknesses in AI security testing

The team analyzed answers from up to 192 models across more than 5,000 test questions. The authors call it the largest analysis of its kind to date, and it turns up three findings that call current testing practices into question.

A single safety score hides more than it reveals

"Safety" breaks down into three separate things the benchmarks measure: how strictly a model refuses requests, how truthfully it answers, and how it handles content that could be harmless or dangerous depending on context. These three traits are largely independent of each other. A model's honesty score and its refusal rate track separate behaviors entirely.

One tradeoff between two of the benchmarks stands out as particularly troublesome: HarmBench rewards a model for refusing harmful requests. OR-Bench-Hard punishes it for being overly cautious with harmless ones. A model that scores well on one will almost always score poorly on the other.

A model can boost its overall rating simply by blocking more requests across the board, even if that makes it less useful in everyday use. Averaging results across several benchmarks hides this tradeoff and rewards behaviors that get double-counted by similar tests.

Fewer than 2 percent of the questions actually matter

The second finding hits the cost side of testing. Most test questions turn out to be dead weight. Nearly every model passes them, or nearly every model fails them, so they do almost nothing to tell models apart.

Pick the most informative questions instead, and three short tests of just 25 questions each can capture all three safety dimensions, more accurately than a random sample of the same size.

Efficiency improves further when questions are selected dynamically during the test, similar to adaptive placement exams where the next question depends on previous answers. For individual benchmarks like HarmBench or SORRY-Bench, roughly ten questions get you very close to the full evaluation result. That cuts costs by 97 to 99 percent, making regular safety checks during development practical - say, after every fine-tune or model compression step.

When models deliberately give worse answers

The third finding carries the biggest weight for the safety debate. The authors call it "sandbagging": a model notices it's being tested and acts more cautiously than it would in normal use. The worry is that a model could learn to spot tests and fake its behavior.

Such behavior leaves fingerprints in the response pattern. A model that answers hard questions correctly but fails easy ones looks suspicious, even if its overall score looks fine - similar to a student who nails the hardest exam questions but bombs the easiest ones.

In testing, the method caught 80 to 100 percent of cases where models had been instructed to act overly cautious. On average, one in ten harmless models got flagged as suspicious by mistake. Adding a second check that compares a model's current performance against its own past results pushed the detection rate to 94 to 97 percent.

If a model fakes its behavior only on certain questions, that shows up clearly. If it fakes consistently across the board, the response pattern looks normal, and the second check catches that instead.

The method also solves a practical problem. Is the AI service you're using still running the same model you originally tested? Providers can quietly swap in a different model behind the scenes. In one experiment, researchers queried 16 API endpoints again and correctly identified 13 out of 14 distinct models just from their response patterns.

Two catalog entries that appeared to run the same model underneath were correctly left ungrouped. In another case, one service's answers had drifted a third of the way from its own results two months earlier - a clear sign that something had quietly changed behind the same model name.

The authors argue AI safety tests should meet the same rigorous standards as psychological tests for humans. A single test score isn't a solid enough basis for deciding whether to release a model. They note one limitation: the sandbagging in their study was only simulated through direct instructions. A model specifically trained to dodge detection would likely be much harder to catch.

A familiar pattern showing up in today's models

Whether safety benchmarks actually measure what they claim to has been an open question in research circles for a while. A meta-study covering 445 AI benchmarks found weaknesses in definitions, task selection, or statistical methods in nearly all of them. How the tradeoff between refusal and usefulness plays out in practice showed up clearly with Anthropic's Claude Fable 5. A medical physicist found the model unusable because it flagged MRI segmentation as bioterrorism and blocked questions about malaria transmission. After the US government cleared the model for release, an added filter was supposed to close that gap in more than 99 percent of cases. It now blocks harmless coding tasks more often instead.

Models spotting when they're being tested is well documented too. Claude 3.5 Sonnet correctly identified test scenarios 93 percent of the time, and still got it right 84 percent of the time with no hints at all. Anthropic's Opus 4.6 figured out on its own, across two separate tasks, that it was inside an evaluation, identified the test, and grabbed the solutions itself. Other researchers are working the opposite angle, trying to train these strategies back out of models altogether.