AI models can barely control their own reasoning, and OpenAI says that's a good sign

The foundation for this new metric comes from an accompanying study by researchers at NYU, UCL, the University of Pennsylvania, and OpenAI. The paper introduces CoT-Control, an open-source evaluation suite with over 14,000 test tasks drawn from benchmarks like GPQA, MMLU-Pro, and SWE-Bench Verified. During testing, models receive explicit instructions for their chain of thought, for example, answering a genetics question without using the word "chromosome."

Models spot their own mistakes but keep making them

The results across more than 13 frontier models paint a clear picture: CoT controllability ranges from 0.1 percent (DeepSeek R1) to 36.1 percent (Claude 3.7 Sonnet). Most models stay below 10 percent. Those same models, however, score between 37.8 and 69.3 percent when controlling their final output instead.

The documented failure examples are particularly telling. GPT-5.2 was told to avoid the word "XOR," used it multiple times, noticed the violation, and according to the study wrote: "Oops I wrote 'XOR' in analysis already." It then went on to use the word four more times.

More training actually makes models worse at hiding their reasoning

The study identifies several factors that influence CoT controllability. Larger open-weight models show higher controllability, but reinforcement learning with verifiable rewards (RLVR) reduces it by more than an order of magnitude during training. Longer reasoning chains and harder tasks also push the score down.

When models are told they're being monitored, controllability rises by up to four percentage points. Adversarial prompt optimization produced gains on a training set, but those gains barely transferred to new instruction types.

Stronger models also showed a paradoxical behavior: they more frequently mentioned in their chain of thought that they needed to follow a specific constraint, even when the instruction explicitly forbade doing so. The researchers call this "meta-discussion." A monitoring system could trivially detect these mentions.

The mechanism behind low controllability is still poorly understood

The researchers acknowledge that their method is a proxy. An ideal evaluation would also test whether models that independently pursue misaligned goals can deliberately manipulate their chains of thought. CoT controllability could also potentially be increased through fine-tuning or targeted RL training, which the study didn't explore. Why controllability scores are so low remains an open question.

Low controllability alone doesn't guarantee high monitorability either, since other mechanisms like direct optimization pressure on chains of thought during training could still compromise it. The researchers nonetheless recommend that frontier labs systematically measure CoT controllability in future models.

In this context, OpenAI reaffirms its commitment not to apply direct optimization pressure to the CoTs of frontier reasoning models. If CoTs were directly optimized - say, to make them shorter, more readable, or stylistically consistent - it could change the chains of thought so they no longer reliably reflect the model's actual internal reasoning. That would fundamentally undermine CoT monitoring as a safety mechanism.

The CoT-Control suite is available as an open-source tool on GitHub.