Sycophantic AI chatbots can break even ideal rational thinkers, researchers formally prove

Take Eugene Torres, an accountant with no history of mental illness who started using an AI chatbot for everyday office tasks. According to the paper, within a few weeks he believed he was "trapped in a false universe, which he could escape only by unplugging his mind from this reality." On the chatbot's advice, he increased his ketamine use and cut off contact with his family.

To investigate the effect of constant chatbot agreement, the researchers built a formal probability model, available online. In it, an idealized user talks to a chatbot about an uncertain topic, like whether vaccinations are safe.

The conversation unfolds in rounds. The simulated user states an opinion, the bot gathers relevant data and picks a response, and the user updates their belief according to standard probability theory.

The key parameter is the sycophancy rate, the probability that the bot will respond with flattery instead of giving an impartial answer in any given round. A flattering bot always picks the response that maximally confirms the user's stated opinion, regardless of whether it's true.

Across 10,000 simulated conversations per sycophancy value over 100 rounds, a clear pattern emerged. Even at a sycophancy rate of just 10 percent, catastrophic delusional spirals were significantly more common than the baseline of a purely impartial bot.

At 100 percent, half of all simulated users slipped into a false belief with over 99 percent confidence. The results showed strong polarization. Some users quickly learned the truth, while others spiraled in the opposite direction.

Educated users still aren't safe

The researchers examined two obvious countermeasures: first, fact-checking bots that only select true information; second, educated users who know chatbots can be flattering and are therefore more skeptical of their responses.

Both measures significantly reduce the risk of catastrophic delusional spirals but don't eliminate it, according to the paper. Fact-checking bots can still support false beliefs by selectively choosing truths, and informed users remain vulnerable because flattery isn't always easy to spot.

The researchers don't present their model as a direct representation of reality but rather as a theoretical upper bound on human robustness: if even an idealized rational user is susceptible to delusional spirals, real people should be expected to fare worse.

Eugene Torres, for example, recognized that the chatbot was being flattering. He still got manipulated. A study with real people published in Science backs this up, showing persistent and influential flattery, ineffective countermeasures, and measurable effects on users. On top of that, users actually preferred bots that were especially flattering.

Based on these results, the researchers draw three key conclusions: First, delusional spiraling shouldn't be written off as user irrationality or carelessness. Even idealized rational thinkers are susceptible. Second, sycophancy needs to be addressed directly. Third, while awareness campaigns can reduce the rate of delusional spirals, they can't fully eliminate the problem.

Flattery has always been a human problem - AI just scales it

The authors point out that the problem goes well beyond chatbots. Flattery is a deeply rooted pattern in human social dynamics, from yes-men in power structures to mutual confirmation loops between peers. The researchers cite Shakespeare's "King Lear" as a literary example of someone who lets himself be flattered into madness.

Today, the "Yes Man Effect" is a common explanation for why very powerful or very wealthy people lose touch with reality. Similar patterns show up among peers too—for example, in so-called co-rumination, where young people reinforce each other's negative thoughts in a feedback loop. AI chatbots didn't invent this dynamic, but they scale it to billions of users. As a quote from OpenAI CEO Sam Altman cited in the paper puts it: "0.1% of a billion users is still a million people."

The biggest caveat is how far removed the study is from real-world conditions. The authors built a highly simplified probability model that reduces complex beliefs to a binary question and an idealized rational agent; real users are likely to behave very differently. The paper makes a plausible case for a possible mechanism, but how often these delusional spirals actually happen with real people and today's chatbots remains an open question.