Large language models have a tendency to escalate military scenarios - sometimes all the way to nuclear war.

"The AI is always playing Curtis LeMay," Jacquelyn Schneider of Stanford University told Politico, describing her team's experiments using language models in military wargames. LeMay, a US general during the Cold War, was famous for his aggressive stance on nuclear weapons. "It’s almost like the AI understands escalation, but not de-escalation. We don’t really know why that is."

In these simulations, the models consistently pushed situations toward escalation - often ending in nuclear strikes. Schneider and her team think the root of the problem is the training data: large language models learn from existing literature, which usually highlights conflicts and escalation. Peaceful resolutions, like the Cuban Missile Crisis, are rarely covered in detail. With so few examples of "non-events" in the data, de-escalation is hard for the AI to model. These tests used older language models, including GPT-4, Claude 2, and Llama-2.