MIT study explains why scaling language models works so reliably

Until now, many explanations assumed that only the most common concepts get cleanly represented while the rest is lost ("weak superposition"). The MIT team shows, using a simplified model from Anthropic, that this picture doesn't match how real LLMs actually work.

Two regimes offer two different explanations

The researchers built a heavily simplified AI model with a training dial that let them control how much stored concepts were allowed to overlap. This made it possible to compare two extreme cases.

In the first case—weak superposition—the model only stores the most common concepts cleanly and ignores the rest. Prediction error here comes mainly from the rare concepts that get dropped. Whether performance scales cleanly as a power law depends on how concepts are distributed in the training data. Only when that distribution itself follows a power law does the error follow one too. The paper calls this "power law in, power law out."

In the second case—strong superposition—the model stores all concepts at once by letting their vectors overlap slightly. The error no longer comes from missing concepts but from the noise created by these overlaps. Here, a robust pattern emerges: doubling the model's width roughly cuts the error in half, predicted by a simple geometric relationship (1/m, where m is the model's width). How concepts are distributed in the data barely matters anymore.

Real language models confirm the theory

To check which regime applies to real systems, the team examined the output layers of open-source models: OPT, GPT-2, Qwen2.5, and Pythia, ranging from roughly 100 million to 70 billion parameters. The result is clear: all tokens are represented in the model, their vectors overlap, and the strength of those overlaps shrinks at exactly the predicted 1/m ratio. Language models operate in the strong superposition regime.

The measured scaling exponent lines up too, landing at 0.91, close to the theoretical value of 1. Deepmind's Chinchilla data produces a nearly identical 0.88. According to the researchers, these scaling laws fall directly out of how language models organize meaning geometrically within their representations.

Practical implications for scaling and architecture

The work provides concrete answers to two open questions in AI research. First: does scaling eventually stop working? According to the researchers, yes, once a model's width matches the size of its vocabulary. At that point, there's enough room to represent every token without overlap, and the error caused by cramped representations vanishes. The power law breaks down at that boundary.

Second: Can scaling laws be sped up to squeeze more performance out of each added parameter? For natural language, probably not; word frequency distributions are relatively flat. But for specialized applications where relevant concepts are distributed very unevenly, steeper scaling could be on the table.

This also has implications for architecture design: models that actively encourage superposition should perform better at the same size. One example is Nvidia's nGPT, which forces internal vectors onto a unit sphere, packing them more densely.

There's a catch, though: the more concepts overlap, the harder it gets to trace what's actually happening inside the model. That's a real problem for mechanistic interpretability and, by extension, AI safety research.