The SSI founder and former OpenAI Chief Scientist sees AI development at a turning point. Instead of ever-larger models, fundamental research is now required again. Models need to learn more efficiently, similar to humans. He has ideas on how this might be possible, but says we now live in a world where one can no longer speak freely about such things.
In an extensive interview with Dwarkesh Patel, Ilya Sutskever, co-founder of Safe Superintelligence Inc. (SSI), describes a fundamental shift in AI development. While the years 2020 to 2025 were the "age of scaling," in which progress was primarily achieved through more compute and data, the industry is now returning to an "age of research."
"Scaling sucked out all the air in the room," explains Sutskever. Because the recipe—putting more data and compute into a neural net—worked so reliably, every company did the same thing. But pre-training data is finite. While scaling is still happening, particularly in Reinforcement Learning (RL), simply increasing resources no longer guarantees qualitative leaps.
According to Sutskever, we are back at a point similar to the time before 2020: new ideas and paradigms are needed, not just larger clusters.
AI models suffer from "Jaggedness"
A central problem with current models, according to Sutskever, is their inconsistency or "jaggedness." Models might perform excellently on difficult benchmarks, but often fail at basic tasks. He cites "vibe coding" as an example: A model recognizes a bug, introduces a new one while fixing it, only to restore the old bug on the next correction attempt.
Sutskever suspects that Reinforcement Learning (RL) training makes models "a little too single-minded." Unlike pre-training, where one simply used "all the data," one has to be selective with RL. This leads researchers—often unintentionally—to optimize models for specific benchmarks ("reward hacking"), which impairs generalization capabilities in the real world.
Human emotions as a biological "Value Function"
To reach the next level of intelligence, AI systems need to learn to generalize as efficiently as humans. A teenager learns to drive in about 10 hours, a fraction of the data an AI requires.
Sutskever theorizes that human emotions play a crucial role here by serving as a kind of robust "value function." These biologically anchored assessments help humans make decisions and learn from experiences long before an external result (as in classical RL) is available.
"Maybe it suggests that the value function of humans is modulated by emotions in some important way that’s hardcoded by evolution," says Sutskever.
AGI is the wrong goal – Superintelligence is created on the job
Sutskever also fundamentally questions the established term AGI. The success of pre-training created the false expectation that an AI must be able to do everything immediately ("General AI"). However, this overshoots the target: "A human being is not an AGI," says Sutskever. Humans lack enormous amounts of prior knowledge; instead, they rely on continual learning.
His vision of a superintelligence, therefore, resembles an extremely gifted student rather than an all-knowing database. "I produce a superintelligent 15-year-old that’s very eager to go," explains Sutskever. This system knows little at first but is capable of learning any profession. The actual competence only arises after release: The model must go through a phase of "trial-and-error" in real-world deployment to fully develop its capabilities, rather than coming out of the training cluster as a finished product.
Sutskever will not speak freely about his ideas
When asked how training must be conceptually changed to achieve this human learning efficiency, Sutskever remains tight-lipped. While he has "a lot of opinions" on the matter, the times of open exchange are over.
"Unfortunately, we live in a world where not all machine learning ideas are discussed freely, and this is one of them," Sutskever explains.
The mere existence of humans is proof to him that there must be a way. However, he qualifies that there could be a potential obstacle: There is a possibility that human neurons "do more compute than we think." If this plays a significant role, implementation could be more difficult.
Regardless, Sutskever implies that this points to the existence of a specific "machine learning principle" about which he has theories. But "circumstances make it hard to discuss in detail."
SSI wants to opt out of the commercial rat race
With his new company SSI, Sutskever is pursuing an approach intended to deliberately set itself apart from current market dynamics. The company, which has raised $3 billion, plans no quick product releases.
Sutskever argues that while SSI has less compute than the big tech giants, it can use it more efficiently for pure research, as no resources are tied up in operating commercial products or massive inference clusters.
The plan is a "straight shot" to superintelligence. They want to avoid the "rat race" of throwing products onto the market, instead researching in seclusion until the system is safe and mature. However, he concedes that a gradual release might be necessary to demonstrate the power of AI to society—and thus initiate regulatory processes.
Alignment towards sentient life
Regarding the safety (alignment) of a future superintelligence, Sutskever proposes a new goal: The AI should be aligned to care about "sentient life." Since the AI itself will be sentient, it is more natural to anchor empathy for other feeling beings than to program abstract human values.
Sutskever also predicts that the safety strategies of the major AI labs will converge as AI systems become noticeably more powerful. Current competition will give way to pragmatic collaboration as the risks become more tangible. A first sign of this is the nascent cooperation between competitors like OpenAI and Anthropic.
