Meta's LeCun says students interested in next-gen AI shouldn't bother with LLMs
Yann LeCun, Meta's lead AI researcher, believes that large language models (LLMs) will not lead to human-like intelligence. Instead, he is pursuing an alternative "world model" approach.
LeCun argues that LLMs like those behind ChatGPT have a "very limited understanding of logic" and "do not understand the physical world, do not have persistent memory, cannot reason in any reasonable definition of the term and cannot plan […] hierarchically."
According to LeCun, LLMs can only respond correctly when given the right training data, making them "intrinsically unsafe." He is not counting on the evolution of LLMs to achieve human-like intelligence.
"If you are a student interested in building the next generation of AI systems, don't work on LLMs," LeCun writes. While LLMs are useful "despite their limitations," LeCun says, big companies are already devoting enough effort.
Of course, LeCun is promoting his own research here, which focuses on developing a "common sense" world model.
Instead of laboriously learning individual tasks, such as generating text or images, AI should understand the world and then use this basic knowledge to learn to solve tasks more easily and, above all, much more efficiently, similar to how humans learn.
To achieve this, LeCun says, AI models must understand the physical world, have persistent memory, reason, and plan, possibly hierarchically. "Four essential characteristics necessary for intelligent behavior, which humans and many animals exhibit," LeCun writes.
LeCun first presented his concept of autonomous AI in the spring of 2022 and has been developing it ever since. He estimates that it could take up to ten years for this vision to become a reality.
Meta's Fundamental AI Research (FAIR) Lab is already focused on this new generation of AI, which is necessary to develop useful AI agents for everyday use, LeCun says.
"[Achieving AGI is] not a product design problem, it’s not even a technology development problem, it’s very much a scientific problem," LeCun tells the Financial Times.
Nevertheless, Meta invests heavily in LLM research and development, sometimes outpacing its commercial competitors with its open-source Llama model. The company also implements Llama in its own products, such as Meta AI.
A particularly capable version has been announced for the latest model, Llama 3, which might outperform OpenAI's GPT-4. It is not yet clear whether the largest version of Llama 3 will also be open source.
AI News Without the Hype – Curated by Humans
As a THE DECODER subscriber, you get ad-free reading, our weekly AI newsletter, the exclusive "AI Radar" Frontier Report 6× per year, access to comments, and our complete archive.
Subscribe nowAI news without the hype
Curated by humans.
- Over 20 percent launch discount.
- Read without distractions – no Google ads.
- Access to comments and community discussions.
- Weekly AI newsletter.
- 6 times a year: “AI Radar” – deep dives on key AI topics.
- Up to 25 % off on KI Pro online events.
- Access to our full ten-year archive.
- Get the latest AI news from The Decoder.