ChatGPT has the public excited, but the experts are reserved in their praise. Now, Meta's Chief AI Scientist and renowned AI researcher Yann LeCun comments on the new AI hype.
Since the end of November, AI has hit the mainstream thanks to ChatGPT: major media have reported on it several times, and hardly any article on the subject is without the passage "ChatGPT wrote this part". People who have never had any contact with AI technology are amazed by the texting capabilities of OpenAI's latest AI model, which gives the impression that it understands every instruction.
ChatGPT offers no innovation
Unlike GPT-3, however, the professional community has not widely embraced ChatGPT. The reason may be that ChatGPT successfully combines existing technologies and methods in a new interface. Technically, however, the system offers nothing new.
Large transformer models for language have been common for years, they were invented by Google in 2017, and the fine-tuning with human feedback used for ChatGPT is also an established method that OpenAI has been using since before ChatGPT.
"In terms of underlying techniques, ChatGPT is not particularly innovative," said Yann LeCun at a side event attended by ZDNet. While the technology may be perceived by the public as revolutionary, LeCun says it is not. ChatGPT is just "well put together," he said.
LeCun believes that compared to other companies in the field, OpenAI is not particularly advanced. Google, Meta and "half a dozen" other startups have equivalent technologies, he said, going on to say that the technology used for ChatGPT is well known in the industry and there is no particular mystery.
LeCun criticizes ChatGPT's hype - not OpenAI
After the event, LeCun followed up on Twitter: He was not criticizing OpenAI or the startup's claims. Instead, he said he wanted to "correct" public and media perceptions that see ChatGPT as an "incredibly new, innovative and unique" technology that surpasses all other companies' offerings. "It's just not," LeCun said.
Google and Meta would use the Transformer-based AI technology "absolutely everywhere," including for content moderation, content recommendations, and translations. Nor is OpenAI's infrastructure impressive to LeCun, whose employer, Meta, has been running large-scale AI language models for well over three billion people for years.
The lack of ChatGPT-like products from Meta and Google is due to "major flaws" in the systems and that established companies have more to lose than "cash-hungry startups," he said. Google, at least, is rumored to be planning a chatbot search this year.
Galactica doesn't write more nonsense than ChatGPT
Meta's scientific AI model Galactica, which LeCun supported, would generate less false information than ChatGPT, LeCun said. The language model, trained with scientific data, was made publicly available as a demo for a short time in mid-November.
However, it upset the scientific community so much with incorrect information and citations that the Galactica team decided to take the demo offline. The decision to take Galactica offline was made solely by the team to protect their own mental health. PR and management did not interfere, LeCun said.
"We saw Twitter-mediated backlash and knee-jerk prophecies of impending doom, fueled by anti-Meta prejudice," LeCun writes on Twitter.
If ChatGPT is now better than Galactica, it's because OpenAI spent the last two months collecting feedback from millions of users and hiring cheap labor to label it, according to LeCun.
