Content
summary Summary

ChatGPT has the public excited, but the experts are reserved in their praise. Now, Meta's Chief AI Scientist and renowned AI researcher Yann LeCun comments on the new AI hype.

Ad

Since the end of November, AI has hit the mainstream thanks to ChatGPT: major media have reported on it several times, and hardly any article on the subject is without the passage "ChatGPT wrote this part". People who have never had any contact with AI technology are amazed by the texting capabilities of OpenAI's latest AI model, which gives the impression that it understands every instruction.

ChatGPT offers no innovation

Unlike GPT-3, however, the professional community has not widely embraced ChatGPT. The reason may be that ChatGPT successfully combines existing technologies and methods in a new interface. Technically, however, the system offers nothing new.

Large transformer models for language have been common for years, they were invented by Google in 2017, and the fine-tuning with human feedback used for ChatGPT is also an established method that OpenAI has been using since before ChatGPT.

Ad
Ad

"In terms of underlying techniques, ChatGPT is not particularly innovative," said Yann LeCun at a side event attended by ZDNet. While the technology may be perceived by the public as revolutionary, LeCun says it is not. ChatGPT is just "well put together," he said.

LeCun believes that compared to other companies in the field, OpenAI is not particularly advanced. Google, Meta and "half a dozen" other startups have equivalent technologies, he said, going on to say that the technology used for ChatGPT is well known in the industry and there is no particular mystery.

LeCun criticizes ChatGPT's hype - not OpenAI

After the event, LeCun followed up on Twitter: He was not criticizing OpenAI or the startup's claims. Instead, he said he wanted to "correct" public and media perceptions that see ChatGPT as an "incredibly new, innovative and unique" technology that surpasses all other companies' offerings. "It's just not," LeCun said.

Google and Meta would use the Transformer-based AI technology "absolutely everywhere," including for content moderation, content recommendations, and translations. Nor is OpenAI's infrastructure impressive to LeCun, whose employer, Meta, has been running large-scale AI language models for well over three billion people for years.

The lack of ChatGPT-like products from Meta and Google is due to "major flaws" in the systems and that established companies have more to lose than "cash-hungry startups," he said. Google, at least, is rumored to be planning a chatbot search this year.

Recommendation

Galactica doesn't write more nonsense than ChatGPT

Meta's scientific AI model Galactica, which LeCun supported, would generate less false information than ChatGPT, LeCun said. The language model, trained with scientific data, was made publicly available as a demo for a short time in mid-November.

However, it upset the scientific community so much with incorrect information and citations that the Galactica team decided to take the demo offline. The decision to take Galactica offline was made solely by the team to protect their own mental health. PR and management did not interfere, LeCun said.

"We saw Twitter-mediated backlash and knee-jerk prophecies of impending doom, fueled by anti-Meta prejudice," LeCun writes on Twitter.

If ChatGPT is now better than Galactica, it's because OpenAI spent the last two months collecting feedback from millions of users and hiring cheap labor to label it, according to LeCun.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Yann LeCun, Chief AI Scientist at Meta, doesn't think ChatGPT is particularly innovative from a technical standpoint.
  • Besides Google and Meta, there are half a dozen startups with similar technology, LeCun said.
  • The fact that Google and Meta do not offer a ChatGPT-like system is due to the "major flaws" of these systems.
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.