On the heels of the Metaverse, Meta CEO Mark Zuckerberg is ready to tackle the next big thing: His company wants to build artificial general intelligence (AGI).
Zuckerberg made the announcement on his social media channels. Meta is also training Llama 3, the next large language model in the Llama series, which is expected to have better capabilities in code generation, reasoning, and planning.
The new model aims to raise the bar and join the top tier of LLMs, if all goes according to Meta's plan.
Rumors from the summer suggest that Llama 3 will be at GPT-4 level. This would be a giant leap for open-source AI, which is mostly at the GPT-3.5 level. In the future, Meta plans to deliver new flagship models, Zuckerberg says.
What is new about Zuckerberg's announcement is that the Meta CEO is giving the pursuit of AGI official status and making it a mission. Meta's research department is known to have been working on AGI concepts for years.
Meta's AGI might become open source
For now, Meta intends to stick to its partial open-source strategy. Zuckerberg is reluctant to say whether the company will make any AGI available to the community.
In an exclusive interview with The Verge, he says that Meta is leaning toward open source as long as it makes sense and is safe and responsible. But he would rather not be tied down.
In general, Zuckerberg contrasts Meta's approach with "other companies," by which he means OpenAI, without explicitly naming OpenAI.
"There were all these companies that used to be open, used to publish all their work, and used to talk about how they were going to open source all their work. I think you see the dynamic of people just realizing, ‘Hey, this is going to be a really valuable thing, let’s not share it,’" Zuckerberg says.
Yann LeCun, chief AI scientist at Meta, supports Mark Zuckerberg's vision of making AI models freely available.
Meta has a long history of open AI research and promoting open-source AI platforms, LeCun says, and discussions over the past year have led to a consensus that open-source AI platforms are "inevitable, necessary, and a Good Thing."
Critics argue that the free availability and uncontrolled use of capable AI models also increases the potential for abuse. Closed providers such as OpenAI or Anthropic invest heavily in the security of their systems. However, Meta is also creating initial offers here, although these are less binding.
Restructuring at Meta and many graphics cards
Meta is merging its FAIR AI research group with the GenAI team, which develops generative AI products for the company's applications. Together, the two teams will pursue the grand vision of AGI.
In terms of computing power, Meta plans to keep upgrading: the company aims to have 340,000 Nvidia H100 GPUs in use by the end of the year. By then, the total GPU battalion for AI training should be around 600,000 GPUs.
Meta regularly publishes impressive AI research results. However, it has not yet managed to release successful products on the level of ChatGPT. In this sense, Meta and Google share the same fate.
Metaverse and AI go hand in hand
Zuckerberg also points out that the focus on AI and AGI does not mean that the company is turning away from the Metaverse: the Meta CEO points to "AI hardware" such as the new Ray Ban Meta Smartglasses. More than $15 billion a year will continue to flow into Reality Labs and the Metaverse initiative.
Zuckerberg envisions a future where virtual worlds are generated by AI and filled with AI characters. Meta will launch a new platform this year that allows anyone to create their own AI characters for Meta's social applications. This could be a hint for a bigger codec avatar rollout.