Ad
Skip to content
Subscribe Now
AI in practice
Copy the url to clipboard Share this article Go to comment section

Meta to release GPT-4 competition as open-source model next year

Matthias Bastian
Matthias Bastian View the LinkedIn Profile of Matthias Bastian
Sep 11, 2023
Image description
Midjourney prompted by THE DECODER

The Wall Street Journal also reports on Meta's plans for Llama 3, which is said to be many times more capable than Llama 2.

According to the WSJ, Meta is planning the new model for 2024, and is upgrading its data centers specifically for AI training with H100s. Training is expected to begin in early 2024, and the resulting model is said to be "roughly" on par with GPT-4 and significantly better than Llama 2. The WSJ cites sources familiar with the project.

Meta CEO Mark Zuckerberg is reportedly sticking to the open-source strategy of making the Llama models available for free for many, though not all, use cases - including commercial ones - despite the performance increase.

The release of the new model is said to be part of Zuckerberg's strategy to reposition Meta as a leading AI company. However, Google should have released Gemini by then, and OpenAI is also expected to release more innovations by next year - though probably not GPT-5.

Ad
DEC_D_Incontent-1

WSJ confirms rumors about Llama 3

The Wall Street Journal report coincides with statements made by OpenAI engineer Jason Wei. He reported in late August about a group meeting with Meta AI developers who claimed to have enough computing power to train Llama 3 and 4. Llama 3 is expected to reach the level of GPT-4 but will remain open source.

Llama 2 reaches the level of GPT-3.5 in some applications and is also being optimized by the open-source community through fine-tuning and additional applications. Reaching the level of GPT-4 is likely to be much more challenging, as the current OpenAI model is based on a more complex architecture of 16 expert networks with about 111 billion parameters each.

This may explain why Meta's goal with an open-source approach is not to surpass GPT-4, but only to be somewhat on par with GPT-4. Distributing a model with the complexity of GPT-4 could be much more difficult.

AI News Without the Hype – Curated by Humans

As a THE DECODER subscriber, you get ad-free reading, our weekly AI newsletter, the exclusive "AI Radar" Frontier Report 6× per year, access to comments, and our complete archive.

AI news without the hype
Curated by humans.

  • Over 20 percent launch discount.
  • Read without distractions – no Google ads.
  • Access to comments and community discussions.
  • Weekly AI newsletter.
  • 6 times a year: “AI Radar” – deep dives on key AI topics.
  • Up to 25 % off on KI Pro online events.
  • Access to our full ten-year archive.
  • Get the latest AI news from The Decoder.
Subscribe to The Decoder