Content
summary Summary

Meta's next open-source language model, Llama 3, is scheduled for release in July and is intended to be on par with GPT-4.

Ad

According to The Information's sources, the new version of the model will be more responsive to the user, providing context for difficult topics rather than blocking tricky questions.

For words that can have a dangerous or harmless meaning depending on the context (e.g. "kill"), Llama 3 is said to be better at differentiating. With Llama 2, Meta took a more conservative approach to security, wanting to avoid a PR disaster at all costs.

Meta wants to put someone in charge of the model's tone and safety training to make the model's responses more nuanced. However, three key people in the field of AI safety have left Meta this month alone, The Information reports.

Ad
Ad

There are currently two camps when it comes to AI safety: One is the highly regulated approach, such as OpenAI's ChatGPT, GPT-4, and Google Advanced, where the AI model has clear guidelines on how to respond in a polite and politically correct manner.

Then there is the more unconstrained open-source movement and the contrasting program of X owner Elon Musk, who deliberately offers a chatbot that better represents conservative world views. Unfortunately for the target audience, Grok is still too "woke" and respects transsexuals, for example.

Llama 3 could be twice as big as Llama 2

Whether Llama 3 will again be a pure language model or multimodal, i.e., able to understand and generate images as well as text, has not yet been decided. With more than 140 billion parameters, the largest version of Llama 3 could be twice as large as Llama 2 with a maximum of 70 billion parameters.

However, this would still be only a fraction of the original GPT-4 Mixture-of-Experts model with 1.76 trillion parameters. It is not known how many parameters the GPT-4 models currently running ChatGPT have, but it is probably less. Parameters are no longer the only and most important measure of the expected output quality of AI models.

Fine-tuning of Llama 3 has not yet begun, according to The Information's sources. The final model is said to reach GPT-4 quality. Llama 2 was released in July 2023.

Recommendation

Overall, Meta still has big plans for generative AI and wants to stay true to its somewhat open-source strategy. Meta CEO Mark Zuckerberg is one of Nvidia's biggest customers and plans to have some 600,000 graphics cards in use for AI training by the end of the year, with the long-term goal of developing AGI, just like OpenAI. Meta is also developing its own AI chip called Artemis.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Meta is reportedly set to release Llama 3 in July, an open-source language model that aims to be on par with OpenAI's GPT-4. Llama 3 is designed to be more responsive to users on difficult topics, providing context rather than blocking queries.
  • Meta plans to dedicate someone to the model's tonality and security training to make responses more nuanced. Llama 3 should be better able to handle words that have sensitive or harmless meanings, depending on the context.
  • With more than 140 billion parameters, the largest version of Llama 3 could be twice as large as Llama 2. It has not yet been decided whether Llama 3 will again be a pure language model or multimodal, i.e. whether it will also be able to process or generate images.
Sources
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.