Content
summary Summary

At Meta's 11th annual Connect conference, CEO Mark Zuckerberg showcased innovations in mixed reality, AI, and wearables.

Ad

Meta AI, the company's AI assistant, was a major focus, with Zuckerberg revealing that it is used by 400 million people every month, and 185 million on a weekly basis.

To make interactions with Meta AI more natural, the company is introducing voice input for the assistant on Facebook, Messenger, WhatsApp, and Instagram, initially in Australia, Canada, New Zealand, and the USA, and only in English. Well-known voices like Awkwafina, Dame Judi Dench, and John Cena will be used to give the AI assistant more personality.

Video: Meta

Ad
Ad

Meta is also experimenting with automatic video and lip-syncing for reels on Instagram and Facebook to make content in different languages accessible to a wider audience. This could impact the business models of dedicated startups like HeyGen.

Video: Meta/via @altryne/X

The AI-supported editing tools are also being expanded, allowing users to ask questions about uploaded photos and add, remove, or change elements in real photos using voice commands, thanks to the vision capabilities of the new Llama 3.2 models.

Image: Meta
Image: Meta

Meta has also released smaller models with 1B and 3B parameters that are optimized for use on devices like smartphones or future AR headsets.

Overall, Meta seems to be catching up with the commercial competition from OpenAI, with the performance gap to models like GPT-4 shrinking. Multimodality and the voice interface are two important functions that end customers can actually use in everyday life.

Recommendation

Ray-Ban Meta Smart Glasses become a city guide

The Ray-Ban Meta Smart Glasses are also receiving updates, including voice command reminders, QR code and phone number scanning, and a personal city tour in real-time thanks to extended Meta-AI support. The headsets will soon be able to translate Spanish, French, and Italian in real-time into English.

Thanks to a cooperation with Be My Eyes, users can also take on the perspective of visually impaired people and describe their surroundings. The collection is being expanded to include new transition lenses and a limited edition of transparent Wayfarer frames.

Orion: Meta's first AR headset prototype

One of the highlights of the conference was the unveiling of Orion, Meta's first AR headset prototype, which the company says it has been working on for ten years.

The wireless headset weighs less than 100 grams and is equipped with ten customized silicon chips and bright displays with a wide field of view (70 degrees).

Image: Meta

It features voice control, hand and eye tracking to navigate through the user interface, and a new, discreet, and "socially acceptable" interaction option via an EMG wristband.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.

Although Orion is only a prototype, Meta emphasizes that it could be an end customer product, showing how far development has already progressed. Even if the headset is never launched on the market, an even better version could be developed more quickly. For Meta CEO Mark Zuckerberg, it was particularly surprising that the AI elements are ready for the market earlier than the actual hardware - something he had predicted a few years ago.

Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Meta AI, Meta's in-house AI assistant, is experiencing strong user growth with 400 million monthly active users. New features such as voice input, celebrity voices and automatic video and lip dubbing are designed to make interaction more natural.
  • Meta's Llama 3.2 AI models allow users to ask questions about photos and edit elements using voice commands. Smaller models with 1B and 3B parameters are specially optimized for use on devices such as smartphones or AR headsets.
  • The Ray-Ban Meta Smart Glasses receive updates such as reminder functions, QR code and phone number scanning as well as a personal city tour in real time. Meta also unveiled Orion, its first AR headset prototype, which offers a display and AI functions despite weighing less than 100 grams.
Sources
Jonathan works as a technology journalist who focuses primarily on how easily AI can already be used today and how it can support daily life.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.