summary Summary

A neural network in Apple's iOS keyboard, face scanning and hand tracking in its new Apple Vision Pro headset. Apple is putting AI, sorry, machine learning, into its software and hardware.


The most striking thing about Apple's keynote was the deliberate avoidance of the term "artificial intelligence", which is loaded with many ideas and explanations. Instead, Apple prefers to talk about "machine learning", a method from the field of AI, mostly in connection with "on-device" to emphasize privacy in data processing.

This can be seen as a meta-commentary on the current AI hype, which Apple has at least partially avoided so far. However, the choice of words is also in line with previous announcements. At least the latest market movements have not changed Apple's position not to use the term AI.

Anyway, such a "machine learning language model" now drives the autocorrection and word prediction of Apple's iOS 17 keyboard. The company emphasizes privacy here: the model runs Apple's "neural engine" on the mobile device and learns from the user's writing style, so the additions should become more personal over time. AI text suggestions are also used in Apple's Journal app, and the dictation function is more accurate thanks to a language model.


During the presentation of its AI-driven keyboard, Apple got carried away for a moment and mentioned a "Transformer" model, the neural network architecture behind recent breakthroughs in major language models. For a keynote watched by millions, this was surprisingly specific and probably more of a friendly salute to the AI community.

AI is in the details

Apple uses machine learning in other areas, such as animating static photos on the lock screen or auto-filling PDFs. Apps on the Apple Watch and AirPod headphones use machine learning to learn user habits and deliver information accordingly, or adjust volume automatically. Voicemails are automatically converted to text using AI.

These numerous quality-of-life improvements to Apple's software through machine learning are likely part of the vision that Apple's AI chief, John Giannandrea (formerly of Google), laid out in 2020: AI should transform every aspect of Apple's operating systems. Even then, Giannandrea emphasized that Apple would focus on local computing.

Interestingly, AI shows up in Giannandrea's job description: He is responsible for Apple's "AI strategy."

AI at the heart of Apple's latest hardware

AI and machine learning are fundamental to Apple's Vision Pro headset, which uses computer vision for room and hand tracking, as well as environment and object recognition via twelve cameras, five sensors, and six microphones.


Perhaps the biggest innovation is Apple's "persona," which is synthesized from a 3D facial scan of the user and then used, for example, in Facetime conversations with the headset. This "digital representation" is generated by an "advanced neural encoder-decoder network," according to Apple.

Über die in die Brille integrierten Sensoren erzeugt man einen 3D-Scan des eigenen Gesichts, der dann per KI, sorry, Machine Learning, zum Leben erweckt wird.
Using the sensors built into the headset, you create a 3D scan of your face, which is then brought to life via AI, sorry, machine learning. | Image: Apple

Meta has presented such high-end avatars in the past as a research project called "Codec Avatars". But Apple is now the first company to actually use them in a product.

According to Apple, there is also a lot of AI future in the new high-end "M2 Ultra" chip: It has a 32-core neural engine, as well as 24 CPU and 76 GPU cores, which together can perform up to 31.6 trillion operations per second. This is 40 percent more performance than the M1.

The extra power is suitable for training large transformers at levels that even "the most powerful discrete GPUs" could not handle due to memory constraints, according to Apple. The M2 Ultra offers 192GB of unified memory, which could make high-end Macs serious computers for AI development and research in the future.

Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
  • Apple emphasizes "machine learning" rather than "artificial intelligence" in its messaging about its latest technology, and emphasizes privacy through local data processing.
  • Machine learning iOS enhancements include personalized keyboard predictions, animated photos, customizable apps, and more.
  • The new Apple Vision Pro headset and M2 Ultra chip feature advanced AI capabilities such as 3D facial scanning and the ability to train powerful neural networks.
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.