Visual effects specialist Chris Umé and his business partner Thomas Graham land a new deepfake hit. With their performance, they also want to promote a positive metaverse future with synthetic media.
For more than a year, deepfake Tom Cruise, also known as actor Miles Fisher, has been collecting clicks and likes on TikTok and Co. Thanks to deepfake technology, he imitates Tom Cruise so credibly that he fooled Justin Bieber and lured the real Tom Cruise onto TikTok.
Behind his transformation is the start-up Metaphysic.ai, which specialized in deepfakes early on - and is now making the leap to the big stage.
America's Got Talent with deepfake performance
On the talent show "America's Got Talent," Umé and colleagues present deepfake technology to a large US audience. They put singers on stage to perform in real-time as deepfake - with the faces of the judges singing to themselves in this way. Their faces appear on a large screen behind them. On YouTube alone, the performance reached around 20 million views.
In the recent semi-final, Metaphysic then went one better and had all three judges sing to themselves as deepfake. Voting is underway for the remainder of the show.
Similar to the Tom Cruise deepfakes, Metaphysic went to considerable production effort to achieve this deepfake quality. Among other things, they trained their own neural network with a lot of image data of the jurors and selected singers who resembled the jurors at least in basic features.
Synthetic reality in the metaverse age
Under synthetic media, Metaphysic - and many others - subsumes AI-generated media. Deepfakes would fall into this category, but the term has a rather negative connotation due to its origin.
Metaphysic pitches its technology in the metaverse context: instead of colorful comic worlds, people could interact as "hyperrealistic" avatars in realistic virtual worlds in the future thanks to synthetic media.
With "every anyone," the company offers an app that is supposed to be able to create such an avatar based on photos: Using one or more 2D photos, the app generates a simple 3D front view of one's own face, which can then be modified in near real-time based on parameters such as age, gender or facial expression. In the future, these hyperrealistic avatars will be available in more variants and across platforms.
Me, as an old, delighted lady avatar (according to Metaphysic). | Video: Metaphysic / MIXED
Metaphysic also founded "Synthetic Futures" in 2021, a global organization that advocates for a positive future with synthetic media.
