Snapchat's researchers have developed a new method for AI images on smartphones. This should allow users to eliminate the hardware that would otherwise be required and enjoy greater privacy.
Recent versions of image AI, such as Midjourney 5.1, Stable Diffusion XL, and Adobe Firefly, have raised the quality of generated graphics to a new level. However, these models also have undeniable drawbacks: they are very large and have complex network architectures, which makes them computationally intensive and slow.
Running these models at scale requires high-end GPUs and cloud-based inference, which is expensive and raises privacy concerns. Researchers at Snapchat's parent company, Snap Inc, and Northeastern University are now demonstrating SnapFusion. This model is said to be the first to run diffusion models on a smartphone in less than two seconds.
Although chipmaker Qualcomm showed in February that it could generate AI images on a smartphone in less than 15 seconds, SnapFusion runs much faster, at least on the iPhone 14 Pro.
Images on par with Stable Diffusion v1.5
By introducing a more efficient network architecture and fewer inference steps, SnapFusion is able to generate a 512-by-512-pixel image from a text prompt in a short time, approaching the quality of Stable Diffusion v1.5, according to the team. To do this, SnapFusion requires only eight denoising steps, while Qualcomm's method requires 20 steps.
A demo video from the researchers shows SnapFusion in action on Apple's most powerful smartphone to date, an iPhone 14 Pro. Qualcomm's method was previously only possible with its latest high-end chip, the Snapdragon 8 Gen 2.
Democratizing image AIs
"Our work democratizes content creation by bringing powerful text-to-image diffusion models to the hands of users," the researchers say, explaining their motivation for working on the project. But SnapFusion is far from perfect.
According to the researchers, the model still has a relatively large number of parameters. In addition, in the near future, work will need to be done to make the technology work on more smartphones than just the iPhone 14 Pro to make it accessible to a broader mass.
Snapchat already has experience with generative AI, but more in the text space with its personal chatbot, My AI.
