Adobe's VideoGigaGAN turns low-res videos into high-res, detailed clips
Key Points
- Adobe researchers demonstrate VideoGigaGAN, a new generative model for video super resolution (VSR) that can scale low-resolution video to higher resolution while maintaining fine-grained detail and temporal consistency.
- To address problems such as temporal flicker and aliasing artifacts, the researchers introduced new components into the GigaGAN image architecture that improve both temporal consistency and image quality.
- Experiments show that compared to previous methods, VideoGigaGAN finds a good trade-off between temporal consistency and image detail, and produces videos with significantly finer detail than the state of the art, even at eight times the resolution.
Researchers from Adobe and the University of Maryland built VideoGigaGAN, a new model for Video Super Resolution (VSR). It can take low-resolution video and turn it into higher-resolution video, adding fine detail while maintaining frame consistency.
Other ways to upscale video often use regression-based networks that make the results look blurry. VideoGigaGAN instead is based on GigaGAN, which is really good at upsampling images.
But the researchers found some problems using GigaGAN for VSR, such as flickering and aliasing between frames. To fix this, they added new parts to GigaGAN that make the frames more consistent and higher quality.
Video: Xu et al.
Tests show that VideoGigaGAN balances image consistency and detail better than previous methods, and that the model produces video with far more detail than the current best options. VideoGigaGAN can increase video resolution by a factor of 8 by adding more and better matching detail to the scene.
However, this also means that the video is to some extent AI-generated and no longer fully represents reality if that is a concern. The model also has some limitations for long videos because of errors that spread across frames, and for small things like text that get lost in the low-res input.
You can see many demos and comparisons with other methods on the VideoGigaGAN project website. It's not clear from the paper if and when Adobe will incorporate this model into its products. But it might, since it recently announced that it is adding more generative AI to its video suite.
Overall, VideoGigaGAN is a promising new way to create high-resolution video. It can add more detail than older methods without losing consistency between frames by using GAN technology. Like GigaGAN for images, VideoGigaGAN proves that GANs are far from obsolete.
AI News Without the Hype – Curated by Humans
As a THE DECODER subscriber, you get ad-free reading, our weekly AI newsletter, the exclusive "AI Radar" Frontier Report 6× per year, access to comments, and our complete archive.
Subscribe now