Content
summary Summary

Researchers from Adobe and the University of Maryland built VideoGigaGAN, a new model for Video Super Resolution (VSR). It can take low-resolution video and turn it into higher-resolution video, adding fine detail while maintaining frame consistency.

Ad

Other ways to upscale video often use regression-based networks that make the results look blurry. VideoGigaGAN instead is based on GigaGAN, which is really good at upsampling images.

But the researchers found some problems using GigaGAN for VSR, such as flickering and aliasing between frames. To fix this, they added new parts to GigaGAN that make the frames more consistent and higher quality.

Video: Xu et al.

Ad
Ad

Tests show that VideoGigaGAN balances image consistency and detail better than previous methods, and that the model produces video with far more detail than the current best options. VideoGigaGAN can increase video resolution by a factor of 8 by adding more and better matching detail to the scene.

However, this also means that the video is to some extent AI-generated and no longer fully represents reality if that is a concern. The model also has some limitations for long videos because of errors that spread across frames, and for small things like text that get lost in the low-res input.

You can see many demos and comparisons with other methods on the VideoGigaGAN project website. It's not clear from the paper if and when Adobe will incorporate this model into its products. But it might, since it recently announced that it is adding more generative AI to its video suite.

Overall, VideoGigaGAN is a promising new way to create high-resolution video. It can add more detail than older methods without losing consistency between frames by using GAN technology. Like GigaGAN for images, VideoGigaGAN proves that GANs are far from obsolete.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Adobe researchers demonstrate VideoGigaGAN, a new generative model for video super resolution (VSR) that can scale low-resolution video to higher resolution while maintaining fine-grained detail and temporal consistency.
  • To address problems such as temporal flicker and aliasing artifacts, the researchers introduced new components into the GigaGAN image architecture that improve both temporal consistency and image quality.
  • Experiments show that compared to previous methods, VideoGigaGAN finds a good trade-off between temporal consistency and image detail, and produces videos with significantly finer detail than the state of the art, even at eight times the resolution.
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.