Researchers at Nvidia demonstrate a new AI method designed to enable the efficient use of artificial intelligence for computer graphics.
Artificial Intelligence is finding its way into computer graphics, from AI upscalers like Nvidia's DLSS to neural rendering via GANs or 3D representations in NeRFs to ray-tracing helpers like Neural Radiance Caching (NRC). Nvidia showed some of these approaches at the SIGGRAPH 2021 tech conference in August 2021.
For example, NRC combines the AI and raytracing hardware built into RTX graphics cards to create AI-powered global illumination that works with all types of materials and textures. To do this, NRC learns to predict the paths and ray-tracing effects of path-traced light rays in real time during the rendering process of a scene. This improves rendering efficiency by up to a factor of 100 when combined with advanced lighting algorithms.
Neural Reflectance Field Textures (NeRF-Tex), on the other hand, are designed to simplify the modeling of complex materials such as fur or fabric. NeRF-Tex represents these materials and can be placed as a texture over a classic mesh.
AI approaches have not been fast enough - so far
Artificial neural networks can be used to realize numerous other applications in computer graphics. But in many cases, AI solutions are still slower than their traditional rendering alternatives. A new research paper from Nvidia now introduces a new method that could change that.
Instant-NGP can learn gigapixel images, SDFs, NeRFs and neural volumes in seconds. | Video: Nvidia
The research group demonstrates "Instant Neural Graphics Primitives" (Instant-NGP), a framework that allows a neural network to learn representations of gigapixel images, 3D objects, and NeRFs in seconds.
Neural Radiance Caching, which was introduced in August, also benefits from the new method and achieves significantly better image quality with only minor performance degradation.
Instant-NGP speeds up NeRF training many times over. | Video: Nvidia
A real-world model trained in under five minutes from just 34 photos. | Video: Nvidia
The researchers rely on a combination of a learned hash table of voxel vertices and a neural network. This allows higher quality with up to eight times faster training compared to alternative methods.
Video: Nvidia
Instant NGP aims to pave way for the widespread use of AI technology for computer graphics
Many problems in computer graphics previously required specific data structures to solve the problem at hand, the team said. The approach presented should offer a practical, learning-based alternative that automatically focuses on relevant details, regardless of the task.
Moreover, because the method is so efficient, it can be used in online training and inference, matching the performance of traditional 3D reconstruction techniques, they said.
"We have demonstrated that single-GPU training times measured in seconds are within reach for many graphics applications, allowing neural approaches to be applied where previously they may have been discounted," the researchers conclude.
More information about Instant-NGP is available on the project page. The code is available on GitHub.
