Researchers from Nvidia, the University of Toronto, and the Vector Institute present Adaptive Shells, a method that takes NeRFs to the next level.
Neural Radiance Fields (NeRFs) enable impressive image quality in 3D graphics, but in their traditional form, they are computationally intensive because they require a large number of samples to generate high-resolution images. In volume rendering, multiple data points are sampled along each ray, providing details about the color and translucency of the point in space. Particularly complex and irregular structures such as leaves or hair benefit from this volumetric representation and can be rendered realistically. However, many scenes consist mainly of solid surfaces, which could theoretically be rendered with much less effort.
Nvidia's Adaptive Shells builds on this insight and only calculates certain parts of a scene in detail, while other parts are rendered with less detail. Specifically, Adaptive Shells switches between volumetric rendering (for complex structures) and surface-based rendering (for solid surfaces) depending on the content and uses hardware-accelerated ray tracing to collect the necessary information.
Adaptive Shells runs in 4K at over 60 frames per second
According to the researchers, the method allows images to be rendered much faster than alternative methods such as Nvidia's Instant NGP, and with significantly better image quality. In examples, Nvidia shows that Adaptive Shells sometimes renders twice as many frames per second and achieves between 40 and 300 FPS in 1080p, depending on complexity of the scene. In addition, even complex scenes in 4K resolutions with DLSS can achieve frame rates of over 60 frames per second.
The team also shows that Adaptive Shells is capable of rendering individual objects with 'fuzzy' materials such as fur or wool, as well as entire 3D scenes, such as a table in a garden, with a high level of detail. The method also directly constructs an explicit outer shell mesh, which contains all regions of space that contribute to the rendered appearance. The extracted representations can therefore be used for further applications such as animation or physical simulation.
Adaptive Shells seems to be another step towards a transformation of computer graphics in which neural rendering techniques will play an increasingly important role.
More information can be found on the Adaptive Shells project page.
