Nvidia researchers have developed a method to render complex materials in real-time using artificial neural networks. This technique enables photorealistic renderings at significantly higher speeds than conventional methods.
The new approach, called "Real-Time Neural Appearance Models," allows for high-detail rendering of materials like ceramics, metals, and plastics in real-time. It uses artificial neural networks to simulate the optical properties of surfaces.
According to the authors, this new method can render materials up to 10 times faster than traditional techniques while maintaining visual quality. Nvidia first demonstrated the method last year and presented it in detail at this year's SIGGRAPH conference.
The system consists of several components: An encoder first converts the material properties stored in a hierarchical texture. Two neural decoders then interpret this to calculate reflection properties and sampling directions. The researchers also incorporate certain graphical priors in the networks to make the method more efficient.
Real-time neural appearance models provide a high level of detail in real time
To run the neural networks in real-time shaders, the developers made special optimizations. Depending on the execution situation, calculations are performed either on tensor cores or with classic shader instructions.
In test scenes, the AI materials reduced rendering time by a factor of 1.6 to 4.1 compared to optimized classical implementations, depending on the size of the network, achieving rendering times of less than 10 milliseconds. Visual quality remained largely preserved.
The researchers see their system as a foundation for using film-quality materials in real-time applications such as games or interactive previews. However, there are still limitations, such as with refraction effects.
More information and examples of the new method are available on Nvidia's project page.
