Researchers develop Gaussian Frosting, a technique for rendering and editing high-quality 3D effects in real-time.
Scientists at Ecole des Ponts ParisTech in France have presented Gaussian Frosting, an innovative method for rendering and editing complex 3D content. The approach combines 3D Gaussian and classical meshes to capture subtleties and lighting effects in detail.
Gaussian Frosting first extracts a base mesh of Gaussians and then builds an adaptive layer of Gaussians of variable thickness on top. The thickness of the layer varies depending on the material: thinner for flat surfaces, thicker for "fluffy" materials such as hair or grass.
Gaussian Frosting combines neural and classical rendering techniques
Frosting allows for the efficient rendering of complex volumetric effects and can be built from RGB images. It is capable of rendering scenes with realistic detail that often cannot be achieved with conventional rendering techniques. It combines the rendering quality of 3DGS with the editability of mesh-based methods. The approach is similar to Nvidia's Adaptive Shells, which are not based on 3D Gaussians. According to the researchers, Gaussian Frosting achieves higher rendering quality and speed than Nvidia's method.
Compared to SuGaR, which also extracts a mesh from Gaussian splatting, but presses the Gaussians flat onto the mesh, Gaussian Frosting maintains rendering quality for materials and volumetric effects. This can be seen, for example, in the correct display of overlaps and occlusions, which typically lead to errors with other methods.
The method still has some limitations, such as being larger than pure 3D Gaussians. However, the team believes that many of these limitations can be overcome. Frosting could also be useful outside image-based rendering, for example in general computer graphics applications where frosting could render complex materials in real-time.
More examples are available on the project page, with code to follow shortly.