TRIPS is a new approach to point-based radiance field rendering that achieves higher image quality than alternatives such as Gaussian Splatting and runs in real-time.
A team of researchers at the Friedrich-Alexander University of Erlangen-Nuremberg has developed TRIPS (Trilinear Point Splatting), a new approach to point-based radiance field rendering. TRIPS combines ideas from 3D Gaussian Splatting and ADOP, two recent successful approaches in this field, to render sharp, complete, and alias-free images. The goal of these and other methods is to find more efficient alternatives to the often computationally intensive NeRFs.
According to the team, Gaussian Splatting has trouble rendering detailed scenes in practice, frequently struggling with blur and cloudy artifacts. ADOP can produce better images, but the neural reconstruction it uses reduces performance. The method also suffers from temporal instability and large gaps in the point cloud. TRIPS aims to solve these problems and, according to the researchers, outperforms Gaussian splatting and ADOP in terms of image quality and performance.
TRIPS uses trilinear splatting for efficient 3D representation
At its core, TRIPS works with a concept known as "point-based rendering". This means that 3D scenes are not represented by traditional geometric shapes such as triangles, but rather as a point cloud. TRIPS uses a technique where these points are arranged in a kind of pyramid in the image space (the area you see on the screen). This pyramid consists of several layers with different resolutions, each layer showing a different view of the 3D scene with different details.
TRIPS uses "trilinear splatting" for this. The points are distributed over the levels of the pyramid so that they appear on different levels depending on their size and distance from the camera. Large or nearby points are placed on higher resolution levels, while smaller or more distant points are placed on lower resolution levels. Trilinear splatting is what makes this method so efficient.
After the points are distributed on the pyramid, a small neural network (CNN) is also used. The purpose of this network is to merge the images from the different levels of the pyramid into a single, detailed, and coherent 3D image. It fills gaps and smoothes transitions to create a realistic overall image.
According to the team, TRIPS achieves impressive results when rendering highly detailed interiors and expansive landscapes, with the detailed textures and extensive point clouds rendered in real-time on an Nvidia RTX 4090. Training a scene takes between two and four hours on an Nvidia A100 GPU.
More examples and information can be found on the TRIPS project page. An open-source implementation will also be published on GitHub shortly.
