Content
summary Summary

RecolorNeRF can change the color of objects or lighting in NeRFs. This enables easy image processing for photorealistic 3D scenes.

Neural Radiance Fields learn 3D representations of a scene, replacing older methods such as photogrammetry. Once learned, the scene can be rendered from different angles or exported as a 3D object.

Researchers at the University of Hong Kong and the University of Hong Kong (Shenzhen) now present RecolorNeRF, a method for selectively changing the colors in the neural network.

Basic Photoshop for NeRFs

According to the team, RecolorNeRF is the first approach that can directly modify photorealistic NeRFs in high quality. The researchers divide the scene into different layers on a learned color palette, which they can then modify. In this way, they can edit the NeRF directly and then render the scene as usual.

Ad
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
RecolorNeRF learns a color palette during training and decomposes the scene into different layers corresponding to these colors. This color palette can then be modified and the modified NeRF rendered in the new color palette.| Image: Gong, Wang et al.

In the paper, the team shows that the method produces realistic results even in complex real-world scenes. For example, RecolorNeRF changes the color of a car or the green grass behind the car.

Semantic analysis for complex NeRF processing

RecolorNeRF provides an easy way to change colors in a NeRF. Since layers are only decomposed at the color level, however, the method modifies the color of all objects that have the same color. Currently, it is not possible to change only a single object.

Still, methods like RecolorNeRF show the potential of NeRFs in the computer graphics world. In the future, more complex NeRF processing will be possible.

The Chinese team is also planning improvements to RecolorNeRF: semantic segmentation could decompose the scene into multiple layers based on semantic information instead of just a color palette, allowing for targeted processing of individual objects in the future.

More examples and the code are available on the RecolorNeRF project page.

Recommendation
Ad
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Researchers present a method for editing colors in a NeRF.
  • RecolorNeRF divides a 3D scene into layers sorted by a color palette. These colors can then be changed and the NeRF can be edited directly.
  • RecolorNeRF can change the color of a car in a photorealistic scene.
Max is managing editor at THE DECODER. As a trained philosopher, he deals with consciousness, AI, and the question of whether machines can really think or just pretend to.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.