Content
newsletter Newsletter

With RawNeRF, Google scientists introduce a new tool for image synthesis that can create well-lit 3D scenes from dark 2D photos.

Ad

In the summer of 2020, a research team at Google first introduced "Neural Radiance Fields" (NeRF): Put simply, an AI system recognizes where light rays end in images. Based on this, the system can automatically create a 3D scene from multiple photos of the same 2D scene. Imagine a kind of automated photogrammetry that reduces manual effort and the number of photos while generating flexible, customizable 3D scenes of high quality.

Over the past two years, Google teams have regularly demonstrated new use cases for NeRFs, such as for Google Maps Immersive View or to render Street View in 3D based on photographs and depth data.

RawNeRF processes RAW images

With RawNeRF, the research team around AI researcher Ben Mildenhall now presents a NeRF that can be trained with RAW camera data. This image data contains the full dynamic range of a scene.

Ad
Ad

According to the research team, thanks to RAW data training, RawNERF can "reconstruct scenes from extremely noisy images captured in near-darkness." In addition, the camera viewpoint, focus, exposure, and tone mapping can be changed after the fact.

"Training directly on raw data effectively turns RawNeRF into a multi-image denoiser capable of combining information from tens or hundreds of input images," the team writes.

You can download RawNeRF from Github. There you can also find Mip-NeRF 360, which can render photorealistic 3D scenes from 360-degree footage, and Ref-NeRF.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.