Extremely miniaturized cameras enable new applications in medicine or robotics. Thanks to AI, researchers are now reaching an important milestone.
The steady miniaturization of cameras in recent decades has enabled their widespread use in medical imaging, security technology, smartphones, robotics or autonomous vehicles. Cameras that are an order of magnitude smaller could also enable numerous new applications in minimally invasive medicine, nanorobotics, wearables, or AR and VR. However, this further miniaturization is hindered by fundamental limitations of current approaches, such as the use of lenses.
Researchers are therefore working on entirely new camera approaches, including so-called meta-surface optics (meta-optics). In this approach, tiny cylindrical structures that vary in geometry are attached to a surface. These structures capture incoming photons similarly to small antennas. The signals are then transmitted to a computer and converted into an image. However, previous systems produce blurry, distorted images and have an extremely small field of view.
As small as a grain of salt: neuronal nano-optics
Researchers at Princeton University and the University of Washington are now overcoming these limitations with a camera the size of a coarse grain of salt. The new camera system can produce sharp, full-color images comparable to a lens 500,000 times larger than a conventional camera, the team says.
"It was a challenge to design and configure these small nanostructures to do what you want them to do," said Ethan Tseng, a computer science doctoral student at Princeton who co-led the study. "For this particular task of capturing RGB images with a large field of view, it was previously unclear how to design the millions of nanostructures along with the image processing algorithms."
The researchers developed the nano-optics with the help of artificial intelligence. They automated the testing of different nanoantenna configurations and customized imaging algorithms with an AI algorithm. Because of the number of antennas and the complexity of their interactions with light, simple automated testing without AI help would have required "enormous amounts of memory and time" according to the researchers.
Meta-optics as a smartphone interface
The integrative design of the meta-optics, with 1.6 million structures and an image-processing algorithm, is one of the key innovations, said Felix Heide, senior author of the study and an assistant professor of computer science at Princeton. For example, the tiny camera's performance under natural lighting conditions increased significantly. Previous systems required pure laser light or other artificially created ideal conditions.
Video: Princeton Computational Imaging Lab
Heide and his team now want to expand the camera's capabilities. They like to improve image quality and add capabilities such as object recognition to the camera, so it can be used in applications such as medicine and robotics.
Meta-optics could also turn entire surfaces into sensors: "We could turn individual surfaces into ultra-high resolution cameras, so you wouldn't need three cameras on the back of the smartphone, but the entire back would become one big camera." Meta-optics will enable entirely new ways to design products, the researchers said.
The footage and code are available on Github.
