Using thermal and infrared data combined with AI processing, a new visual system can see its surroundings in total darkness as it would in daylight.
HADAR (heat-assisted detection and ranging) combines thermal physics, infrared imaging, and machine learning to pave the way for "fully passive and physics-aware machine perception," according to the Purdue University research team.
Traditional sensors such as LiDAR, radar, and sonar have problems with signal interference and eye safety risks. Video cameras, on the other hand, perform poorly or not at all in low-light conditions such as night, fog, or rain. However, they do provide a realistic image of the environment.
Thermal imaging systems, meanwhile, work well in the dark and are largely independent of weather and light. But the images captured are blurry, lack depth and provide only an abstract picture of the environment.
Robust vision system for robots and autonomous vehicles that is still too slow and bulky for everyday use
The HADAR system aims to combine the best elements of both visualization technologies: Visual data from the robust thermal imaging system is enriched with depth data from an infrared system, and artificial intelligence is used to add more detail, depth, and texture.
This allows HADAR to see the world as clearly in total darkness as it does in daylight, according to the research team. It can also detect physical features beyond the range of human vision.
But before HADAR can be built into cars or robots for everyday use, the system needs to get smaller and faster. Currently, it can only capture about one frame per second.
For an image that is reasonably smooth to the human eye, at least 24 frames per second are needed; for safety-related applications, such as autonomous driving cars, 30 frames per second or more would be better. In addition, the system is still too expensive, says Fanglin Bao, a researcher involved in its development.
In addition to autonomous driving and robotics, the research team sees potential applications in agriculture, defense, health care, geosciences, and wildlife monitoring.
The research is supported by the U.S. defense agency DARPA, among others. A patent application is in the works.