EfficientViT brings massive speedup to computer vision

EfficientViT brings efficient computer vision to high-resolution cameras and could lead to more accurate results in autonomous driving or medicine.

MIT researchers have unveiled a new computer vision model that can perform semantic segmentation of high-resolution images much more efficiently than previous models. Semantic segmentation involves tagging and categorizing all pixels in an image. With this, autonomous vehicles can accurately recognize objects such as other cars, pedestrians, and their surroundings, or medical AI systems can recognize tumor tissue.

EfficientViT reduces computational complexity of vision transformers

The new model, called EfficientViT, is based on the Vision Transformer but changes the attention mechanism of the model. This is because conventional vision transformers suffer from the same drawback as classical transformers for language processing: the computational complexity of the attention mechanism increases quadratically, making the processing of high-resolution images extremely computationally intensive and thus making real-time processing impossible on current edge devices.

For this, the researchers' model, EfficientViT, simplifies the process of creating "attention maps," representations that capture the relationships between different parts of the image. These help the model form a "global receptive field," meaning it can access all relevant parts of the image.