Nvidia has announced major enhancements to its Jetson platform for edge AI and robotics, including generative AI for industrial robots, manufacturing systems, and other edge devices.
According to the company, generative AI on robots and edge devices enables learning with few, and in some applications, no examples. For example, vision transformers could more easily identify new objects without having to train a new model. This could make systems more adaptive and responsive.
Natural language interfaces also simplify the development of AI applications by allowing users to control them through text input. In one example, the company shows how a computer vision system is prompted to recognize new objects via a simple text description.
According to Nvidia, the Jetson platform now has more than 10,000 customers, including Amazon Web Services, Cisco, PepsiCo and Siemens.
Jetson Generative AI Lab for Open-Source Models
The company is also introducing the Jetson Generative AI Lab, a platform where in-house developers can use the latest open source generative AI models on the Jetson platform for edge AI and robotics.
The Nvidia TAO toolkit for edge model optimization also gets an update with VisualChangeNet, a new transformer model for defect detection. In a press briefing, the company also emphasized that Nvidia Orin is the only edge AI hardware that can run many of the foundation models, including large language models such as Meta's LLaMA 2 in its 70 billion parameter variant.
Major enhancements to robotics and video AI platforms
The Isaac ROS 2.0 and Isaac Sim 2023.1 robotics and simulation frameworks are now generally available and are designed to simplify the development of AI-based robotics applications through improvements in the areas of perception and simulation. Metropolis, an industrial plattform for vision AI applications, will also be enhanced with new APIs and microservices by the end of the year.
Nvidia's JetPack will also be released in version 6.0, with a host of new features. According to Deepu Talla, vice president and general manager of autonomous machines at Nvidia, the updates are the result of years of work. To simplify development for customers, Nvidia is providing complete reference workflows for areas such as video surveillance, robotic inspection, and mobile robots.