summary Summary

Nvidia's CEO Jensen Huang announces several innovations for Nvidia's AI business during his GTC keynote. Here is the most important information at a glance.


In March, Nvidia unveiled the Hopper H100 GPU for the first time. The AI accelerator is expected to replace Nvidia's successful A100 model and showed speeds up to 4.5 times faster than A100-based systems in the latest MLPerf benchmark.

During the GTC keynote, Nvidia CEO Jensen Huang now announced the sale of the first H100 systems via partners starting in October. The company's own DGX H100 systems are scheduled to ship in the first quarter of 2023.

With a mixed AI workload of training and inference of different AI models, an AI data center with 64 HGX H100 systems is expected to achieve the same performance as an A100 data center with 320 HGX A100 systems.


This leap is possible in part because of the integrated Transformer engine, which can process Transformer AI models faster.

Nvidia announces NeMo language model service

According to Nvidia, large-scale language models are taking on an increasingly important role in AI research and the AI market. The number of Transformer and language model papers has grown rapidly since 2017. Hopper was therefore explicitly designed for such models, Nvidia said.

In practice, Hopper is said to speed up the training of such models by five times and inference by 30 times. With ten DGX-A100 systems, it is currently possible to serve ten users simultaneously with large language models, and with ten DGX-H100 systems, it is 300, according to the company.

Nvidia sees numerous applications for large language models. The company is launching new cloud services to provide easy access to the models. | Image: Nvidia

This performance enables the use of these models for specific applications, areas, and experiences, according to the company. Nvidia offers both the corresponding hardware and now a cloud service for large language models.

With Nvidia NeMo, companies can adapt and use large language models such as Megatron or T5 to their needs through prompt training. This puts Nvidia in direct competition with other providers of large language models. The focus is on enabling enterprises with concrete products, Nvidia said.


Nvidia brings AI hardware and models to medicine

An important application area for such large language models could be medicine. Specialized variants of the models predict molecules, proteins, or DNA sequences, for example. Chemistry and biology have their own language, according to Nvidia.

With the BioNeMo cloud service, the company brings pre-trained models for chemistry and biology applications, such as ESM-1b, a language model trained with protein databases, or OpenFold, a model that predicts protein folding.

BioNeMo launches in early access in October and is part of Nvidia's Clara framework for AI applications for healthcare. Nvidia is also bringing other models for specific applications in medicine, such as image recognition via Clara Holoscan.

The company also announced a collaboration with the Broad Institute to bring Clara to the Terra Cloud Platform, an open-source platform for biomedical researchers.

Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.

IGX systems for medicine and autonomous factories

For the medical market, Nvidia showed the IGX system, an all-in-one platform for developing and deploying AI applications in smart machines. IGX is said to enable real-time image analysis via Holoscan, for example. Companies such as Activ, Moon Surgical, and Proximie are expected to use Holoscan with IGX for their new medical platforms.

IGX is expected to accelerate AI models for medicine and smart factories. | Image: Nvidia

The IGX systems are also to be used in industrial automation, where they will enable smart spaces in which cameras, LIDAR, and radar sensors are processed simultaneously. Smart spaces in factories could reduce safety risks because companies no longer have to rely exclusively on sensor technology in robots, Nvidia said.

Jetson Orin Nano and Thor for the Edge

For edge computing, Nvidia pointed to the new Jetson Orin Nano, which can be installed in robots, for example. Orin Nano is supposed to offer 80 times higher performance compared to its predecessor. The AI models for Orin Nano can be trained in Nvidia's Isaac Sim robot environment.

In the autonomous vehicle market, Nvidia last year showed the Atlan chip for 2024 with 1,000 TOPS of computing power, four times the current Orin chip. However, autonomous vehicles need far more computing power to be even safer, Nvidia said. Therefore, the company decided to develop the Thor chip, which delivers a full 2,000 TOPS.

Drive Thor is a computer system for autonomous vehicles with an AI chip, GPU, and CPU from Nvidia. | Image: Nvidia

Thor is scheduled to hit the market in 2024 and will be installed in Nvidia Drive Thor system. Drive Thor is a centralized computer for autonomous vehicles that includes the Thor chip, a next-gen GPU, and a Grace CPU. Nvidia wants to make other computer systems redundant with it. Drive Thor is supposed to be able to run Linux, QNX, and Android and comes with the Transformer engine of the Hopper GPUs.

According to Nvidia, the first company to opt for Thor was car manufacturer Zeekr. In total, 40 companies use the Drive platform. It is also designed for training autonomous vehicles and receives updates such as a neural engine that can replicate real-world driving in a realistic simulation. This should simplify the training of edge cases, for example.

Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
  • Nvidia shows new products and services during CEO Jensen Huang's keynote.
  • The Hopper GPU for AI calculations is in production, first systems will ship in October.
  • Nvidia launches NeMo cloud service for large-scale language models.
  • The BioNeMo cloud service, also new, specializes in models for biology and chemistry.
  • The new IGX platform is designed to accelerate AI applications in healthcare and smart factories.
  • Jetson Orin Nano and Drive Thor are set to become central computers for robots and autonomous vehicles.
Max is managing editor at THE DECODER. As a trained philosopher, he deals with consciousness, AI, and the question of whether machines can really think or just pretend to.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.