Nvidia's Avatar Cloud Engine began as an AI service for the service industry. Now, Nvidia is bringing generative AI tools to video games.
Since the breakthrough of large language models, there have been attempts to use such AI systems in video games to enable dynamic dialog with non-player characters (NPCs). At Computex, Nvidia announced that it will open the Avatar Cloud Engine to video games.
Nvidia's ACE consists of three components: NeMo, Riva, and Omniverse Audio2Face. NeMo provides large language models that developers can customize with backstory and dialogue data. Riva can recognize speech and convert text to audio, enabling live conversations with NeMo. Audio2Face converts Riva's audio output into facial animations that can be used, for example, to control MetaHuman characters in Unreal Engine 5 via Omniverse links.
In collaboration with AI startup Convai, Nvidia shows what an ACE-powered MetaHuman could look like in a video game.
According to Nvidia, NPC Jin answers the player's questions in natural language, realistically and in line with a background story. The low latency in the demo is striking, details about the language model and the hardware in the background are not available.
Different models should enable low latencies
"With NVIDIA ACE for Games, Convai’s tools can achieve the latency and quality needed to make AI non-playable characters available to nearly every developer in a cost efficient way," said Purnendu Mukherjee, founder and CEO of Convai.
According to Nvidia, the AI models in ACE vary in size, performance, and quality. "The ACE for Games foundry service will help developers fine-tune models for their games, then deploy via Nvidia DGX Cloud, GeForce RTX PCs or on premises for real-time inferencing." The models are optimized for latency, a "critical requirement for immersive, responsive interactions in games," the company said.
GSC Game World is one of the first development studios to use at least one of ACE's tools: Audio2Face will be used in S.T.A.L.K.E.R. 2 Heart of Chernobyl. However, AI-generated stories will not be told by NPCs around the campfire for the time being.
Nvidia's DGX GH200 is like a giant GPU
In addition to ACE for Games, Nvidia announced a number of other collaborations and products, including the DGX GH200 AI supercomputer with 256 Grace Hopper chips totaling 144 terabytes of memory and exaflop AI performance. According to Nvidia, the DGX GH200 GPT-3 trains 2.2 times faster than a DGX H100 cluster.
According to Nvidia CEO Jensen Huang, Google, Meta, and Microsoft are among the first companies to gain access to the DGX GH200 and evaluate the platform's capabilities.