Large language models enable more realistic dialog in video games. The company Replica, which offers an AI voice plugin for 3D engines, has found a suitable application.
Generic non-player characters (NPCs) in video games speak few, if any, sentences with generic phrases. That's because someone has to write them down and then record them. This is a lot of work for a feature that is not central to most games, but can be quite costly, depending on the scale of your game.
Smart NPCs could be a building block for the gaming future
Large language models combined with high-quality artificial voices eliminate both bottlenecks: Using game world-specific prompts or fine-tuning with the video game script, the LLM generates meaningful sentences that can be lip-synced by NPCs with matching body language using the AI voice.
This allows for realistic, flexible dialog between your character and NPCs, further enhancing your immersion in the world. This is exactly what Replica promises with its "AI Voice Plugin" for Unity and Unreal, which is compatible with various LLMs.
It is obvious that the current system of populating video games with characters and creating immersion through storytelling is not easily scalable. We see the potential for AI to solve many of these challenges and allow even small teams to think big and make their games epic adventures.
Replica
You can download and try the Matrix demo here. Since the sentences for the AI voices in the demo are generated using a cloud language model, there may be some latency in the dialog. This is one of the problems that still needs to be solved to make intelligent NPCs more widespread. Many more AI demo dialogs can be seen in the video below.
The next step would be to have these dialogs dynamically affect gameplay in flexible game worlds: For example, if you insult an NPC in GTA 6, you will receive an appropriate response depending on the severity of the insult - or a ticket with a fine.