Content
summary Summary

Large language models enable more realistic dialog in video games. The company Replica, which offers an AI voice plugin for 3D engines, has found a suitable application.

Ad

Generic non-player characters (NPCs) in video games speak few, if any, sentences with generic phrases. That's because someone has to write them down and then record them. This is a lot of work for a feature that is not central to most games, but can be quite costly, depending on the scale of your game.

Smart NPCs could be a building block for the gaming future

Large language models combined with high-quality artificial voices eliminate both bottlenecks: Using game world-specific prompts or fine-tuning with the video game script, the LLM generates meaningful sentences that can be lip-synced by NPCs with matching body language using the AI voice.

This allows for realistic, flexible dialog between your character and NPCs, further enhancing your immersion in the world. This is exactly what Replica promises with its "AI Voice Plugin" for Unity and Unreal, which is compatible with various LLMs.

Ad
Ad

It is obvious that the current system of populating video games with characters and creating immersion through storytelling is not easily scalable. We see the potential for AI to solve many of these challenges and allow even small teams to think big and make their games epic adventures.

Replica

You can download and try the Matrix demo here. Since the sentences for the AI voices in the demo are generated using a cloud language model, there may be some latency in the dialog. This is one of the problems that still needs to be solved to make intelligent NPCs more widespread. Many more AI demo dialogs can be seen in the video below.

The next step would be to have these dialogs dynamically affect gameplay in flexible game worlds: For example, if you insult an NPC in GTA 6, you will receive an appropriate response depending on the severity of the insult - or a ticket with a fine.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Large language models and artificial voices enable more realistic and flexible dialog between player characters and generic non-player characters (NPCs) in video games, as demonstrated by Replica's AI Voice Plugin" for Unity and Unreal.
  • Through the use of AI-generated content and lip-synced body language, enhanced NPCs contribute to immersion and storytelling, enabling epic gaming experiences for smaller development teams.
  • Challenges such as dialog latency when using cloud language models and game world impact still need to be addressed before smart NPCs make sense for widespread adoption in game worlds.
Sources
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.