An AI system from Google simulates DOOM in real time, opening new avenues for game development.
Researchers from Google Research, Google DeepMind, and Tel Aviv University have created GameNGen, an AI system that can fully simulate and play back the classic game DOOM in real time. This breakthrough could pave the way for AI-assisted game development.
GameNGen can simulate DOOM at over 20 frames per second on a single Google TPU chip. The simulation's image quality achieves a Peak Signal-to-Noise Ratio (PSNR) of 29.4, comparable to lossy JPEG compression. In tests, human evaluators could only distinguish short sections of the simulation from actual gameplay slightly better than chance.
The system was trained in two stages: First, an AI agent learned to play DOOM, with training sessions recorded. Then, a diffusion model was trained to generate the next image based on previous images and actions.
GameNGen can perform complex game state updates, including tracking health and ammo, picking up weapons, attacking enemies, damaging objects, and opening doors. It can maintain the game state for extended periods, with researchers demonstrating clips lasting around 2.5 minutes.
Video: Valevski et al.
However, the system has limitations. Its "memory" is restricted to about 3 seconds, meaning it can't account for events that occurred more than 3 seconds ago. This constrains its ability to recognize and respond to long-term relationships in the game.
GameNGen demonstrates a new paradigm for game engines
GameNGen improves upon previous approaches like GameGAN or GAN Theft Auto in terms of simulated game complexity, simulation speed, long-term stability, and visual quality.
The researchers view GameNGen as a significant step towards a new paradigm for game engines, where games could be automatically generated by neural models. However, questions remain about how to train such neural game engines and create games effectively, including optimal use of human input.
More examples and the code are available on GitHub.
