Generative AI can create text, audio, images, and 3D assets. The games industry combines these media - and could therefore benefit greatly.
At least that's what venture capitalist Andreessen Horowitz (a16z) assumes, making it the second major Silicon Valley investor after Sequoia Capital to embrace the potential of generative AI. In a blog post, the company specifically addresses the benefits of generative AI for the gaming industry, where it expects the technology to have the most significant impact.
Magic potions made by machines
The investors' basic thesis: Games are the most complex form of media and therefore have a particularly high barrier to entry in development. Generative AI lowers this barrier by reducing complexity in asset production while increasing quality, quantity, and speed.
"There hasn’t been a technology this revolutionary for gaming since real-time 3D," write authors James Gwertzman and Jack Soslow. "There hasn’t been a technology this revolutionary for gaming since real-time 3D."
Initial conversations with developers have shown that they are excited about the "dramatic reduction in time and cost," they say. One developer says he has reduced the time it takes to complete a single piece of concept art from three weeks to one hour. The venture capitalist expects similar savings rates across the entire production pipeline.
The developer "Emm" shows concrete examples on Twitter. He focuses on the production of 2D assets with Stable Diffusion. For this, he fine-tunes Stable Diffusion with sample assets using the Dreambooth API. This allows him to generate game-specific and consistent graphics. With Scenario_gg, he plans to offer a platform for AI-generated game assets starting in December.
.@Scenario_gg lets you create infinite consistent content, similar to what I described in my recent posts (swords, chests, books, characters, potions, cards & more)
Game creators can create incredible, high-quality assets in hours (vs. days or weeks using a traditional process) pic.twitter.com/14JNvBmhxF
— Emm (@emmanuel_2m) November 17, 2022
While the productive use of generative AI for concept art, 2D assets and text is already underway, 3D objects are still further away. Early approaches include Google's Dreamfusion research project and Nvidia's GET3D, which is open-source.
More games, new games and micro studios
The following graphic from a16z shows an overview of commercial projects and startups working on generative AI that could be of interest to gaming studios. Despite all the enthusiasm, generative AI for gaming is in its infancy, a16z writes. In addition to large foundational models like Stable Diffusion, AI tools developed specifically for the industry could add value.
The venture capitalist expects the use of generative AI to become a professional discipline in its own right. Moreover, if generative AI lowers barriers to entry, it could lead to riskier and more creative gaming projects.
Another possible outcome of the generative AI revolution could be "micro game studios" built on AI technology that have only one or two employees but still create extensive and elaborate games.
This in turn would lead to an even higher number of games - the marketing problem of games would thus be exacerbated - but possibly also to the development of new game ideas and genres.
For example, a16z mentions the role-playing game Arrowmancer, in which players can experience new adventures with infinite AI-generated characters.
An even better example is the text-based RPG AI Roguelite, where all locations, characters, enemies, and items are generated by GPT-3 and VQGAN-CLIP (see video). A pioneering project in the field of AI-generated games is the text and image adventure AI Dungeon.
Andreessen Horowitz sees potential stumbling blocks in copyright issues. Large game studios in particular are unlikely to make their resources available free of charge for training or fine-tuning publicly available AI models.
Stable Diffusion, for example, is trained with datasets that integrate unauthorized copyright-protected material. Stable Diffusion co-inventor Emad Mostaque recently announced that future training datasets should include only licensed material and offer an opt-out option. So solutions are probably already on the horizon here.
Andreessen Horowitz sees the impact of generative AI primarily in the creative field. Game programmers should have greater job security despite increasing code automation with tools like Copilot. Code generation still requires more human input and therefore has less impact on productivity, the firm says.
AI already offers another benefit for games: in rendering graphics. Currently, the most popular example is Nvidia's DLSS, which uses AI upscaling to get significantly more performance out of the same hardware. A further spread of such methods with increasing efficiency could lead to even faster development of sophisticated real-time 3D graphics.
