Meta has introduced a new AI system called 3D Gen that can create high-quality 3D objects from text descriptions in less than a minute.
Meta's new AI tool for generating 3D content, named 3D Gen, can produce high-quality 3D objects from text descriptions in under a minute, according to the company.
3D Gen combines two of Meta's existing models: AssetGen for generating 3D objects and TextureGen for texturing. Meta says this integration leads to higher-quality 3D generation for immersive content.
Video: Meta
Meta's 3D Gen outperforms other methods and is up to 60 times faster
The system works in two steps: In the first step, AssetGen generates a 3D object with texture and Physical Based Rendering (PBR) from a text description in about 30 seconds. In the second step, TextureGen can either optimize the texture of the object generated in step 1 based on the original description or create a texture for any unstructured 3D mesh using a new specification. Step 2 takes about 20 seconds each. Since 3D Gen supports PBR, relighting the generated assets is possible.
Both stages are based on Meta's text-to-image models from the Emu family. These are optimized with synthetic 3D data for multi-view generation in image and UV space, which the team says improves textures.
According to Meta, 3D Gen outperforms leading industry solutions in terms of speed and quality. In user studies, professional 3D artists rated the tool better than the competition in most categories, especially for complex requests, the paper states. Meta says it is 3 to 60 times faster than comparable systems.
Meta sees 3D Gen as an important step toward personalized, user-generated 3D content. This content could be generated by AI-assisted 3D assistants and used for building virtual worlds in the metaverse.
There are additional papers for AssetGen and TextureGen.
