Original article from June 19, 2024:
Luma AI adds "Extend Video" to the capabilities of its Dream Machine. This feature allows videos to be extended based on prompts, with the system considering the new context when extending the video. According to Luma AI, the maximum length of a clip that has been extended multiple times is currently 1:20. Previously, clips were limited to five seconds.
Video: Luma AI
Standard, Pro and Premier users can now remove watermarks. Luma AI also announces that the Dream Machine will soon be equipped with a range of editing features and intuitive prompt controls.
Video: Luma AI
Original article from June 12, 2024:
Luma AI unveils "Dream Machine", an AI video generator you can try now
AI startup Luma Labs has revealed Dream Machine, a video generator built to create high-quality, realistic videos from text prompts and images. The tool is now available for testing.
Luma Labs says Dream Machine is built on a scalable, efficient, and multimodal transformer architecture and directly trained on video. This allows it to generate physically accurate, consistent, and even action-packed scenes.
The startup claims that the model understands how people, animals, and objects interact with the physical world, resulting in high character consistency and correct physics. The videos are prompted by text or image input.
Video length is currently limited to five seconds. A video of up to 120 frames can be generated in as little as 120 seconds. Dream Machine also aims to allow experimentation with a variety of fluid, cinematic and natural camera movements.
There are limitations in terms of consistency, such as objects changing while moving, movement in general, text rendering and spelling, and an effect the developers have named "Janus" after the two-headed Roman god. Generated objects can (temporarily) have two heads. Luma plans to improve the model.
Luma AI promises a new set of leading generative models
Dream Machine is set to be the first in a series of "Frontier Generative Models" by Luma AI that will help users generate new content from images, videos, text, and other inputs. Initial reactions to the video model have been "enthusiastic," according to the startup. A test version is available on the site, but you need a Google Login.
LumaLabs AI is an AI startup mainly known for applying generative AI in the 3D modeling process, particularly with NeRFs. With Luma's video-to-3D API and web interface, developers and end users can create 3D models from images or videos. Luma AI also offers a text-to-3D interface called Genie, which generates 3D models from text descriptions.
With its video generator, the startup is entering a new field that recently saw a significant leap in quality with Sora and KLING - at least in demos. None of the generators are yet publicly accessible.
The current leading video generator available, RunwayML, is likely to follow soon with a new model, as CEO Cristóbal Valenzuela recently hinted. Questions remain about the consistency of quality, as well as the cost, effort, and time required for generation.
