Adobe's generative AI service, Firefly, will become a co-pilot for video production, speeding up the process.
Adobe is following in the footsteps of companies such as RunwayML that are enabling AI-powered video editing and, in some cases, experimenting with text-to-video models. Adobe's plans, unveiled at this year's NAB (National Association of Broadcasters) conference, initially involve integrating the Firefly family of generative AI models into existing workflows.
At NAB, Adobe demonstrated how Firefly can help with video editing, as well as audio, animation, and motion graphics. Firefly is designed to be a creative co-pilot, reducing post-production time from days to minutes.
Adobe Firefly as a co-pilot in video production
The company is currently experimenting with various applications: For example, you can use text to change the time of day or color scheme in a video; generate sounds or short pieces of music; create subtitles, logos, or animations; convert scripts directly into storyboards with initial visualizations; or find appropriate B-roll clips from existing video footage.
Video: Adobe
The new generative AI capabilities for video, audio, animation and motion graphics design are expected to roll out later this year. Those interested can sign up for the Firefly beta. There will also be a live community session on YouTube on Thursday, April 20.
Adobe not bringing text-to-video models for now
A text-to-video model has not yet been officially announced by the company. However, since the Firefly announcement refers to the synthesis of image and text effects as only the first step, it is very likely that Adobe will also work on a text-to-video model at some point in the future. RunwayML is currently testing such a generative model for video with Gen-2.
In addition to the new applications for Firefly, Adobe also announced a text editing feature for Adobe Premiere Pro at NAB. The text editing feature is based on Adobe Sensei and automatically transcribes source media. This allows for faster searches of transcribed footage, as well as text-based editing: Users can paste sentences from the transcript into the timeline, and Adobe Premiere Pro automatically edits the video to match the transcript.
