Pika Labs AI video generator now includes sound effects and lip-sync voices

Until now, AI videos have been silent movies. But that is about to change: Pika Labs is introducing a new generative audio model.

Pika Labs has introduced text prompt-based sound effect generation for its generative AI videos. With this feature, users can add simple sounds to their videos: sizzling bacon, screeching eagles, or roaring engines.

The audio generation is still independent of the video content, so the text prompt that guides the video generation also generates the audio, which is then overlaid on the video. Alternatively, you can enter specific audio prompts to better control the generation. Pika Labs says it has trained its own model for audio generation.

Future models could perform audio and video generation in a single step, automatically generating the sounds that match the video based on an analysis of individual video frames, and inserting them in the right places. In large language models such as GPT-4, visual comprehension is good enough for detailed image descriptions.