Ad
Skip to content
Subscribe Now
AI in practice
Copy the url to clipboard Share this article Go to comment section

Resemble AI's open-source model transforms noisy audio into crystal-clear speech

Maximilian Schreiner
Maximilian Schreiner View the LinkedIn Profile of Maximilian Schreiner
Dec 28, 2023
Image description
DALL-E 3 prompted by THE DECODER

Resemble Enhance is an open-source AI model that can significantly improve the quality of audio recordings.

The startup Resemble AI offers several AI tools for voice cloning, blending, and localization, as well as text-to-speech, speech-to-speech, and voice dubbing capabilities for various applications.

Now, the company has released Resemble Enhance, an AI model that converts noisy audio into clear speech. Unlike the company's other models, Resemble Enhance is open source.

Resemble Enhance for podcasts and historical recordings

Resemble sees applications for the technology in areas such as podcasting, the general entertainment industry, and the restoration of historical audio documents. The company shows what this sounds like with an example of an old lecture.

Ad
DEC_D_Incontent-1

Video: Resemble AI

The model consists of two main components: a denoiser and an enhancer. The denoiser uses a UNet model to separate speech from background noise to improve intelligibility. The enhancer uses a latent conditional flow matching (CFM) model to correct audio distortion and expand audio bandwidth.

The development team plans to continue improving Resemble Enhance, including optimizing processing times and extending control over individual speech elements to further improve audio quality. In the long run, the model should also be able to improve audio recordings that are more than 75 years old.

Resemble offers a demo of Resemble Enhance on HuggingFace. The code is available on GitHub.

Ad
DEC_D_Incontent-2

AI News Without the Hype – Curated by Humans

As a THE DECODER subscriber, you get ad-free reading, our weekly AI newsletter, the exclusive "AI Radar" Frontier Report 6× per year, access to comments, and our complete archive.

AI news without the hype
Curated by humans.

  • Over 20 percent launch discount.
  • Read without distractions – no Google ads.
  • Access to comments and community discussions.
  • Weekly AI newsletter.
  • 6 times a year: “AI Radar” – deep dives on key AI topics.
  • Up to 25 % off on KI Pro online events.
  • Access to our full ten-year archive.
  • Get the latest AI news from The Decoder.
Subscribe to The Decoder