AI voice clones without regulation pose more social risks than benefits
Video game voice actress Amelia Tyler says a clone of her voice was used to read out rape pornography, according to a report from Eurogamer.
Tyler, who received a BAFTA nomination for her role as the narrator in the video game "Baldur's Gate 3", describes a very negative experience with AI-generated speech. She says her voice and those of other actors in the game were cloned by AI without their consent.
During a livestream, Tyler heard her cloned voice read rape pornography after viewers typed the text into a text-to-speech generator. She explained that she and several other actors have experienced such incidents since the game's release.
While she welcomes players to record their own voiceover versions because she enjoys game modifications, Tyler believes it should be illegal to use her voice for mods without her permission.
She argues that using actors' voices for AI without their consent is not only stealing their work, but also their identity. Tyler calls for a ban on this practice if those involved did not consent.
We have plenty of evidence that voice clones are more dangerous than useful
OpenAI recently announced it would not release its voice clone generator "Voice Engine" for now due to misuse risks. But that matters little because there is open-source software and numerous commercial products on the market that can generate convincing voice clones from just a few seconds of source audio. There's no regulation for this software.
Cybersecurity researchers recently demonstrated that fraudsters can splice voice clones into phone calls to enable financial scams. This is one of the reasons OpenAI urged financial institutions to rethink voice verification when it unveiled its voice clone model. Voice clones also run the risk of enabling voter fraud.
There are some positive uses for voice cloning, such as preserving the original voices of people with vocal disorders, like actor Val Kilmer. However, from an overall societal perspective, the risks and potential harms of AI voice cloning technology seem to outweigh the benefits.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Subscribe nowRead on for the full picture.
Subscribe for hype-free coverage.
- Access to all THE DECODER articles.
- Read without distractions – no Google ads.
- Access to comments and community discussions.
- Weekly AI newsletter.
- 6 times a year: “AI Radar” – deep dives on key AI topics.
- Up to 25 % off on KI Pro online events.
- Access to our full ten-year archive.
- Get the latest AI news from The Decoder.