A British artificial intelligence firm has said it is rethinking its ‘safeguards’ after its audio tool was used to clone the voices of celebrities and make them say racist and homophobic slurs.
Eleven Labs tweeted on Monday that it had been a “crazy weekend” and admitted to finding “an increasing number of cases of voice cloning abuse” just days after releasing a demo version of the tool.
The company did not give details of the cases, but the 4chan internet forum was inundated with deepfakes of celebrity voices using racist, sexist and homophobic slurs.
Many posts mentioned Eleven Labs, with some users claiming to have been banned by the company and others providing links to the demo version of the tool.
Among the celebrities who came to seek treatment on the forum, which is known for its offensive humor, were stars of the Harry Potter films and directors Quentin Tarantino and George Lucas.
Eleven Labs has touted its voice cloning technology as having “the potential to revolutionize the way content is produced, delivered and interacted with across a range of industries”.
In its publicity material, the firm produced a video of actor Leonardo DiCaprio being voiced by several other celebrities.
The company tweeted that its technology had been “massively applied to positive use”, but said “further safeguards” were needed.
Among the options considered were manually verifying each clone request or tightening account verifications.
AI tools are trained on large expanses of data – images, audio, text, or video – that allow them to essentially predict the output the user wants.
The ChatGPT text tool has taken the world by storm with its ability to generate essays at a relatively high level with just a few prompts.
Imaging tools like Dall-E 2 and Midjourney threaten to upend the world of art and design with their ability to generate complex images in the style of famous artists.
But critics have long feared that cloning technology, especially in audio and video, is leading to a flood of deepfakes and misinformation.
And several AI companies are facing lawsuits from people who claim the companies infringed copyright by using their material as the basis for AI models.