ElevenLabs and Mistral launch tools for multi-language voice cloning

Real-time global translation

ElevenLabs recently announced a new live translation model designed for enterprise customers. The tool allows people on a call to speak their own native languages while hearing the other person in their own language as the conversation happens. It is intended for use in phone networks, contact centers, and live events.

The company's goal is to let a single agent assist callers in many different languages without needing a human interpreter on the line. This technology is already being integrated into systems like the Magenta AI Call Assistant by Deutsche Telekom. The model uses Eleven v4 Turbo, which is a low-latency speech model designed to keep a voice consistent across different markets.

Customers can choose from a library of over 17,500 voices or build their own custom clones. ElevenLabs states that the chosen voice will remain consistent throughout the entire session. This allows a brand to maintain a single, recognizable persona regardless of the language being spoken.

Mistral's high-fidelity cloning

At the same time, the Paris-based company Mistral released Voxtral 3. This family of speech models includes tools for transcription, voice agents, and text-to-speech. A specific feature called Pro Voice is designed for professional use cases where standard voice cloning is not sufficient.

Mistral claims that its Pro Voice adaptation can reproduce a speaker's distinctive character, emotional range, and speaking style. It can do this across 35 or more languages using only 30 minutes of audio. This is significant because many public figures already have more than 30 minutes of clear audio available from interviews or earnings calls.

The models can be accessed through an API, including endpoints hosted in the EU, or deployed on a company's own infrastructure. Voice agents using these models currently cost between $0.04 and $0.08 per minute. However, Mistral has not yet published a specific consent or acceptable-use framework for the Pro Voice feature.

The shift in voice rights

These tools raise significant questions for professional voice actors regarding who has the right to point these models at a specific voice. ElevenLabs has established some rules for its Professional Voice Clones, requiring users to pass a spoken verification step. This involves reading lines into a microphone in a tone similar to the original samples.

In contrast, the industry is seeing a shift in how contracts are written. For example, SAG-AFTRA is currently voting on a deal that would require studios to get specific consent before using a digital replica to dub a performance into another language. This is a change from previous contracts that did not include such protections.

In Japan, the Actors Union released voluntary guidelines this month regarding AI translation and dubbing. These guidelines suggest that using AI to translate a performance into another language should require permission and a specific fee. The union also states that AI should not replace human actors in dubbing for films or dramas.

What creators should check

For voice actors and creators, the launch of these tools means that existing contracts may need to be re-evaluated. A license granted for an English-speaking assistant does not automatically grant the right to have that same voice answer in a different language. It is important to look for specific words like 'language,' 'translation,' or 'localization' in current agreements.

Creators who share clones on marketplaces should also review the specific use cases they have permitted. Live translation for call centers is now a viable use case that may not have been considered when the voice was first uploaded. Ensuring that the terms of use are clear can help prevent unauthorized use of a voice in new markets.

If a creator is asked to provide 30 minutes of natural speech for a model, they should ask for specific details. They should know which languages the model will speak, how long it will be used, and what the payment structure looks like for each instance of use. These tools have moved the conversation from a hypothetical concern to a practical reality for the industry.