ElevenLabs Announces 'Dubbing v2', an AI Dubbing Model that Replicates Emotion and Tone in Multiple Languages

ElevenLabs, a leader in AI audio research, has announced 'Dubbing v2,' a new AI model designed to naturally translate and dub video and audio content.

'Dubbing v2' represents a significant leap forward, delivering quality that goes beyond traditional translation by 'conveying the emotion and tone of the performance.' It transforms content into other languages while retaining the unique tone, precise timing, delivery, breathing, and subtle emotional expressions of the original speaker, resulting in a more natural dubbing experience.

The model automatically and comprehensively applies the following features: - Automatic Voice Cloning: Maintains the speaker's voice quality in other languages. - Speaker Separation: Supports multilingual dubbing that reflects the characteristics of each speaker in content with multiple people. - Preservation of Background Music and Ambient Sound: Translates and integrates only the voice, without compromising the background music or environment.

Supporting over 90 languages and accents, 'Dubbing v2' assists in expanding Japanese content—including anime, games, podcasts, lectures, and corporate videos—to global audiences. It is currently available via the ElevenLabs UI, with an API planned for enterprise customers to integrate into large-scale production workflows.