Multi-role dubbing includes two methods: Subtitle-Based Multi-Role Dubbing and Video-Based Multi-Role Dubbing.
One: Subtitle-Based Multi-Role Dubbing
Since v3.74, the "Subtitle Multi-Role Dubbing" feature has been added. Click the Subtitle Multi-Role Dubbing button on the left toolbar. In the pop-up window, import the srt subtitles that need dubbing, and then set a role for each subtitle to achieve multi-role pronunciation.
Two: Original Voice Cloning and Multi-Role Dubbing Based on Video
In the Custom Video Translation function, you can select F5-TTS/index-tts/clone-voice/CosyVoice/GPT-SOVITS/Chatterbox etc. in the dubbing channel. Selecting the clone role will use the original sound as a reference audio for dubbing, resulting in a dubbing with the original timbre.
Using the clone-voice Dubbing Channel
You need to install the https://github.com/jianchang512/clone-voice project. After opening the project homepage, read the instructions carefully. You can deploy the clone-voice project using the source code. If it is a Windows system, you can also directly find Releases in the middle of the right side (https://github.com/jianchang512/clone-voice/releases), directly download the integrated package, and double-click app.exe after downloading and decompressing to start it.
When it shows that the startup is successful, fill in the default api address http://127.0.0.1:9988 in the Menu--TTS Settings--Original Timbre Cloning clone-voice HTTP address of the video translation software. After testing and confirming that there are no problems, you can start using it.
Using the CosyVoice Dubbing Channel
You also need to install the CosyVoice project. For the installation tutorial, see https://pyvideotrans.com/cosyvoice.html
Of course, you can also use a third-party integrated package, but the third-party integrated package does not support cloning the timbre and can only specify a fixed audio.
After installing according to the tutorial, go to this address https://github.com/jianchang512/cosyvoice-api/blob/main/api.py to download the api.py file, put it under the CosyVoice project, and in the same directory as the webui.py file.
Then start api.py, and fill in the api address in the video translation software Menu--TTS Settings-CosyVoice api address, the default address http://127.0.0.1:9233
Using the F5-TTS Dubbing Channel
You need to install the F5-TTS project. See https://pyvideotrans.com/f5tts.html for a detailed installation tutorial.
Fill in the F5-TTS api address in the video translation software menu-TTS settings.
Select clone for the role in the main interface, and you can perform clone timbre dubbing.
Note that in addition to clone-voice supporting more than a dozen languages, F5-TTS and CosyVoice only support Chinese and English language cloning.
Setting Multiple Roles
Also in the Custom Video Translation function, select a single video to be processed. After the translation stage is over, it will pause. At this time, you can set a role for each subtitle separately on the right side of the interface to achieve multi-role dubbing.
You need to select a default dubbing role in the dubbing role of the main interface. If you do not set it separately, all of them will use the default role.
