The project VidBhasa's goal is to develop a video dubbing automation that can translate from English or any other foreign language to any of the Indian regional languages that are being targeted. By utilizing existing package libraries and APIs, we are reducing the process of training a multilingual dataset while retaining high accuracy.
The solution implementation involves:
- Converting spoken language in the video into text using Automatic Speech Recognition.
- Translate the text into the target language using Machine Translation Technology.
- Convert the translated text back into speech in the target language using Text-to-Speech (TTS).
- Replacing the original audio in the video with the dubbed speech, ensuring synchronization with the video.
- Add subtitles using the translated text, ensuring they are time-aligned with the video.