I have summarized how to automatically add English subtitles and English audio to Japanese videos. This uses Azure OpenAI Service’s Whisper and Speech Services.
Overview
The goal this time is to make a Japanese audio video multilingual as follows:
- Japanese version: Original video (Japanese audio, no subtitles)
- English version: English audio + English subtitles
Services Used
| Service | Purpose |
|---|---|
| Azure OpenAI Service (Whisper) | Translation from Japanese audio to English text |
| Azure Speech Services (TTS) | Synthesis from English text to English audio |
| FFmpeg | Audio extraction and video merging |
Procedure
1. Environment Setup
Required Tools
Azure Configuration (.env)
2. Extract Audio from Video
Since the Azure Whisper API has a 25MB file size limit, the audio is compressed and extracted.
3. Generate English Subtitles with Whisper
Using the Azure OpenAI Service’s Whisper API, Japanese audio is transcribed while being translated into English.
Key point: Using the translations endpoint, Japanese audio is directly translated into English text.
4. Generate English Audio with Speech Services
The generated subtitle text is synthesized into audio using Azure Speech Services.
5. Create Timing-Synchronized Audio
To place audio according to subtitle timestamps, the following processing is performed:
- Generate individual audio for each subtitle segment
- Insert silence based on timestamps
- Concatenate all segments
6. Create English Version Video
Finally, combine the original video’s visuals with the English audio.
Generated Files
| File | Description |
|---|---|
original.mp4 | Japanese version (original video) |
output_en.mp4 | English version (with English audio) |
subtitles_en.srt | English subtitle file |
Uploading to YouTube
Japanese Version
- Upload the original video as-is
English Version
- Upload the video with English audio
- Add the SRT file in YouTube Studio > Subtitles tab
Cost
| Service | Price (approximate) |
|---|---|
| Azure OpenAI Whisper | $0.006 / minute |
| Azure Speech Services | $16 / 1 million characters |
For a video of about 8 minutes, the cost is roughly a few dozen yen.
Summary
By combining Azure OpenAI Service’s Whisper and Speech Services, you can automate the process of creating English versions of Japanese videos.
Benefits:
- High-accuracy translation (Whisper’s translation feature)
- Natural English audio (Neural TTS)
- Automatic generation of timestamped subtitles
Notes:
- Adjustment of audio length and subtitle timing is required
- Technical terms and proper nouns may need manual correction