The gospel of audio and video developers, rapid integration of AI dubbing capability

Audio and video content creators have an increasing demand for AI dubbing in editing applications. It has many advantages. It can not only solve the cost of employing human dubbing and the limitations of the creator's own accent and language, but also greatly improve the production efficiency. For example, the playback time of short video is as short as tens of seconds, and that of long vlog video is 4-5 minutes. Using AI dubbing function can realize multiple editing requirements in a short time.

HMS Core audio editing service (Audio Editor Kit) provides AI dubbing service to help developers easily build voice synthesis function in the application. One click output text can convert voice. It supports cute children's voice, kind women's voice, sunshine men's voice, English men's voice, women's voice and other styles, and can also customize the voice you want. With natural and smooth timbre and rich emotion, it can meet the hot scenes concerned by many developers, such as audio reading, audio content production, editing and so on.

Development practice

1. Development preparation

For detailed preparation steps, please refer to the official website of Huawei developer Alliance:

https://developer.huawei.com/consumer/cn/doc/development/Media-Guides/config-agc-0000001154009063?ha_source=hms1

2. Edit project integration

2.1 setting the authentication information of the application

Developers need to use APIs_ Key or Access Token to set application authentication information.

(recommended) set the Access Token through setAccessToken method and initialize the setting when the application starts

HAEApplication.getInstance().setAccessToken("your access token");

Set API through setApiKey method_ Key, which can be initialized and set once when the application starts, without multiple settings.

HAEApplication.getInstance().setApiKey("your ApiKey");

2.2 initialization environment

Initialize the audio editing management class, create the timeline and the required swimlane.

// Create audio editing management class
HuaweiAudioEditor mEditor = HuaweiAudioEditor.create(mContext);
// Initialize the running environment of Editor
mEditor.initEnvironment();
// Create timeline
HAETimeLine mTimeLine = mEditor.getTimeLine();
// Create Lane
HAEAudioLane audioLane = mTimeLine.appendAudioLane();

Import music.

// Add audio resources at the end of the lane
HAEAudioAsset audioAsset = audioLane.appendAudioAsset("/sdcard/download/test.mp3", mTimeLine.getCurrentTime());

3. AI dubbing function integration

Call HAEAiDubbingEngine to realize AI dubbing function.

// Configure AI dubbing engine through this configuration class
HAEAiDubbingConfig haeAiDubbingConfig = new HAEAiDubbingConfig()
// set volume 
.setVolume(volumeVal)
// Set speed of sound
.setSpeed(speedVal)
// Set speaker
.setType(defaultSpeakerType);
// Single AI dubbing task callback
HAEAiDubbingCallback callback = new HAEAiDubbingCallback() {
    @Override
    public void onError(String taskId, HAEAiDubbingError err) {
        // error handling
    }
    @Override
    public void onWarn(String taskId, HAEAiDubbingWarn warn) {}
    @Override
    public void onRangeStart(String taskId, int start, int end) {}
    @Override
    public void onAudioAvailable(String taskId, HAEAiDubbingAudioInfo haeAiDubbingAudioFragment, int i, Pair<Integer, Integer> pair, Bundle bundle) {
        // Start receiving files and saving them as files
    }
    @Override
    public void onEvent(String taskId, int eventID, Bundle bundle) {
        // The synthesis is complete.
        if (eventID == HAEAiDubbingConstants.EVENT_SYNTHESIS_COMPLETE) {
            // AI dubbing task processing is completed, that is, all synthetic audio data are processed
        }
    }
    @Override
    public void onSpeakerUpdate(List<HAEAiDubbingSpeaker> speakerList, List<String> lanList,
         List<String> lanDescList) { }
};
// AI dubbing engine
HAEAiDubbingEngine mHAEAiDubbingEngine = new HAEAiDubbingEngine(haeAiDubbingConfig);
// Set AI dubbing task playback process listening
mHAEAiDubbingEngine.setAiDubbingCallback(callback);
// Real time AI dubbing and playing API, text incoming text of voice to be transferred, modeAI dubbing task playing mode
String taskId = mHAEAiDubbingEngine.speak(text, mode);
// Pause playback
mHAEAiDubbingEngine.pause();
// Resume playback
mHAEAiDubbingEngine.resume();
// Turn off synthesis
mHAEAiDubbingEngine.stop();