1, Foreword

Audio and video playback is one of the indispensable functions for today's Internet applications. As an App developer, developing an audio and video playback function is neither difficult nor simple. We often face several choices:

1. Use native video components (such as MediaPlayer)
2. Use native hard decoding / FFmpeg soft decoding to customize video playback components
3. Use fully open source third-party components (such as ijkplayer)
4. Use commercial third-party components (such as Tencent cloud player and Alibaba cloud player)

The above schemes need to be selected according to business needs and their own technical level. Generally speaking:

1. The use of native components is primitive and only provides the most basic playback functions. It is generally suitable for relatively simple business needs, and it supports relatively few playback formats. If the business functions are complex, you need to spend more time and energy on development and adaptation.
2. Using native hard decoding / FFMpeg soft solution, or using fully open source third-party components, it is highly customizable, but developers need to have a relatively high level of audio and video development, and be very familiar with video codec, OpenGL, FFMpeg and other knowledge.
3. Using commercial third-party components is a relatively compromise scheme. It generally has rich functions, simple use and good compatibility. At the same time, it has a complete back-end resource management system, which can greatly improve the development efficiency of each end and is suitable for use in the early, middle and late stages of the project.

It can be seen that commercial third-party components have incomparable advantages in the early and middle and late stages of the project, and are often the preferred choice of most developers.

Therefore, let's take a look at how to use commercial third-party components to realize video on demand. This time, we try the on-demand components of Tencent cloud to see how to integrate, access and what pitfalls we encounter in the process of use.

2, SDK function point

Tencent cloud on demand supports many function points, which can basically cover daily development and use. In addition to the basic playback function, it also provides a series of functions

1. Full screen play
2. Sliding adjustment progress
3. Slide to adjust brightness and sound
4. screenshot
5. bullet chat
6. Multiple play
7. hardware speedup
8. Suspended window playback
9. Anti theft chain
10. Manage
11. Wait, wait

3, SDK architecture

Let's first learn about the main interfaces of Tencent cloud on demand SDK, so that we can be more confident in the integration and development process.

Here we have to make complaints about official documents, which is really very crude. In addition to the SDK integration instructions, there is almost no other explanation. We have to download Demo to view the source code, so that we can understand the complete access process more specifically, otherwise we will encounter many pits.

Let's take a look at the official Demo structure

These include:

app: main page of Demo

common: a general component. In fact, it is not used in the Demo

superplayerdemo: Player page

superplayerkit: playback component encapsulation

Among them, the most important is the module superplayerkit, which encapsulates the basic playback controls. All functions are developed based on this module,

The superplayerkit is actually the SDK we need to integrate.

4, Integrated SDK

Integrating the third-party SDK, the official document is our first choice for reference. The official provides three ways for us to choose:

• Automatic loading (AAR)
• Manual download (AAR)
• Integration SDK (jar+so)

Relatively speaking, mode 1 [automatic loading (AAR)] is more convenient. We use this mode.

First, create a new project and import the superplayerkit as follows:

Next, we need to make some modifications to the gradle of the superplaykit, because some project configurations are made in the Demo, which may not be suitable for our own projects. The comparison is as follows:

Before modification:

After modification:

Finally, the superplayerkit is introduced into the gradle of the app

dependencies {
    //....
    
    implementation project(':superplayerkit')
}

In addition, the SDK currently supports three types of cpu architectures, which can be configured in the gradle of the app according to recent requirements:

defaultConfig {
    //......
    
    ndk {
        abiFilters "armeabi", "armeabi-v7a", "arm64-v8a"
    }
}

This completes the integration.

5, Realize video playback

1. In activity_ Add playback component to main.xml

<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <com.tencent.liteav.demo.superplayer.SuperPlayerView
        android:id="@+id/superPlayer"
        android:layout_width="match_parent"
        android:layout_height="200dp"
        app:layout_constraintTop_toTopOf="parent"
        app:layout_constraintLeft_toLeftOf="parent" />

</androidx.constraintlayout.widget.ConstraintLayout>

2. Initialize the player

// MainActivity.kt

private fun initPlayer() {
    player = findViewById(R.id.superPlayer)

    // Player configuration
    val prefs = SuperPlayerGlobalConfig.getInstance()
    // Open the floating window to play
    prefs.enableFloatWindow = true
    //Set the initial position, width and height of the suspended window
    val rect = TXRect()
    rect.x = 0
    rect.y = 0
    rect.width = 810
    rect.height = 540
    prefs.floatViewRect = rect
    // Default number of player caches
    prefs.maxCacheItem = 5
    // Set player rendering mode
    prefs.enableHWAcceleration = true
    prefs.renderMode = TXLiveConstants.RENDER_MODE_ADJUST_RESOLUTION
}

The initialization of the playback component is very simple, and the configuration of the floating window mode is also very simple.

3. Play video

3.1 play third-party URL video link

In the official documents, the code to play is very simple, as follows

//Do not open the anti-theft chain
SuperPlayerModel model = new SuperPlayerModel();
model.appId = 1400329073;// Configure AppId
model.videoId = new SuperPlayerVideoId();
model.videoId.fileId = "5285890799710670616"; // Configure FileId
mSuperPlayerView.playWithModel(model);

If you just write code according to this document, you may be surprised: does Tencent's player only support the playback of videos uploaded to Tencent's background, doesn't it mean that it can support the playback of third-party URLs?

Don't panic. SuperPlayerModel also has another parameter url for playing linked videos, so let's change the code

//Do not open the anti-theft chain
SuperPlayerModel model = new SuperPlayerModel();
model.appId = 1400329073;// Configure AppId
model.videoId = new SuperPlayerVideoId();
// Modify to play url
model.url = "http://xxxxxx.mp4";
mSuperPlayerView.playWithModel(model);

Then you will find that after this change, it can't be broadcast! Why?

Take a look at the playback interface of SuperPlayerView

/**
 * Play video
 *
 * @param model
 */
public void playWithModel(final SuperPlayerModel model) {
    if (model.videoId != null) {
        mSuperPlayer.play(model.appId, model.videoId.fileId, model.videoId.pSign);
    } else if (model.videoIdV2 != null) {
    } else if (model.multiURLs != null && !model.multiURLs.isEmpty()) {
        mSuperPlayer.play(model.appId, model.multiURLs, model.playDefaultIndex);
    } else {
        mSuperPlayer.play(model.url);
    }
}

You can see that if the model.videoId is not empty, the process of playing fileId is still followed. If you want to play the url, remove the videoId.

SuperPlayerModel model = new SuperPlayerModel();
// Play url
model.url = "http://xxxxxx.mp4";
mSuperPlayerView.playWithModel(model);

Sure enough, the video can be played normally.

In fact, the SDK also provides another more concise interface. If you don't look at the source code, you can't find it.

/**
 * Start playing
 *
 * @param url Video address
 */
public void play(String url) {
    mSuperPlayer.play(url);
}

Just play the url directly. Change it again and you'll get it in one sentence:

mSuperPlayerView.play("http://xxxxxx.mp4");

3.2 play the video uploaded by Tencent cloud background

If you only play external video links, many functions provided by the SDK cannot be used, such as sprite map, video playback data statistics, video encryption, etc.

With Tencent cloud on demand background system, more personalized functions can be realized. Let's see how to cooperate with Tencent cloud background to realize playback.

The following is the playback code provided in the document, which is realized in combination with Tencent cloud.

//Do not open the anti-theft chain
SuperPlayerModel model = new SuperPlayerModel();
model.appId = 1400329073;// Configure AppId
model.videoId = new SuperPlayerVideoId();
model.videoId.fileId = "5285890799710670616"; // Configure FileId
mSuperPlayerView.playWithModel(model);

We need to get two parameters to realize the playback function, which are

• Appid: appid of Tencent cloud login user
• fileId: ID of the video to be played

Note that cloud on demand requires real name authentication

First, after logging into Tencent cloud, you can get APPID on the account information page

Then upload the video we want to play. Click [1, 2, 3] in the figure below to upload the video. After uploading successfully, you can get the fileId corresponding to the video in [4] in the list.

Finally, fill in appId and fileId respectively to play.

In this way, after playing, you can see the playback data statistics in the background.

What if you want to generate Sprite?

Sprite chart function: when pulling the progress bar, you can preview the screenshot of the screen

When uploading a video, you need to turn on video processing, transcode the video, and generate sprite map. When uploading a video, select the LongVideoPreset task stream according to the following figure [1, 2, 3].

The effects are as follows:

Other functions will not be described here one by one. You can take a look at what you need.

5, Pit stepping & pit filling

Whether playing third-party links or Tencent cloud links is relatively simple. Although there are some pits, it doesn't need to spend a lot of energy to solve them.

However, if you want to access the complete function, you will encounter more large and small pits. Let's take a look.

1. Full screen play

According to the above connected process, we can realize normal playback, and also see that the playback window displays control buttons such as progress bar, play / pause, full screen, floating window and so on.

However, when you click the full screen play button in the lower right corner, you will find that the screen is horizontal, but the video is not played in full screen.

Why? Make complaints about the document.

Lu Xun said: nothing can't be solved by reading the source code!

How is SuperPlayerView implemented?

First, let's look at its xml layout

<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <com.tencent.rtmp.ui.TXCloudVideoView
        android:id="@+id/superplayer_cloud_video_view"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:layout_centerInParent="true" />

    <com.tencent.liteav.demo.superplayer.ui.player.WindowPlayer
        android:id="@+id/superplayer_controller_small"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

    <com.tencent.liteav.demo.superplayer.ui.player.FullScreenPlayer
        android:id="@+id/superplayer_controller_large"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

    <com.tencent.liteav.demo.superplayer.ui.view.DanmuView
        android:id="@+id/superplayer_danmuku_view"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

    <com.tencent.liteav.demo.superplayer.ui.player.FloatPlayer
        android:id="@+id/superplayer_controller_float"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

</RelativeLayout>

You can see that the playback control is actually composed of a playback component and several UI control components.

• TXCloudVideoView: real playback entity
• WindowPlayer: normal window mode UI control
• Fullscreenlayer: full screen mode UI control
• FloatPlayer: floating window mode UI control
• DanmuView: barrage assembly

We can guess the general playback process of this control: TXCloudVideoView is a real video playback component. Then, according to different playback modes, display the corresponding UI controls and hide other irrelevant controls.

In fact, full screen playback is very simple. Rotate the screen, set TXCloudVideoView FullScreenPlayer to full screen, and hide WindowPlayer and FloatPlayer to achieve full screen playback. Let's see how it is done in the SDK.

// SuperPlayerView.java

/**
 * Initialize controller callback
 */
private Player.Callback mControllerCallback = new Player.Callback() {
    @Override
    public void onSwitchPlayMode(SuperPlayerDef.PlayerMode playerMode) {
        if (playerMode == SuperPlayerDef.PlayerMode.FULLSCREEN) {
            fullScreen(true);
        } else {
            fullScreen(false);
        }
        // [1] Hide all windows
        mFullScreenPlayer.hide();
        mWindowPlayer.hide();
        mFloatPlayer.hide();
        //Request full screen mode
        if (playerMode == SuperPlayerDef.PlayerMode.FULLSCREEN) {
            if (mLayoutParamFullScreenMode == null) {
                return;
            }
            // [2] Remove window mode UI control
            removeView(mWindowPlayer);
            // [3] Add full screen UI control
            addView(mFullScreenPlayer, mVodControllerFullScreenParams);
            // [4] Set the playback window parameter to full screen
            setLayoutParams(mLayoutParamFullScreenMode);
            // [5] Rotate the screen to horizontal
            rotateScreenOrientation(SuperPlayerDef.Orientation.LANDSCAPE);
            if (mPlayerViewCallback != null) {
                mPlayerViewCallback.onStartFullScreenPlay();
            }
        } else if (playerMode == SuperPlayerDef.PlayerMode.WINDOW) {// Request window mode
            // ......
        } else if (playerMode == SuperPlayerDef.PlayerMode.FLOAT) {//Request suspended window mode
            // ......
        }
        mSuperPlayer.switchPlayMode(playerMode);
    }

The above code realizes the switching of full screen mode through the event callback interface in SuperPlayerView. The implementation process mainly includes five steps:

1. Hide all UI controls first
2. When requesting to enter full screen mode, remove the UI control WindowPlayer in window mode
3. Add the full screen UI control fullscreenlayer back through the addView method
4. Set the entire SuperPlayerView to full screen
5. Rotate screen to landscape

Through these five steps, we can really realize horizontal screen and full screen playback. Why is it that when we click full screen, it is actually only horizontal screen, not full screen? The official Demo is normal!

This time will test our basic skills, which is actually a basic common sense problem of Android system.

If the Android system rotates the Activity page, the page will be destroyed and recreated by default.

Based on this feature, it is not difficult to understand why! Because the page is destroyed and rebuilt, clicking the full screen button is actually equivalent to re entering a new horizontal screen page. The parameters set in the previous steps [1 ~ 4] are actually invalid, and TXCloudVideoView is also re created, so the video will be played from the beginning.

Then the solution is also very simple, that is, don't let the page destroy and rebuild!

Android provides a way to achieve this goal. As long as you add the following configuration to the Activity corresponding to AndroidManifest.xml, you can set that when the page rotates, it will not be rebuilt.

The official Demo actually does the same.

android:configChanges="orientation|keyboardHidden|screenSize"

<activity android:name=".MainActivity"
    android:configChanges="orientation|keyboardHidden|screenSize"
    android:launchMode="singleTask">
    <intent-filter>
        <action android:name="android.intent.action.MAIN" />

        <category android:name="android.intent.category.LAUNCHER" />
    </intent-filter>
</activity>

2. Suspended window playback

Similarly, what I thought was a very simple floating window play also stepped on a small pit: clicking the floating window button has no effect, but the official Demo has no problem. After looking at my own code, I also applied for the permission of the floating window. I'm really a monk Zhang Er. I can't touch my head!

Note: the suspended window needs to apply for permission: <uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" />

Anyway, that's the same sentence. There are no problems that can't be solved by reading the source code.

Let's look at the event callback interface above. This time, let's look at the code related to the application of suspended window mode.

// SuperPlayerView.java

/**
 * Initialize controller callback
 */
private Player.Callback mControllerCallback = new Player.Callback() {
    @Override
    public void onSwitchPlayMode(SuperPlayerDef.PlayerMode playerMode) {
        if (playerMode == SuperPlayerDef.PlayerMode.FULLSCREEN) {
            fullScreen(true);
        } else {
            fullScreen(false);
        }
        mFullScreenPlayer.hide();
        mWindowPlayer.hide();
        mFloatPlayer.hide();
        //Request full screen mode
        if (playerMode == SuperPlayerDef.PlayerMode.FULLSCREEN) {
           // ......
        } else if (playerMode == SuperPlayerDef.PlayerMode.WINDOW) {// Request window mode
            // ......
        } else if (playerMode == SuperPlayerDef.PlayerMode.FLOAT) {//Request suspended window mode
            TXCLog.i(TAG, "requestPlayMode Float :" + Build.MANUFACTURER);
            SuperPlayerGlobalConfig prefs = SuperPlayerGlobalConfig.getInstance();
            if (!prefs.enableFloatWindow) {
                return;
            }
            // [1] Apply for permission
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) { // 6.0 dynamic application for suspended window permission
                if (!Settings.canDrawOverlays(mContext)) {
                    Intent intent = new Intent(Settings.ACTION_MANAGE_OVERLAY_PERMISSION);
                    intent.setData(Uri.parse("package:" + mContext.getPackageName()));
                    mContext.startActivity(intent);
                    return;
                }
            } else {
                if (!checkOp(mContext, OP_SYSTEM_ALERT_WINDOW)) {
                    showToast(R.string.superplayer_enter_setting_fail);
                    return;
                }
            }
            mSuperPlayer.pause();

            mWindowManager = (WindowManager) mContext.getApplicationContext().getSystemService(Context.WINDOW_SERVICE);
            mWindowParams = new WindowManager.LayoutParams();
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
                mWindowParams.type = WindowManager.LayoutParams.TYPE_APPLICATION_OVERLAY;
            } else {
                mWindowParams.type = WindowManager.LayoutParams.TYPE_PHONE;
            }
            mWindowParams.flags = WindowManager.LayoutParams.FLAG_NOT_TOUCH_MODAL
                    | WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE;
            mWindowParams.format = PixelFormat.TRANSLUCENT;
            mWindowParams.gravity = Gravity.LEFT | Gravity.TOP;

            // [2] Set the window width and height size, and add FloatPlayer to WindowManager
            SuperPlayerGlobalConfig.TXRect rect = prefs.floatViewRect;
            mWindowParams.x = rect.x;
            mWindowParams.y = rect.y;
            mWindowParams.width = rect.width;
            mWindowParams.height = rect.height;
            try {
                mWindowManager.addView(mFloatPlayer, mWindowParams);
            } catch (Exception e) {
                showToast(R.string.superplayer_float_play_fail);
                return;
            }

            TXCloudVideoView videoView = mFloatPlayer.getFloatVideoView();
            if (videoView != null) {
                mSuperPlayer.setPlayerView(videoView);
                mSuperPlayer.resume();
            }
            // Suspension window Report
            LogReport.getInstance().uploadLogs(LogReport.ELK_ACTION_FLOATMOE, 0, 0);
        }
        mSuperPlayer.switchPlayMode(playerMode);
    }

There are only two steps. First, apply for permission, and then add the suspended window to WindowManager. It is reasonable to say that there is no problem. If there's no problem here, it's that the code doesn't come here at all.

Then start at the place where the click event is received. Let's take a look at the place in SuperPlayerView that listens to the click event of the floating window.

// SuperPlayerView.java

@Override
public void onBackPressed(SuperPlayerDef.PlayerMode playMode) {
    switch (playMode) {
        case FULLSCREEN:// The current mode is full screen mode. Return to switch to window mode
            onSwitchPlayMode(SuperPlayerDef.PlayerMode.WINDOW);
            break;
        case WINDOW:// Currently in window mode, return to exit the player
            if (mPlayerViewCallback != null) {
                mPlayerViewCallback.onClickSmallReturnBtn();
            }
            break;
        case FLOAT:// It is currently a suspended window, exit
            mWindowManager.removeView(mFloatPlayer);
            if (mPlayerViewCallback != null) {
                mPlayerViewCallback.onClickFloatCloseBtn();
            }
            break;
        default:
            break;
    }
}

You can see that when you click the return button in the normal window mode, you will call back mPlayerViewCallback.onClickSmallReturnBtn(); Method, and the callback interface is actually implemented by an external page, and our page does not monitor and implement this interface, so even if you click the button, you will not enter the flow of the floating window, so it has no effect.

At this time, we just need to take a look at the official Demo to know what's going on. In the SuperPlayerActivity of Demo, the following interfaces are implemented:

// SuperPlayerActivity.java

@Override
public void onClickSmallReturnBtn() {
    // Click the return button in small window mode to start floating playback
    showFloatWindow();
}

@Override
public void onStartFloatWindowPlay() {
    // After starting suspension playback, return to the desktop directly for suspension playback
    Intent intent = new Intent(Intent.ACTION_MAIN);
    intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
    intent.addCategory(Intent.CATEGORY_HOME);
    startActivity(intent);
}


/**
 * Suspended window playback
 */
private void showFloatWindow() {
    if (mSuperPlayerView.getPlayerState() == SuperPlayerDef.PlayerState.PLAYING) {
        mSuperPlayerView.switchPlayMode(SuperPlayerDef.PlayerMode.FLOAT);
    } else {
        mSuperPlayerView.resetPlayer();
        finish();
    }
}

See, he put the activation of the floating window mode on the Activity page to control it.

Why did you do that? I really don't understand. This can be implemented in the SDK!

After knowing the reason, the solution is also very simple. Just listen to the callback and set the floating window mode.

// MainActivity.kt

private fun initPlayer() {
    player = findViewById(R.id.superPlayer)

    // Player configuration
    val prefs = SuperPlayerGlobalConfig.getInstance()
    // Open the floating window to play
    prefs.enableFloatWindow = true
    //Set the initial position, width and height of the suspended window
    val rect = TXRect()
    rect.x = 0
    rect.y = 0
    rect.width = 810
    rect.height = 540
    prefs.floatViewRect = rect
    // Default number of player caches
    prefs.maxCacheItem = 5
    // Set player rendering mode
    prefs.enableHWAcceleration = true
    prefs.renderMode = TXLiveConstants.RENDER_MODE_ADJUST_RESOLUTION

    // Listening callback interface
    player.setPlayerViewCallback(this)
}

override fun onClickSmallReturnBtn() {
    player.switchPlayMode(SuperPlayerDef.PlayerMode.FLOAT)
}

override fun onStartFloatWindowPlay() {
    // After starting suspension playback, return to the desktop directly for suspension playback
    val intent = Intent(Intent.ACTION_MAIN)
    intent.flags = Intent.FLAG_ACTIVITY_NEW_TASK
    intent.addCategory(Intent.CATEGORY_HOME)
    startActivity(intent)
}

6, Summary

Above, I basically went through the access process of Tencent cloud on demand SDK, and the functions of the whole on demand should be said to be relatively complete.

If you use Tencent cloud background to manage resources, there is another great function: intelligent cooling. Some videos with low historical on-demand rate can be archived and stored according to different strategies, which greatly reduces our storage cost.

However, the official documents are really too simple. It is basically impossible to access them smoothly only through the documents. You must read the official Demo to access them smoothly, and even need an in-depth understanding of the source code to fully access the code. This is a blow to novices and greatly reduces the access efficiency.

For the official Demo implementation, there are many areas to be discussed, such as the startup of floating window mode and the introduction of bullet screen control, which are too simple and rough. It is only displayed by generating some test content. Is there a good external method for developers to call.