Video player kernel switching package
Directory introduction
- 01. Multi video kernel adaptation background
- 1.1 the player kernel is difficult to switch
- 1.2 player kernel and business linkage
- 02. Multi video kernel design objectives
- 2.1 establish a set of video API
- 2.2 key points of defining unified API
- 2.3 seamless kernel modification
- 03. Overall architecture of player kernel
- 3.1 overall kernel architecture
- 3.2 kernel UML class diagram architecture
- 3.3 about dependencies
- 04. Unified video interface design
- 4.1 define video status interface
- 4.2 definition of abstract video player
- 4.3 specific ijk implementation cases
- 05. Easily create different kernel players
- 5.1 how to create different kernels
- 5.2 use one point design mode
- 5.3 design pattern code display
- 06. Other design specifications
- 6.1 stability design description
- 6.2 test case design
- 6.3 expansibility related design
00. General framework of video player
- The basic package of video player player can be in ExoPlayer, MediaPlayer, sound network RTC video player kernel, and the native MediaPlayer can be switched freely
- For view state switching and later maintenance expansion, avoid coupling between functions and services. For example, you need to support high customization of the Player UI, rather than the UI code in the lib library
- For video playback, audio playback, playback, and live video broadcast. It is easy to use, has strong code expansibility and good encapsulation. It is mainly completely decoupled from the business, and exposes interface monitoring to developers to deal with the specific logic of the business
- The overall architecture of the player: player kernel (free switching) + video player + caching while playing + highly customized Player UI view layer
- Project address: https://github.com/yangchong211/YCVideoPlayer
- Introduction document on the overall function of video player: https://juejin.im/post/6883457444752654343
01. Multi adaptation kernel adaptation background
1.1 the player kernel is difficult to switch
- Due to historical reasons, the company uses different video players for many app s. Some use dumpling players, some use native players, and some use Google players. In order to build a player suitable for multiple business lines, you need to package your own video player interface.
1.2 player kernel and business linkage
- And business linkage is difficult to separate
- Player is a core function. In previous projects, you can directly create a player on the Activity page and then play the video. There are also many video businesses... Then the product says, add a list video, make a painful change, and return to the playback function after the change.
- Add new business change
- The player kernel is decoupled from the player, but also from specific services: it supports more playback scenarios and fast access to new playback services, and does not affect other playback services, such as adding Alibaba cloud player kernel or Tencent player kernel later.
02. Multi video kernel design objectives
2.1 establish a set of video API
- Different video player cores have different APIs, so it is difficult to switch operations. If you want to be compatible with kernel switching, you must develop a video interface + implementation class player
2.2 key points of defining unified API
- Question: for different kernel players, such as Google's ExoPlayer, IjkPlayer in station B, and native MediaPlayer, some APIs are different. How to unify the APIs when using them?
- For example, ijk's and exo's video player listener listening APIs are completely different. Compatibility processing is required at this time
- Define the interface, and then implement the interface for different kernel players to rewrite the abstract method. When calling, obtain the interface object and call the API, so that the API can be unified
- Define an interface. What is the purpose of this interface? This interface defines general video player methods, such as video initialization, setting url, loading, and playback status. It can be divided into three parts.
- Part I: Video initialization instance object method, mainly including: initPlayer initializes the video, setDataSource sets the video player address, setSurface sets the video player rendering view, and prepareAsync starts preparing for playback
- Part II: video player status method, mainly including: play, pause, resume, replay, set progress, release resources, obtain progress, set speed and set volume
- Part 3: after the player binds the view, it needs to monitor the playback status, such as playback exceptions, playback completion, playback preparation, playback size changes, and playback preparation
2.2 seamless kernel modification
- It is convenient to create different kernels by passing in different types
- Hide the specific details of kernel player creation. Developers only need to care about the factory corresponding to the required product, do not need to care about the creation details, and even do not need to know the class name of the specific player class. It needs to comply with the opening and closing principle
03. Overall architecture of player kernel
3.1 overall kernel architecture
- Player kernel architecture
3.2 kernel UML class diagram architecture
- The kernel UML class diagram architecture is shown below
3.3 about dependencies
- Google player needs to rely on some Google services. Ijk video player needs to rely on ijk related libraries. If you want to expand other video players, you need to add dependencies. Note that the added library uses compileOnly.
04. Unified video interface design
4.1 define video status interface
- It mainly monitors the playback status of the video during video playback, such as exceptions, playback completion, preparation stage, video size changes, etc.
public interface VideoPlayerListener { /** * abnormal * 1 Link representing error * 2 Represents a parsing exception * 3 Indicates other exceptions * @param type Error type */ void onError(@PlayerConstant.ErrorType int type , String error); /** * complete */ void onCompletion(); /** * video information * @param what what * @param extra extra */ void onInfo(int what, int extra); /** * prepare */ void onPrepared(); /** * Video change monitoring * @param width wide * @param height high */ void onVideoSizeChanged(int width, int height); }
4.2 definition of abstract video player
- This is a unified video player interface, which is mainly divided into three parts. The first part: Video initialization instance object method; Part II: video player state method; Part 3: after the player binds the view, it needs to monitor the playback status
- Part I: Video initialization instance object method
//Video player step 1: create a video player public abstract void initPlayer(); //Set playback address public abstract void setDataSource(String path, Map<String, String> headers); //Used to play video files in raw and asset public abstract void setDataSource(AssetFileDescriptor fd); //Set the View of rendered video, mainly used for TextureView public abstract void setSurface(Surface surface); //Ready to play (asynchronous) public abstract void prepareAsync();
- Part II: video player state method
//play public abstract void start(); //suspend public abstract void pause(); //stop it public abstract void stop(); //Reset player public abstract void reset(); //Is it playing public abstract boolean isPlaying(); //Adjust progress public abstract void seekTo(long time); //Release player public abstract void release(); //Other omissions
- Part 3: after the player binds the view, it needs to monitor the playback status
/** * Bind VideoView, listen for playback exceptions, complete, start preparation, video size change, video information, etc */ public void setPlayerEventListener(VideoPlayerListener playerEventListener) { this.mPlayerEventListener = playerEventListener; }
4.3 specific ijk implementation cases
- The kernel implementation class code of ijk is as follows
public class IjkVideoPlayer extends AbstractVideoPlayer { protected IjkMediaPlayer mMediaPlayer; private int mBufferedPercent; private Context mAppContext; public IjkVideoPlayer(Context context) { if (context instanceof Application){ mAppContext = context; } else { mAppContext = context.getApplicationContext(); } } @Override public void initPlayer() { mMediaPlayer = new IjkMediaPlayer(); //native log IjkMediaPlayer.native_setLogLevel(VideoLogUtils.isIsLog() ? IjkMediaPlayer.IJK_LOG_INFO : IjkMediaPlayer.IJK_LOG_SILENT); setOptions(); mMediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC); initListener(); } @Override public void setOptions() { } /** * ijk Video player monitor listener */ private void initListener() { // Set listening. You can view imedia player source code listening events in ijk // Set video error listener mMediaPlayer.setOnErrorListener(onErrorListener); // Set video playback completion listening event mMediaPlayer.setOnCompletionListener(onCompletionListener); // Set up video message listener mMediaPlayer.setOnInfoListener(onInfoListener); // Set video buffer update listening event mMediaPlayer.setOnBufferingUpdateListener(onBufferingUpdateListener); // Set listening events for preparing video playback mMediaPlayer.setOnPreparedListener(onPreparedListener); // Set video size change listener mMediaPlayer.setOnVideoSizeChangedListener(onVideoSizeChangedListener); // Set video seek to complete listening event mMediaPlayer.setOnSeekCompleteListener(onSeekCompleteListener); // Set time text listener mMediaPlayer.setOnTimedTextListener(onTimedTextListener); mMediaPlayer.setOnNativeInvokeListener(new IjkMediaPlayer.OnNativeInvokeListener() { @Override public boolean onNativeInvoke(int i, Bundle bundle) { return true; } }); } /** * Set playback address * * @param path Playback address * @param headers Playback address request header */ @Override public void setDataSource(String path, Map<String, String> headers) { // Set dataSource if(path==null || path.length()==0){ if (mPlayerEventListener!=null){ mPlayerEventListener.onInfo(PlayerConstant.MEDIA_INFO_URL_NULL, 0); } return; } try { //Parse path Uri uri = Uri.parse(path); if (ContentResolver.SCHEME_ANDROID_RESOURCE.equals(uri.getScheme())) { RawDataSourceProvider rawDataSourceProvider = RawDataSourceProvider.create(mAppContext, uri); mMediaPlayer.setDataSource(rawDataSourceProvider); } else { //Handling UA issues if (headers != null) { String userAgent = headers.get("User-Agent"); if (!TextUtils.isEmpty(userAgent)) { mMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT, "user_agent", userAgent); } } mMediaPlayer.setDataSource(mAppContext, uri, headers); } } catch (Exception e) { mPlayerEventListener.onError(); } } /** * Used to play video files in raw and asset */ @Override public void setDataSource(AssetFileDescriptor fd) { try { mMediaPlayer.setDataSource(new RawDataSourceProvider(fd)); } catch (Exception e) { mPlayerEventListener.onError(); } } /** * Set the View of rendered video, mainly used for TextureView * @param surface surface */ @Override public void setSurface(Surface surface) { mMediaPlayer.setSurface(surface); } /** * Ready to play (asynchronous) */ @Override public void prepareAsync() { try { mMediaPlayer.prepareAsync(); } catch (IllegalStateException e) { mPlayerEventListener.onError(); } } /** * suspend */ @Override public void pause() { try { mMediaPlayer.pause(); } catch (IllegalStateException e) { mPlayerEventListener.onError(); } } /** * play */ @Override public void start() { try { mMediaPlayer.start(); } catch (IllegalStateException e) { mPlayerEventListener.onError(); } } /** * stop it */ @Override public void stop() { try { mMediaPlayer.stop(); } catch (IllegalStateException e) { mPlayerEventListener.onError(); } } /** * Reset player */ @Override public void reset() { mMediaPlayer.reset(); mMediaPlayer.setOnVideoSizeChangedListener(onVideoSizeChangedListener); setOptions(); } /** * Is it playing */ @Override public boolean isPlaying() { return mMediaPlayer.isPlaying(); } /** * Adjust progress */ @Override public void seekTo(long time) { try { mMediaPlayer.seekTo((int) time); } catch (IllegalStateException e) { mPlayerEventListener.onError(); } } /** * Release player */ @Override public void release() { mMediaPlayer.setOnErrorListener(null); mMediaPlayer.setOnCompletionListener(null); mMediaPlayer.setOnInfoListener(null); mMediaPlayer.setOnBufferingUpdateListener(null); mMediaPlayer.setOnPreparedListener(null); mMediaPlayer.setOnVideoSizeChangedListener(null); new Thread() { @Override public void run() { try { mMediaPlayer.release(); } catch (Exception e) { e.printStackTrace(); } } }.start(); } /** * Get the current playback position */ @Override public long getCurrentPosition() { return mMediaPlayer.getCurrentPosition(); } /** * Total duration of video acquisition */ @Override public long getDuration() { return mMediaPlayer.getDuration(); } /** * Get buffer percentage */ @Override public int getBufferedPercentage() { return mBufferedPercent; } /** * Set the View of rendered video, mainly used for SurfaceView */ @Override public void setDisplay(SurfaceHolder holder) { mMediaPlayer.setDisplay(holder); } /** * set volume */ @Override public void setVolume(float v1, float v2) { mMediaPlayer.setVolume(v1, v2); } /** * Set whether to cycle */ @Override public void setLooping(boolean isLooping) { mMediaPlayer.setLooping(isLooping); } /** * Set playback speed */ @Override public void setSpeed(float speed) { mMediaPlayer.setSpeed(speed); } /** * Get playback speed */ @Override public float getSpeed() { return mMediaPlayer.getSpeed(0); } /** * Gets the current buffered network speed */ @Override public long getTcpSpeed() { return mMediaPlayer.getTcpSpeed(); } /** * Set video error listener * int MEDIA_INFO_VIDEO_RENDERING_START = 3;//Video ready to render * int MEDIA_INFO_BUFFERING_START = 701;//Start buffering * int MEDIA_INFO_BUFFERING_END = 702;//End of buffer * int MEDIA_INFO_VIDEO_ROTATION_CHANGED = 10001;//Video selection information * int MEDIA_ERROR_SERVER_DIED = 100;//Video interruption is generally an exception of the video source or an unsupported video type. * int MEDIA_ERROR_IJK_PLAYER = -10000,//Generally, there is a problem with the video source or the data format is not supported. For example, the audio is not AAC * int MEDIA_ERROR_NOT_VALID_FOR_PROGRESSIVE_PLAYBACK = 200;//Data error no valid recycle */ private IMediaPlayer.OnErrorListener onErrorListener = new IMediaPlayer.OnErrorListener() { @Override public boolean onError(IMediaPlayer iMediaPlayer, int framework_err, int impl_err) { mPlayerEventListener.onError(); VideoLogUtils.d("IjkVideoPlayer----listener---------onError -> STATE_ERROR -- what: " + framework_err + ", extra: " + impl_err); return true; } }; /** * Set video playback completion listening event */ private IMediaPlayer.OnCompletionListener onCompletionListener = new IMediaPlayer.OnCompletionListener() { @Override public void onCompletion(IMediaPlayer iMediaPlayer) { mPlayerEventListener.onCompletion(); VideoLogUtils.d("IjkVideoPlayer----listener---------onCompletion -> STATE_COMPLETED"); } }; /** * Set up video message listener */ private IMediaPlayer.OnInfoListener onInfoListener = new IMediaPlayer.OnInfoListener() { @Override public boolean onInfo(IMediaPlayer iMediaPlayer, int what, int extra) { mPlayerEventListener.onInfo(what, extra); VideoLogUtils.d("IjkVideoPlayer----listener---------onInfo -> -- what: " + what + ", extra: " + extra); return true; } }; /** * Set video buffer update listening event */ private IMediaPlayer.OnBufferingUpdateListener onBufferingUpdateListener = new IMediaPlayer.OnBufferingUpdateListener() { @Override public void onBufferingUpdate(IMediaPlayer iMediaPlayer, int percent) { mBufferedPercent = percent; } }; /** * Set listening events for preparing video playback */ private IMediaPlayer.OnPreparedListener onPreparedListener = new IMediaPlayer.OnPreparedListener() { @Override public void onPrepared(IMediaPlayer iMediaPlayer) { mPlayerEventListener.onPrepared(); VideoLogUtils.d("IjkVideoPlayer----listener---------onPrepared -> STATE_PREPARED"); } }; /** * Set video size change listener */ private IMediaPlayer.OnVideoSizeChangedListener onVideoSizeChangedListener = new IMediaPlayer.OnVideoSizeChangedListener() { @Override public void onVideoSizeChanged(IMediaPlayer iMediaPlayer, int width, int height, int sar_num, int sar_den) { int videoWidth = iMediaPlayer.getVideoWidth(); int videoHeight = iMediaPlayer.getVideoHeight(); if (videoWidth != 0 && videoHeight != 0) { mPlayerEventListener.onVideoSizeChanged(videoWidth, videoHeight); } VideoLogUtils.d("IjkVideoPlayer----listener---------onVideoSizeChanged -> WIDTH: " + width + "， HEIGHT: " + height); } }; /** * Set time text listener */ private IMediaPlayer.OnTimedTextListener onTimedTextListener = new IMediaPlayer.OnTimedTextListener() { @Override public void onTimedText(IMediaPlayer iMediaPlayer, IjkTimedText ijkTimedText) { } }; /** * Set video seek to complete listening event */ private IMediaPlayer.OnSeekCompleteListener onSeekCompleteListener = new IMediaPlayer.OnSeekCompleteListener() { @Override public void onSeekComplete(IMediaPlayer iMediaPlayer) { } }; }
05. Easily create different kernel players
5.1 how to create different kernels
- Let's take a look at the code for creating different kernel players. The developer only needs to pass in a type parameter to create instance objects of different classes. The code is shown below
/** * Obtain the specific implementation class of PlayerFactory and obtain the kernel * When creating an object, you only need to pass the type instead of the corresponding factory to create a specific product object * TYPE_IJK IjkPlayer，Packaging player based on IjkPlayer * TYPE_NATIVE MediaPlayer，Based on native player controls * TYPE_EXO Based on Google Video Player * TYPE_RTC RTC based video player * @param type type * @return */ public static AbstractVideoPlayer getVideoPlayer(Context context,@PlayerConstant.PlayerType int type){ if (type == PlayerConstant.PlayerType.TYPE_EXO){ return ExoPlayerFactory.create().createPlayer(context); } else if (type == PlayerConstant.PlayerType.TYPE_IJK){ return IjkPlayerFactory.create().createPlayer(context); } else if (type == PlayerConstant.PlayerType.TYPE_NATIVE){ return MediaPlayerFactory.create().createPlayer(context); } else if (type == PlayerConstant.PlayerType.TYPE_RTC){ return IjkPlayerFactory.create().createPlayer(context); } else { return IjkPlayerFactory.create().createPlayer(context); } }
5.2 use one point design mode
- What is the motivation to create different objects using the factory pattern and why?
- A video player can provide multiple kernel players (such as ijk, exo, media, rtc, etc.). These players all come from the same base class, but after inheriting the base class, different subclasses modify some properties so that they can present different appearances.
- If you want to use these kernel player s, you don't need to know the names of these specific kernels. You only need to know a parameter representing the kernel class, and provide a convenient method to call. You can return a corresponding kernel object by passing the parameter into the method. At this time, you can use the factory mode.
- Firstly, a factory abstract class is defined, and then different kernel players create their specific factory implementation classes respectively
- PlayerFactory: abstract factory. It is the core of the factory method pattern. Any factory class that creates objects in the pattern must implement this interface
- ExoPlayerFactory: specific factory. Specific factory roles contain logic closely related to business and are called by users to create specific product objects.
- How to use is divided into three steps. The specific operations are as follows
- 1. First call the createPlayer method in the specific factory object; 2. Obtain specific product objects according to the passed in product type parameters; 3. Return the product object and use it.
- In short, when creating an object, you only need to pass the type instead of the corresponding factory to create a specific product object
5.3 design pattern code display
- Abstract factory class. The code is as follows
public abstract class PlayerFactory<T extends AbstractVideoPlayer> { public abstract T createPlayer(Context context); } - The specific implementation of the factory class is shown below
public class ExoPlayerFactory extends PlayerFactory<ExoMediaPlayer> { public static ExoPlayerFactory create() { return new ExoPlayerFactory(); } @Override public ExoMediaPlayer createPlayer(Context context) { return new ExoMediaPlayer(context); } } - The biggest advantage of this method of creating objects
- The factory method is used to create the required product and hide the details of which specific product class will be instantiated. Users only need to care about the factory corresponding to the required product, do not need to care about the creation details, or even need not know the class name of the specific product class.
- When adding new products, such as adding an Ali player kernel later, you only need to add a specific factory and specific products. The scalability of the system becomes very good, which fully conforms to the "opening and closing principle"
06. Other design specifications
6.1 stability design description
- No, there's nothing to say here.
6.2 test case design
- At present, when setting the video link for the video kernel player, it may play abnormally. There may be many types of exceptions. Here I simply divide them into three categories. It is roughly as follows:
@Retention(RetentionPolicy.SOURCE) public @interface ErrorType { //Wrong link int TYPE_SOURCE = 1; //Parsing exception int TYPE_PARSE = 2; //Other exceptions int TYPE_UNEXPECTED = 3; } - When is an error a link exception given by Google player
@Override public void onPlayerError(ExoPlaybackException error) { if (mPlayerEventListener != null) { int type = error.type; if (type == TYPE_SOURCE){ //Wrong link mPlayerEventListener.onError(PlayerConstant.ErrorType.TYPE_SOURCE,error.getMessage()); } } } - When an error occurs is a parsing exception. The ijk player gives an example, and the code is as follows
try { //Parse path Uri uri = Uri.parse(path); if (ContentResolver.SCHEME_ANDROID_RESOURCE.equals(uri.getScheme())) { RawDataSourceProvider rawDataSourceProvider = RawDataSourceProvider.create(mAppContext, uri); mMediaPlayer.setDataSource(rawDataSourceProvider); } else { //Handling UA issues if (headers != null) { String userAgent = headers.get("User-Agent"); if (!TextUtils.isEmpty(userAgent)) { mMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT, "user_agent", userAgent); } } mMediaPlayer.setDataSource(mAppContext, uri, headers); } } catch (Exception e) { mPlayerEventListener.onError(PlayerConstant.ErrorType.TYPE_PARSE,e.getMessage()); } - When an error occurs is another exception. The ijk player gives an example. The code is as follows. It is mainly the error monitoring given by the player. For details, see the source code of the video library.
/** * Set video error listener * int MEDIA_INFO_VIDEO_RENDERING_START = 3;//Video ready to render * int MEDIA_INFO_BUFFERING_START = 701;//Start buffering * int MEDIA_INFO_BUFFERING_END = 702;//End of buffer * int MEDIA_INFO_VIDEO_ROTATION_CHANGED = 10001;//Video selection information * int MEDIA_ERROR_SERVER_DIED = 100;//Video interruption is generally an exception of the video source or an unsupported video type. * int MEDIA_ERROR_IJK_PLAYER = -10000,//Generally, there is a problem with the video source or the data format is not supported. For example, the audio is not AAC * int MEDIA_ERROR_NOT_VALID_FOR_PROGRESSIVE_PLAYBACK = 200;//Data error no valid recycle */ private IMediaPlayer.OnErrorListener onErrorListener = new IMediaPlayer.OnErrorListener() { @Override public boolean onError(IMediaPlayer iMediaPlayer, int framework_err, int impl_err) { mPlayerEventListener.onError(PlayerConstant.ErrorType.TYPE_UNEXPECTED,"Listening exception"+ framework_err + ", extra: " + impl_err); VideoLogUtils.d("IjkVideoPlayer----listener---------onError -> STATE_ERROR -- what: " + framework_err + ", extra: " + impl_err); return true; } };
6.3 expansibility related design
- For example, you want to add a player with Tencent Video Kernel in the later stage. The code is as follows, which is simplified
public class TxPlayerFactory extends PlayerFactory<TxMediaPlayer> { public static TxPlayerFactory create() { return new TxPlayerFactory(); } @Override public TxMediaPlayer createPlayer(Context context) { return new TxMediaPlayer(context); } } public class TxMediaPlayer extends AbstractVideoPlayer { //Omit the implementation method code of the interface }