01

Background

At present, various styles of rich media controls are complicated, and the interface methods are not unified. Developers often use rich media controls. When developing, they need to pay more attention to details, such as understanding the api request methods of various controls in advance.

In addition, rich media is developing rapidly, and rapid iteration breeds many and miscellaneous media controls (there are many kinds of picture and video controls in the project).
The gap in the current environment is that there is no unified encapsulated rich media dynamic control in the whole market, there is no overall rich media resource caching mechanism at the whole application level, and there is no comprehensive rich media control that forms an efficient reuse with list controls.

02

Target

The unified rich media dynamic control is designed to meet two scenarios:

1. Solve the efficient switching of complex media in a single operation scenario, and one control can meet a variety of effects;

2. Solve the uniform and efficient reuse of templates in the form of feed stream list, and reduce the number of templates and views.

03

Revenue

1. A single control has rich functions, which will simplify the design and development of Feed flow template; 2. After the unified API call, the development and learning cost of rich media controls will be greatly reduced.

04

Research

JD.COM 1. Basic database research

Picture loading Library: fresco

Dynamic synthesis Library: lottie, ValueAnimator

Video playback Library: ijkplayer

2.feed flow research

com.jingdong.app.mall.home.floor.view.linefloor.widget.VideoSkuView / IjkVideoView

3. The dynamic effect is basically not encapsulated, and attribute animation is widely used

TaoBao

1. Basic database research

Image loading Library: ImageLoadFeature

Dynamic synthesis Library: lottie, HGifView

Video playback Library: ijkplayer

2.feed flow research

com.taobao.homepage.view.widgets.MultiLayout / HGifView,HVideoView

3. The video and GIF are encapsulated into a rich layout MultiLayout
HGifView and HImageView integrate the same parent component TUrlImageView and use the same resource loader ImageLoadFeature

Research results

The summary of the two e-commerce companies in the following four aspects can be summarized as follows:

1. On the static diagram, Taobao encapsulated the picture components, and JD directly used the picture components provided by fresco library;

2. In terms of dynamic effect, Taobao has made packaging based on picture components, and JD mostly uses attribute animation to produce animation effect;

3. The lottie automation view provided by the L library is directly used for lottie processing, and there is no secondary encapsulation;

4. In video, both use the open source ijkplayer library to customize their own video loading and playing components;

Initial output

Based on the research on the two e-commerce and our own business, our output results are as follows:

• MultiImageView is used for static diagram, and Fresco or Glide is used for third-party library
  

• GIF uses MultiGifView, and the third-party library uses Fresco
  

• Multi lottieview is used for dynamic effect, and Lottie is selected for the third-party library
  

• Banner uses MultiPagerView and ViewPager to produce banner effect
  

• MultiVideoView is used for video, and LeadingVideoView is selected for one party library to support loading local resources
  

• The live broadcast uses MultiLiveView, and the second-party library uses WPlayer
  

• The whole is uniformly assembled into WubaMultiView to produce a unified dynamic control

05

Design

After the requirements are clear, three problems should be solved in the design:

• Uniformity: the unification of interfaces and templates called by different media types;

• Reusability: cache reuse of rich media controls at the whole APP level;

• Efficiency: how to cooperate with RecycleView to further penetrate into the control level to achieve efficient reuse.

1. Uniformity

Firstly, the interface call method is highly encapsulated and unified, so that the external callers can realize a unified call method for the original diversified dynamic request methods, and the control itself can be loaded on demand and is highly cohesive;
In addition, new styles can be extended.
Control realizes the whole chain unification from resource request, playback interface to list template through unified interface encapsulation and flexible switching of media form.

2. Reusability

Give rich media a variety of item caching and reuse capabilities. Through a global static media control cache pool and user-defined size, the reuse of rich media controls at the APP level is realized.

3. Efficiency

By combining with the caching strategy of RecycleView, it meets a variety of rich media dynamic effect styles. In the list view display, it completes the efficient dual reuse from the primary template layout to the secondary dynamic effect control;
There is no need to create and maintain redundant rich media dynamic effect templates, and there is no need to spend additional memory space for dynamic effect controls that are not displayed.

Overall control design flow chart:

06

Realize

1. Uniformity

1) The control has enumerated single graph (supporting Webp format), GIF graph, Lottie graph, Pager customized dynamic effect and Video video. It supports expansion capability. Developers can also continue to access other customized media dynamic effects.

public final enum class MediaType private constructor() : kotlin.Enum<MediaType> {    IMAGE,
    GIF,
    LOTTIE,
    PAGER,
    VIDEO;}

2) The control carries out unified interface encapsulation for these different types of dynamic media, realizes a high degree of unity of request loading and playback control, and only provides four basic method calls visible in the flow chart:

1) Set current media type

2) Set URL request

3) Playback control

4) Set cache switch

Through simple method encapsulation and internal unified interface, the high cohesion of the control is completed.

3) The API provided externally is simpler than other single media controls:

// Set media type ABCD setmediatype (mediatype: mediatype)
// Get the current media interface fun getiview(): iView?

// Set the requested address url... fun setUrl(vararg url: String)
// Set clipping type ABCD setactualimagescaletype (scaletype: scalingutils. Scaletype)
// Set the default graph and its clipping type fun setPlaceholderImage(resourceId: Int, scaleType: ScalingUtils.ScaleType?)
// Set request result callback fun setOnRequestStatusListener(listener: WubaMultiView.OnRequestStatusListener)
// Dynamic play (fun play)
// Action stop (fun stop)
// Fu pause ()
// Whether the motion effect is playing fun isPlaying(): Boolean

// Set cache fun setUseCache(useCache: Boolean)

At the same time, the interface implementation method of internal media components is unified and simple to understand:

/** * Unified dynamic interface, abstract class */interface IView {
    fun getMediaType() : MediaType
    fun getView(context: Context) : View
    fun setUrl(vararg url: String) {        val list = ArrayList<String>()        list.addAll(url)        setUrl(list)    }
    fun setUrl(urls: List<String>)
    fun setActualImageScaleType(scaleType: ScalingUtils.ScaleType)
    fun setPlaceholderImage(resourceId: Int, scaleType: ScalingUtils.ScaleType?)
    fun setFailureImage(resourceId: Int, scaleType: ScalingUtils.ScaleType?)
    fun play()
    fun pause()
    fun stop()
    fun isPlaying() : Boolean
    fun setOnRequestStatusListener(listener: WubaMultiView.OnRequestStatusListener)
}

2. Reusability

1) The control sets the global cache policy. The cache pool is designed based on the whole APP by enumerating class static member variables. It can realize APP level cache and has the ability of global reuse.

enum class MediaType {
    IMAGE {
        override fun getViewClass(): Class<IView> {
            return MultiImageView::class.java as Class<IView>
        }
        override fun getViewCacheSize(): Int {
            return 10
        }
    },
    ...

    companion object {
        private val CACHE_VIEWS = mutableMapOf<String, ArrayList<IView>>()
    }

    internal abstract fun getViewClass() : Class<IView>

    internal abstract fun getViewCacheSize() : Int

    internal fun getView(useCache: Boolean): IView {
        var v: IView? = null
        if (useCache && CACHE_VIEWS.containsKey(this.name) && (CACHE_VIEWS[this.name]?.size ?: 0) > 0) {
            v = CACHE_VIEWS[this.name]?.removeLast()
        }
        return v ?: getViewClass().newInstance()
    }

    internal fun addViewCache(view: IView) {
        if (!CACHE_VIEWS.containsKey(this.name)) {
            CACHE_VIEWS[this.name] = arrayListOf<IView>()
        }
        if ((CACHE_VIEWS[this.name]?.size?:0) < this.getViewCacheSize()) {
            CACHE_VIEWS[this.name]?.add(view)
        }
    }
    
}

2) At the same time, it supports opening and closing the cache. For some specific scenes or media controls with less reuse frequency, you can temporarily close the cache policy, and support setting the cache pool size of each media control, which is very flexible.

See the left part of the unified rich media dynamic control design flow chart for the specific caching strategy.

3. Efficiency

RecycleView realizes efficient reuse at the template level through multi-level caching strategy, as shown in the figure below. In the figure, a template corresponds to a type value.

1) When the list is drawn out of the mobile phone screen, the drawn template will enter the cache pool:

 

2) For templates that are transferred to the screen, first consider extracting and reusing them from the cache pool:

3. After using the unified rich media control, each dynamic effect template will be laid out with this control, and the value of the corresponding type will be reduced to 1 (there is only one template left). The simplified cache strategy of RecycleView is shown in the following figure:

4. Then, by setting different rich media types, different rich media styles are displayed with the same set of templates. Unified rich media control combined with RecycleView will form a four level cache policy.

It feels like there is an additional cache level. In fact, the control has previously reduced the template type (i.e. type value) of RecycleView. In fact, the cache level has not increased, but there is only one more layer logically.

07

Use

XML configuration

Only note: when configuring in XML, you need to set the media type, that is, media_type=“xxx”

<com.wuba.views.media.WubaMultiView    android:id="@+id/iv_feed_nopic_avator"    android:layout_width="30dp"    android:layout_height="30dp"    android:layout_alignParentEnd="true"    android:layout_marginTop="2.5dp"    app:media_type="IMAGE"    app:scaleType="centerCrop"    app:placeholderImage="@drawable/xxx"    app:placeholderImageScaleType="centerCrop"    app:failureImage="@drawable/xxx"    app:failureImageScaleType="centerCrop"/>

Code prompt

Preview effect

Memory fluctuation

Modification of coupon suspension window

1. The memory fluctuated twice before the transformation

2. After the transformation, the memory fluctuation is reduced once

The fluctuation before the transformation is due to the fact that two media controls are used in the scene to carry different effects respectively, and the static picture control and Lottie control will be loaded at the same time;

The modified control will only load one user specified media control at the same time, so the fluctuation in memory will only occur once at most.

08

Summary

1. Use system A rich media dynamic effect control can simplify the template of RecycleView, which can realize efficient reuse of item s, thus greatly reducing the cost of development and maintenance.

2. Using the unified rich media dynamic control, the rich media control can be reused as a whole at the APP level, so as to reduce the memory consumption of the APP.

3. Using the unified rich media dynamic control, there are few interfaces exposed to developers. It is easy to get started with rich media related business development. Simple api design produces rich media dynamic effects.