This article has authorized the exclusive release of Wechat Public No. Jingdong Technology.
Image Source Analysis of Flutter
With the rapid development of the hardware level of mobile devices, users are increasingly demanding for the display of pictures, and memory overflow can easily be caused by a slight mishandling. So when we use Image, it is normal to build a picture cache mechanism. Android currently provides a rich picture framework, such as Image Loader, Glide, Fresco and so on. For Flutter, in order to explore its caching mechanism or customize its own caching framework, we start with its Image.
Recently, in the study of Image caching in Flutter, in order to find the entrance or breakthrough of caching, I read the source code of Image once.
Image usage
Image is a control provided by Flutter to display pictures similar to ImageView in Android, but its use is somewhat similar to that of image frames such as Glide.
Let's first look at the use of Image. Flutter provides a variety of constructors for Image controls:
new Image, which is used to retrieve images from the Image Provider. new Image.asset, which is used to retrieve images from AssetBundle using key. new Image.network for getting images from URL addresses. new Image.file, which is used to get images from File.
We only analyze the source code of Image.network. After analyzing and understanding this, the other ideas are the same.
Let's start with the use of Image.network: it's very simple to display a network picture, and you can carry a url parameter directly through Image.network.
Example:
return new Scaffold(
appBar: new AppBar(
title: new Text("Image from Network"),
),
body: new Container(
child: new Column(
children: <Widget>[
// Load image from network
new Image.network(
'https://flutter.io/images/flutter-mark-square-100.png'),
],
)),
);
Image Structure UML Class Diagram
Let's first look at the UML class diagram of Image:
As you can see, the framework of Image is still a bit complicated. In fact, Flutter does a lot of work for you when you only call one line of code.
Preliminary carding of each category of concepts:
- Stateful Widget is a stateful Widget, which is an element displayed on a page.
- Image inherits from Stateful Widget to display and load images.
- State controls the life cycle of state changes in a Stateful Widget. When a Widget is created, the Widget configuration information changes, or the Widget is destroyed, a series of methods of State are invoked.
- _ ImageState inherits from State, handles state lifecycle changes and generates widgets.
- ImageProvider provides entries for loading images. Different image resources load in different ways, as long as the load method is rewritten. Similarly, the key value of the cached image is generated.
- NetWorkImage is responsible for downloading network pictures, transforming the downloaded pictures into ui.Codec objects and handing them to ImageStream Completer for processing and parsing.
- ImageStream Completer parses images frame by frame.
- ImageStream handles Image Resource, and ImageState connects with Image Stream Completer through ImageStream. ImageStream also stores monitored callbacks after the image has been loaded.
- MultiFrame Image Stream Completer is a multi-frame image parser.
The first step is to understand the framework of Image, which will help us to analyze the code more clearly.
Source code analysis
Let's take a look at what Image.network has done.
class Image extends StatefulWidget {
Image.network(String src, {
Key key,
double scale = 1.0,
this.width,
this.height,
this.color,
this.colorBlendMode,
this.fit,
this.alignment = Alignment.center,
this.repeat = ImageRepeat.noRepeat,
this.centerSlice,
this.matchTextDirection = false,
this.gaplessPlayback = false,
Map<String, String> headers,
}) : image = new NetworkImage(src, scale: scale, headers: headers),
assert(alignment != null),
assert(repeat != null),
assert(matchTextDirection != null),
super(key: key);
......
We see that Image is a StatefulWidget object that can be placed directly in containers such as Container or Column. Its properties are explained as follows:
- Width: widget width
- Height: height of widget
- Color: Used in conjunction with color BlendMode to blend images in BlendMode mode
- colorBlendMode: Hybrid Mode Algorithms
- fit: Like android:scaletype, control how the image resized/moved to match the size of the Widget
- Alignment: widget alignment
- repeat: How to draw parts that are not covered by an image
- Center Slice: Supports 9patch, the middle area of stretching
- matchTextDirection: Direction of drawing pictures: from left to right
- Gapless Playback: Whether to show old pictures or nothing when the pictures change
- headers: http request header
- image: An ImageProvide object, which has been instantiated at the time of invocation, is mainly responsible for loading images from the network. It is the most important way to load pictures. Different ways of loading pictures (assert file loading, network loading, etc.) are also the way to rewrite the ImageProvider to load pictures (load()).
Image is a StatefulWidget object, so let's look at its State object.
class _ImageState extends State<Image> {
ImageStream _imageStream;
ImageInfo _imageInfo;
bool _isListeningToStream = false;
}
class ImageStream extends Diagnosticable {
ImageStreamCompleter get completer => _completer;
ImageStreamCompleter _completer;
List<ImageListener> _listeners;
/// Assigns a particular [ImageStreamCompleter] to this [ImageStream].
void setCompleter(ImageStreamCompleter value) {
assert(_completer == null);
_completer = value;
print("setCompleter:::"+(_listeners==null).toString());
if (_listeners != null) {
final List<ImageListener> initialListeners = _listeners;
_listeners = null;
initialListeners.forEach(_completer.addListener);
}
}
/// Adds a listener callback that is called whenever a new concrete [ImageInfo]
void addListener(ImageListener listener) {
if (_completer != null)
return _completer.addListener(listener);
_listeners ??= <ImageListener>[];
_listeners.add(listener);
}
/// Stop listening for new concrete [ImageInfo] objects.
void removeListener(ImageListener listener) {
if (_completer != null)
return _completer.removeListener(listener);
assert(_listeners != null);
_listeners.remove(listener);
}
}
Let's explain the two attribute objects of _ImageState:
-
ImageStream handles Image Resource. ImageStream stores the monitored callbacks of loaded images, and ImageStream Completer is a member of it. So ImageStream handles the process of image analysis to ImageStream Completer.
-
ImageInfo contains information about the data sources of Image: width and height, and ui.Image.
Setting ui.Image in ImageInfo to RawImage will show you. RawImage is what we really render, a control that displays ui.Image, and we'll see that next.
We know the life cycle of a State, first the initState of the State is executed, then the didChange Dependencies are executed. We see that the initState of the parent class is not rewritten in the ImageState, so let's see its didChange Dependencies ():
@override
void didChangeDependencies() {
_resolveImage();
if (TickerMode.of(context))
_listenToStream();
else
_stopListeningToStream();
super.didChangeDependencies();
}
_ Resolution Image Method Analysis
We see that the _resolveImage() is called first. Let's look at the _resolveImage method:
void _resolveImage() {
final ImageStream newStream =
widget.image.resolve(createLocalImageConfiguration(
context,
size: widget.width != null && widget.height != null ? new Size(widget.width, widget.height) : null
));
assert(newStream != null);
_updateSourceStream(newStream);
}
This method is to process the entrance of the picture. widget.image, which is the NetworkImage object created above, is an ImageProvider object that calls its resolve and passes in the default ImageConfiguration.
We looked at the resolve method and found that NetworkImage did not exist. If not, we found it in its parent class, ImageProvider:
ImageStream resolve(ImageConfiguration configuration) {
assert(configuration != null);
final ImageStream stream = new ImageStream();
T obtainedKey;
obtainKey(configuration).then<void>((T key) {
obtainedKey = key;
stream.setCompleter(PaintingBinding.instance.imageCache.putIfAbsent(key, () => load(key)));
}).catchError(
(dynamic exception, StackTrace stack) async {
FlutterError.reportError(new FlutterErrorDetails(
exception: exception,
stack: stack,
library: 'services library',
context: 'while resolving an image',
silent: true, // could be a network error or whatnot
informationCollector: (StringBuffer information) {
information.writeln('Image provider: $this');
information.writeln('Image configuration: $configuration');
if (obtainedKey != null)
information.writeln('Image key: $obtainedKey');
}
));
return null;
}
);
return stream;
}
We see that this method creates an ImageStream and returns, calls obtainKey to return a future with Network Image, which will be used later as a cached key, and calls the setCompleter method of ImageStream.
void setCompleter(ImageStreamCompleter value) {
assert(_completer == null);
_completer = value;
if (_listeners != null) {
final List<ImageListener> initialListeners = _listeners;
_listeners = null;
initialListeners.forEach(_completer.addListener);
}
}
This method is to set an ImageStream Completer object for ImageStream. Each ImageStream object can only be set once. ImageStream Completer is designed to assist ImageStream in parsing and managing Image frames, and to determine whether there is an initialization listener and do some initialization callbacks.
Let's continue with the PaintingBinding.instance.imageCache.putIfAbsent method
ImageStreamCompleter putIfAbsent(Object key, ImageStreamCompleter loader()) {
assert(key != null);
assert(loader != null);
ImageStreamCompleter result = _pendingImages[key];
// Nothing needs to be done because the image hasn't loaded yet.
if (result != null)
return result;
// Remove the provider from the list so that we can move it to the
// recently used position below.
final _CachedImage image = _cache.remove(key);
if (image != null) {
_cache[key] = image;
return image.completer;
}
result = loader();
void listener(ImageInfo info, bool syncCall) {
// Images that fail to load don't contribute to cache size.
final int imageSize = info.image == null ? 0 : info.image.height * info.image.width * 4;
final _CachedImage image = new _CachedImage(result, imageSize);
_currentSizeBytes += imageSize;
_pendingImages.remove(key);
_cache[key] = image;
result.removeListener(listener);
_checkCacheSize();
}
if (maximumSize > 0 && maximumSizeBytes > 0) {
_pendingImages[key] = result;
result.addListener(listener);
}
return result;
}
This is the entry method of the memory caching api provided by Flutter by default. This method first obtains the previous ImageStreamCompleter object by key. This key is the NetworkImage object. Of course, we can also override the obtainKey method to customize the key. If it exists, it returns directly. If it does not exist, load method is executed to load I. MageStreamCompleter object, and put it first (least recently used algorithm). That is to say, ImageProvider has implemented memory caching: the maximum number of default cached pictures is 1000, and the maximum space of default cached pictures is 10MiB.
The first time we load an image is definitely not cached, so let's look at the loader method. We see that the ImageProvider is an empty method. Let's look at NetWorkImage. As we expected, it's really here:
@override
ImageStreamCompleter load(NetworkImage key) {
return new MultiFrameImageStreamCompleter(
codec: _loadAsync(key),
scale: key.scale,
informationCollector: (StringBuffer information) {
information.writeln('Image provider: $this');
information.write('Image key: $key');
}
);
}
//The Method of Network Request Loading Pictures
Future<ui.Codec> _loadAsync(NetworkImage key) async {
assert(key == this);
final Uri resolved = Uri.base.resolve(key.url);
final HttpClientRequest request = await _httpClient.getUrl(resolved);
headers?.forEach((String name, String value) {
request.headers.add(name, value);
});
final HttpClientResponse response = await request.close();
if (response.statusCode != HttpStatus.ok)
throw new Exception('HTTP request failed, statusCode: ${response?.statusCode}, $resolved');
final Uint8List bytes = await consolidateHttpClientResponseBytes(response);
if (bytes.lengthInBytes == 0)
throw new Exception('NetworkImage is an empty file: $resolved');
return await ui.instantiateImageCodec(bytes);
}
This method creates a MultiFrame ImageStreamCompleter object for us, and we can also know by name that it inherits from ImageStreamCompleter. Remember what ImageStream Completer does to help manage and parse ImageStream.
Parametric analysis:
- _ Load Async () is a method of requesting the network to load pictures.
- scale is the scaling factor.
- Information Collector is the object of information collection, providing errors or other logging.
MultiFrame Image Stream Completer is a multi-frame image processing loader. We know that Flutter's Image supports loading gif. Through MultiFrame Image Stream Completer, GIF files can be parsed:
MultiFrameImageStreamCompleter({
@required Future<ui.Codec> codec,
@required double scale,
InformationCollector informationCollector
}) : assert(codec != null),
_informationCollector = informationCollector,
_scale = scale,
_framesEmitted = 0,
_timer = null {
codec.then<void>(_handleCodecReady, onError: (dynamic error, StackTrace stack) {
FlutterError.reportError(new FlutterErrorDetails(
exception: error,
stack: stack,
library: 'services',
context: 'resolving an image codec',
informationCollector: informationCollector,
silent: true,
));
});
}
ui.Codec _codec;
final double _scale;
final InformationCollector _informationCollector;
ui.FrameInfo _nextFrame;
We see that MultiFrame ImageStream Completer takes the codec data object returned by _loadAsync, processes the data through _handleCodecReady, and then calls the _decodeNextFrameAndSchedule method.
Future<Null> _decodeNextFrameAndSchedule() async {
try {
_nextFrame = await _codec.getNextFrame();
} catch (exception, stack) {
FlutterError.reportError(new FlutterErrorDetails(
exception: exception,
stack: stack,
library: 'services',
context: 'resolving an image frame',
informationCollector: _informationCollector,
silent: true,
));
return;
}
if (_codec.frameCount == 1) {
// This is not an animated image, just return it and don't schedule more
// frames.
_emitFrame(new ImageInfo(image: _nextFrame.image, scale: _scale));
return;
}
SchedulerBinding.instance.scheduleFrameCallback(_handleAppFrame);
}
Get the next frame through _codec.getNextFrame(). For the static frame Count is 1, assemble the image directly with ImageInfo and hand it to the _emitFrame method, which calls setImage, as follows:
@protected
void setImage(ImageInfo image) {
_current = image;
if (_listeners.isEmpty)
return;
final List<ImageListener> localListeners = new List<ImageListener>.from(_listeners);
for (ImageListener listener in localListeners) {
try {
listener(image, false);
} catch (exception, stack) {
_handleImageError('by an image listener', exception, stack);
}
}
}
The setImage method is to set up the current ImageInfo and check the list of listeners to inform the listener that the picture has been loaded and the UI can be refreshed.
For motion maps, it is to give Scheduler Binding frame by frame to call setImage, notify the UI to refresh, the code will not be pasted, interested can view it on their own.
So far, we're done with the _resolveImage call process. Next let's look at _listenToStream:
_ Analysis of listenToStream Method
We continue to analyze the didChange Dependencies method, which judges the value of TickerMode.of(context), which defaults to true, and is related to the Anmation Conrol, which can be studied in depth later. Then call _listenToStream().
Let's look at this method:
void _listenToStream() {
if (_isListeningToStream)
return;
_imageStream.addListener(_handleImageChanged);
_isListeningToStream = true;
}
This is a callback that adds the loaded image. Remember, when the image is loaded and parsed, the setImage method of MultiFrameImageStreamCompleter calls the callback method passed here. Let's see what's done in the callback method here.
void _handleImageChanged(ImageInfo imageInfo, bool synchronousCall) {
setState(() {
_imageInfo = imageInfo;
});
}
Obviously, you get the upper layer and pass it over to ImageInfo, calling setState to update the UI.
Let's look at the build method:
Widget build(BuildContext context) {
return new RawImage(
image: _imageInfo?.image,
width: widget.width,
height: widget.height,
scale: _imageInfo?.scale ?? 1.0,
color: widget.color,
colorBlendMode: widget.colorBlendMode,
fit: widget.fit,
alignment: widget.alignment,
repeat: widget.repeat,
centerSlice: widget.centerSlice,
matchTextDirection: widget.matchTextDirection,
);
}
RawImage is encapsulated with the information of imageInfo and widget. RawImage is the RenderObjectWidget object, which is really rendered by the application. It displays our pictures on the interface.
summary
Comb the following process:
- Starting with the entry, Image inherits from StatefulWidget, which implements State:_ImageState for us, and provides an instantiated NetWorkImage object, which inherits from the ImageProvider object.
- _ After the ImageState is created, _ImageState returns an ImageStream object by calling _resolveImage(), _resolveImage() and the Resolution () method of ImageProvider. _ ImageState also registered a listener for ImageStream, which performs a callback method when the image is downloaded.
- Then in the resolution () method of the ImageProvider, not only the ImageStream is created, but also the setComplete method of the ImageStream is set to set the ImageStream Completer, where we can judge whether there is a cache or not. Without the cache, we call the load method to create the ImageStream Completer and add a listener to execute the Image after loading. Caching work. ImageStream Completer is designed to parse the loaded Image.
- NetWorkImage implements ImageProvider's load method, which is the real place to download pictures. It creates a MultiFrame ImageStreamCompleter object and calls _loadAsync to download pictures. When the picture is downloaded, the callback method of the UI is called to notify the UI to refresh.
Last
So far, the source code analysis of Image.network is over. You can also go back and see the structure of Image. What's more, after the analysis, you have a good understanding of how Flutter loads network pictures, and you have found a breakthrough in Flutter's image caching. Flutter itself has provided memory caching (though not perfect). Then you can add your hard disk caching or customize your picture framework.