Glide caching mechanism
Glide cache mechanism
- Glide's cache is divided into two types: one is memory cache and the other is hard disk cache. The memory cache includes weak reference cache HashMap and LruCache, and the hard disk cache is DiskLruCache. If there is no URL in the cache and no URL in the cache on the hard disk, the request is to find the image in the cache first. If there is no URL in the cache on the hard disk, and then get the image in the cache through the cache on the network.
Fetching resources from memory cache
In the introduction of the previous blog, we know that the Glide request process only starts loading logic by calling the load method of the Engine at the end, so naturally, the operation of reading cache also starts from the load method of the Engine:
public synchronized <R> LoadStatus load(......){
// Get the key of the resource. There are 8 parameters
EngineKey key = keyFactory.buildKey(model, signature, width, height, transformations,
resourceClass, transcodeClass, options);
EngineResource<?> memoryResource;
synchronized (this) {
//Get from memory
memoryResource = loadFromMemory(key, isMemoryCacheable, startTime)
}
// If none can be obtained, go to the hard disk cache or network load
....
}
}
From the above lines of code, we divide the steps of memory caching into the following steps to analyze
Step 1: get cache key
// Get the key of the resource. There are 8 parameters
EngineKey key = keyFactory.buildKey(model, signature, width, height, transformations,
resourceClass, transcodeClass, options);
We see that when creating a key, 8 parameters will be passed in, so if we want to get the same picture every time, but the size is different, a new cache key will also be generated. To cache.
-
Glide turns on the memory cache by default. If we want to turn off the memory cache, we can call:
-
//Turn off memory cache skipMemoryCache(false)
Step 2: get from the weak reference cache
EngineResource<?> active = loadFromActiveResources(key, isMemoryCacheable);
The loadFromActiveResources() method was called.
loadFromActiveResources()
@Nullable
private EngineResource<?> loadFromActiveResources(Key key, boolean isMemoryCacheable) {
if (!isMemoryCacheable) {
return null;
}
//Go to weak reference activeResources and get it from the HashMap
EngineResource<?> active = activeResources.get(key);
//If you can get the resource, call the acquire() method to make the counter + 1
if (active != null) {
active.acquire();
}
return active;
}
@Nullable
synchronized EngineResource<?> get(Key key) {
ResourceWeakReference activeRef = activeEngineResources.get(key);
if (activeRef == null) {
return null;
}
EngineResource<?> active = activeRef.get();
//If this weak reference object is not queried, delete this key
if (active == null) {
cleanupActiveReference(activeRef);
}
return active;
}
synchronized void acquire() {
if (isRecycled) {
throw new IllegalStateException("Cannot acquire a recycled resource");
}
++acquired;
}
This method is to get the weakly referenced objects through the key in the HashMap of weakly referenced active resources. If the key cannot be obtained, the formation may be GC, and the key will be removed from the map; If it is obtained, call the acquire () method to make the counter acquire plus 1, which means it is used. We will summarize the application of the counter at the end.
Step 3: get from memory LruCache
// Get object from lrucache EngineResource<?> cached = loadFromCache(key, isMemoryCacheable);
The loadFromCache method was called
loadFromCache()
private EngineResource<?> loadFromCache(Key key, boolean isMemoryCacheable) {
if (!isMemoryCacheable) {
return null;
}
//Get resources from memory according to key
EngineResource<?> cached = getEngineResourceFromCache(key);
//If the resource is obtained, the counter is incremented by 1 and stored in the weak reference cache
if (cached != null) {
cached.acquire();
activeResources.activate(key, cached);
}
return cached;
}
getEngineResourceFromCache()
private EngineResource<?> getEngineResourceFromCache(Key key) {
//Call the remove method to get the resource, that is, after obtaining the resource, the resource will be removed from LruCache
Resource<?> cached = cache.remove(key);
final EngineResource<?> result;
if (cached == null) {
result = null;
} else if (cached instanceof EngineResource) {
//Save an object allocation if we've cached an EngineResource (the typical case).
result = (EngineResource<?>) cached;
} else {
result =
new EngineResource<>(
cached, /*isMemoryCacheable=*/ true, /*isRecyclable=*/ true, key, /*listener=*/ this);
}
return result;
}
- Through the above two methods, we know that to obtain resources from LruCache, we first obtain resources through the remove method, then remove the resources from LruCache, increase its counter by 1, and store the resources in the weak cache.
So far, we have gone through the process of fetching resources from the memory cache: first, get the object from the weak reference. If it exists, the reference count is + 1. If it does not exist, get it from the LruCache. If it does exist, the reference count is + 1, store it in the weak reference, and remove itself from the LruCache.
Memory resources of memory cache
Storage resources must also be stored in the load method of the image loading method Engine. Let's take a look:
Engine.load()
public synchronized <R> LoadStatus load(
..
// Get key
EngineKey key = keyFactory.buildKey(model, signature, width, height, transformations,
resourceClass, transcodeClass, options);
// Get object from weak reference
EngineResource<?> active = loadFromActiveResources(key, isMemoryCacheable);
...
// Get object from LruCache
EngineResource<?> cached = loadFromCache(key, isMemoryCacheable);
...
EngineJob<R> engineJob =
engineJobFactory.build(
key,
isMemoryCacheable,
useUnlimitedSourceExecutorPool,
useAnimationPool,
onlyRetrieveFromCache);
DecodeJob<R> decodeJob =
decodeJobFactory.build(
glideContext,
model,
key,
signature,
width,
height,
resourceClass,
transcodeClass,
priority,
diskCacheStrategy,
transformations,
isTransformationRequired,
isScaleOnlyOrNoTransform,
onlyRetrieveFromCache,
options,
engineJob);
jobs.put(key, engineJob);
engineJob.addCallback(cb, callbackExecutor);
engineJob.start(decodeJob);
...
}
Two key objects will be created here. One is EngineJob, which is a thread pool that maintains coding, resource resolution, network download, etc; The other is DecodeJob, which is used to decode pictures. It inherits Runnable, which is equivalent to a task of EngineJob. Then call EngineJob.. Start (DecodeJob). Let's not talk about the details first. After the method is completed, the onResourceReady () method will be called back
onResourceReady()
@Override
public void onResourceReady(Resource<R> resource, DataSource dataSource) {
synchronized (this) {
this.resource = resource;
this.dataSource = dataSource;
}
notifyCallbacksOfResult();
}
notifyCallbacksOfResult()
@Synthetic
void notifyCallbacksOfResult() {
ResourceCallbacksAndExecutors copy;
Key localKey;
EngineResource<?> localResource;
synchronized (this) {
stateVerifier.throwIfRecycled();
//Cancelled
if (isCancelled) {
resource.recycle();
release();
return;
} else if (cbs.isEmpty()) {
throw new IllegalStateException("Received a resource without any callbacks to notify");
} else if (hasResource) {
throw new IllegalStateException("Already have resource");
}
engineResource = engineResourceFactory.build(resource, isCacheable);
hasResource = true;
copy = cbs.copy();
// Reference count + 1
incrementPendingCallbacks(copy.size() + 1);
localKey = key;
localResource = engineResource;
}
// put object on weak reference
listener.onEngineJobComplete(this, localKey, localResource);
// Traverse all pictures
for (final ResourceCallbackAndExecutor entry : copy) {
// Load resources into imageview, reference count + 1
entry.executor.execute(new CallResourceReady(entry.cb));
}
//Reference count - 1
decrementPendingCallbacks();
}
From the above, the notifyCallbacksOfResult() method does the following
- Reference count of pictures + 1
- Through listener Onenginejobcomplete(), whose callback is engine Onenginejobcomplete(), put ting resources on weak references
- Traverse the loaded image. If the loading is successful, the reference count is + 1 and passes CB Onresourceready (engine resource, datasource) callback to target (imageview) to load
- Release resources through decrementPendingCallbacks(), reference count - 1
When the reference count is reduced to 0, that is, the image is no longer used, the onResourceReleased() interface will be called
onResourceReleased()
@Override
public synchronized void onResourceReleased(Key cacheKey, EngineResource<?> resource) {
// Remove from weak references
activeResources.deactivate(cacheKey);
if (resource.isCacheable()) {
//Add to LruCache
cache.put(cacheKey, resource);
} else {
// Recycling resources
resourceRecycler.recycle(resource);
}
}
Disk cache
If you can't get the picture in the memory cache, Glide will judge from two aspects:
- One is the resource type: whether the picture has been decoded, converted and written to the disk cache before.
- One is the data source: whether the resources that build this image have been written to the file cache before.
Our hard disk cache; Glide's hard disk strategy can be divided into the following types:
- DiskCacheStrategy.RESOURCE: only decoded pictures are cached
- DiskCacheStrategy.DATA: cache only the original picture
- DiskCacheStrategy.ALL: cache both the original picture and the decoded picture. For remote pictures, cache DATA and RESOURCE; Cache only RESOURCE for local use.
- DiskCacheStrategy.NONE: do not use hard disk cache
- DiskCacheStrategy.AUTOMATIC: the default policy, which will use the best policy for local and remote pictures; For downloading network pictures, use DATA; for local pictures, use RESOURCE
When describing the source code of disk cache, attach a flow chart:
As we know from the above, an image is loaded in the class DecodeJob, and this task is executed by the thread pool of EngineJob. Go to the run method and you can see a runWrapped() method:
runWrapped()
private void runwrapped() {
switch (runReason){
case INITIALIZE:
stage = getNextstage(stage.INITIALIZE);
currentGenerator = getNextGenerator();
runGenerators();
break;
case swITCH_To_sOURCE_SERVICE:
runGenerators();
break;
case DECODE_DATA:
decodeFromRetrievedData();break;
default:
throw new IllegalstateException("Unrecognized run reason: " + runReason);
}
At first, runReason is initialized to INITIALIZE, so it will take the first case.
private Stage getNextStage(Stage current) {
switch (current) {
case INITIALIZE:
return diskCacheStrategy.decodeCachedResource()
? Stage.RESOURCE_CACHE : getNextStage(Stage.RESOURCE_CACHE);
case RESOURCE_CACHE:
return diskCacheStrategy.decodeCachedData()
? Stage.DATA_CACHE : getNextStage(Stage.DATA_CACHE);
case DATA_CACHE:
// Skip loading from source if the user opted to only retrieve the resource from cache.
return onlyRetrieveFromCache ? Stage.FINISHED : Stage.SOURCE;
case SOURCE:
case FINISHED:
return Stage.FINISHED;
default:
throw new IllegalArgumentException("Unrecognized stage: " + current);
}
}
getNextStage is actually to judge the current cache strategy, because our strategy is diskcachestrategy All, so diskcachestrategy Decodecachedresource() is always true, that is, it will parse the decoding process, that is, try to get the decoded picture resources in the disk, so the State is assigned to stage RESOURCE_ Cache (State is a tag). Then go back to the runWrapped () method, continue to execute the call currentGenerator = getNextGenerator(), get the current decoder ResourceCacheGenerator, and then call the runGenerators() method, which is the key:
runGenerators()
private void runGenerators () {
currentThread = Thread.currentThread( );
startFetchTime = LogTime.getLogTime( );
boolean isstarted = false;
while ( !iscancelled && currentGenerator != null && ! (isstarted = currentGenerator. startNext())){
stage = getNextstage(stage);
currentGenerator = getNextGenerator();
if (stage == stage.SOURCE) {
reschedule();
return;
}
if ((stage == stage.FINISHED || iscancelled) && !isstarted){
notifyFailed();
}
}
A while loop is maintained, which constantly tries to get the object from DiskLruCache by calling the startNext() method of the current decoder. If it cannot be obtained, it gets the next decoder through the while loop through the getNextstage(stage) and getNextGenerator() methods, and looks for it again when stage = = stage During source, reschedule() will be called to re call the run method of DecodeJob and re run the RunWrapper method.
startNext()
@Override
public boolean startNext() {
List<Key> sourceIds = helper.getCacheKeys();
...
while (modelLoaders == null || !hasNextModelLoader()) {
resourceClassIndex++;
if (resourceClassIndex >= resourceClasses.size()) {
sourceIdIndex++;
// Since there is no cache, the loop will eventually exit here
if (sourceIdIndex >= sourceIds.size()) {
return false;
}
resourceClassIndex = 0;
}
Key sourceId = sourceIds.get(sourceIdIndex);
//Get cache key
currentKey =
new ResourceCacheKey(// NOPMD AvoidInstantiatingObjectsInLoops
helper.getArrayPool(),
sourceId,
helper.getSignature(),
helper.getWidth(),
helper.getHeight(),
transformation,
resourceClass,
helper.getOptions());
// Trying to get data from DiskLruCache
cacheFile = helper.getDiskCache().get(currentKey);
if (cacheFile != null) {
sourceKey = sourceId;
modelLoaders = helper.getModelLoaders(cacheFile);
modelLoaderIndex = 0;
}
}
loadData = null;
boolean started = false;
while (!started && hasNextModelLoader()) {
ModelLoader<File, ?> modelLoader = modelLoaders.get(modelLoaderIndex++);
loadData = modelLoader.buildLoadData(cacheFile,
helper.getWidth(), helper.getHeight(), helper.getOptions());
if (loadData != null && helper.hasLoadPath(loadData.fetcher.getDataClass())) {
started = true;
loadData.fetcher.loadData(helper.getPriority(), this);
}
}
return started;
}
Because the cache cannot be obtained for the first time, the while (modelloaders = = null |! Hasnextmodelloader()) loop will run until false is returned.
Similarly, it is the same after getting the DataCacheGenerator. Finally, when stage = = stage It will exit and call the reschedule() method.
@Override
public void reschedule() {
runReason = RunReason.SWITCH_TO_SOURCE_SERVICE;
callback.reschedule(this);
}
In the reschedule() method, the state of runReason will be changed to runReason SWITCH_TO_SOURCE_SERVICE and call back the run method again, and the runWrapped() method will be called again, but the runReason has become switch at this time_ TO_ SOURCE_ Service, so it will execute the runGenerators() method (the code above has):
At this time, the current decoder currentGenerator is not null, so the startNext() method of SourceGenerator will be called:
@Override
public boolean startNext() {
if (dataToCache != null) {
Object data = dataToCache;
dataToCache = null;
// Store to DiskLruCache
cacheData(data);
}
if (sourceCacheGenerator != null && sourceCacheGenerator.startNext()) {
return true;
}
sourceCacheGenerator = null;
loadData = null;
boolean started = false;
while (!started && hasNextModelLoader()) {
loadData = helper.getLoadData().get(loadDataListIndex++);
if (loadData != null
&& (helper.getDiskCacheStrategy().isDataCacheable(loadData.fetcher.getDataSource())
|| helper.hasLoadPath(loadData.fetcher.getDataClass()))) {
started = true;
//Load data
loadData.fetcher.loadData(helper.getPriority(), this);
}
}
return started;
}
First, it will determine whether dataToCache is null. It will be null for the first time, so it can be ignored first. Since sourceCacheGenerator has not been assigned a value, it is null. You can directly look at loaddata fetcher. loadData(); For this method, loaddata () is an interface. It has many implementation methods. Because we download it from the network, we go to httpurlfetcher In loaddata():
loadData()
@override
public void loadData(@NonNull Priority priority,
@NonNull Datacallback<?super Inputstream> callback){
long startTime = LogTime.getLogTime( );
try {
Inputstream result = loadDatawithRedirects(glideur1.toURL()，0，null，glideur1.getHeaders());
callback.onDataReady(result);
}catch (TOException e) {
if (Log.isLoggable(TAG，Log.DEBUG))
Log.d(TAG，msg: "Failed to load data for ur1", e);
}
}
You can see that after it gets the inputStream, it calls back through the onDataReady() method
@Override
public void onDataReady(Object data) {
DiskCacheStrategy diskCacheStrategy = helper.getDiskCacheStrategy();
if (data != null && diskCacheStrategy.isDataCacheable(loadData.fetcher.getDataSource())) {
dataToCache = data;
// We might be being called back on someone else's thread. Before doing anything, we should
// reschedule to get back onto Glide's thread.
cb.reschedule();
} else {
cb.onDataFetcherReady(loadData.sourceKey, data, loadData.fetcher,
loadData.fetcher.getDataSource(), originalKey);
}
}
In sourcegenerator In ondataready(), assign a value to dataToCache; Then call CB The reschedule () method starts all over again, but this time dataToCache is not null, so the cacheData method will be executed (the code logic is shown above):
private void cacheData(object dataTocache){
long startTime = LogTime.getLogTime();
try {
Encoder<object> encoder = helper.getsourceEncoder(dataToCache);
Datacachewriter<object> writer=new Datacachewriter<>(encoder， dataTocache，helper.getoptions());
originalKey = new DatacacheKey(loadData.sourceKey，helper.getsignature());
helper.getDiskcache().put(originalKey,writer);
if (Log.isLoggable( TAG，Log.VERBOSE))
Log.v(TAG，msg: "Finished encoding source to cache"
+ ",key: " + originalKey
+ ", data: " + dataTocache+ ", encoder: " +encoder
+ ", duration: " + LogTime.getELapsedMillis(startTime)) ;
}finally {
loadData.fetcher.cleanup();
}
sourcecacheGenerator =new DataCacheGenerator(collections.singLetonList(loadData.sourceKey)， helper，cb: this);
}
You can see that for the parsed data, you can use the helper getDiskCache(). Put() method, which is stored in the hard disk cache of DiskLruCache. And through loaddata fetcher. Clearup () clears the task and assigns sourcecacheGenerator to DataCacheGenerator.
Go back to the startNext method of the SourceGenerator
@override
public boolean startNext() {
if (dataTocache != null){
object data = dataTocache;dataTocache = null;
cacheData(data);
}
if (sourceCacheGenerator != null && sourcecacheGenerator.startNext()){
return true;
}
sourcecacheGenerator = null;
loadData = null:
}
At this time, sourceCacheGenerator is not null, so the startNext() method of DataCacheGenerator will be used.
Before that, the pictures obtained from the network request have been loaded into DiskLruCache, so the requested picture resources have been obtained. Then, after a series of calls, it will call the onResourceReady method of EngineJob. This method, as we said in the memory cache, will store the resources in the weak reference and load the pictures.
Presumably, through the above pile of code, the understanding of hard disk cache will still be blurred, so let's summarize it below.
Hard disk cache summary
To summarize the steps of hard disk caching:
- 1: Select different cache strategies according to the hard disk strategy. The following steps are based on the hard disk policy as diskcachestrategy The original picture is also cached after all decoding.
- 2: First call the run method of DecodeJob, and then call the runWrapped method to generate different decoders according to the runReason flag.
- 3: First obtain the ResourceCaheGenerator to check whether there are decoded picture resources in the hard disk DiskLruCache. If not, get the next decoder DataCacheGenerator and call its startNext method to check whether there are original picture resources in the hard disk DiskLruCache. If not, then call the rescgedule method again to call the run method of DncodeJob again and change the value of runReason.
- 4: According to runReason, assign the SourceGenerator to the current decoder. For the first time, dataToCache must be null, so a network request will be made to obtain image resources according to the URl, and then call back the run method of DecodeJob
- Five: also will go to this step, this time because the network has been acquired, so dataToCache is not null, that is, the image resources will be placed in the hard disk cache DiskLruCache, then assign DataCacheGenerator to SourceGenerator, then call SourceGenerator.. startNext is the startNext method of datacachegenerator.
- 6: Because the data can be obtained from the hard disk at this time, it is different from the execution process of step 3, that is, it will go through a series of steps, call back to the onResourceReady method to display the picture, and put the resources into the weak reference cache.
It will be better understood by combining the following figure again: