In order to provide response efficiency, Eureka Server provides a two-tier cache structure, which directly stores the registration information required by Eureka Client in the cache structure. The implementation principle is shown in the figure below.
Layer 1 cache: readOnlyCacheMap, which is essentially a concurrent HashMap, synchronizes data from readWriteCacheMap depending on timing. The default time is 30 seconds.
readOnlyCacheMap: it is a read-only cache of CurrentHashMap, which is mainly used by the client when obtaining registration information. Its cache update depends on the update of timer. By comparing with the value of readWriteCacheMap, if the data is inconsistent, the data of readWriteCacheMap shall prevail.
Layer 2 Cache: readWriteCacheMap, which is essentially Guava cache.
readWriteCacheMap: the data of readWriteCacheMap is mainly synchronized with the storage layer. When obtaining the cache, judge whether there is no data in the cache. If there is no data, load it through the load method of CacheLoader. After loading successfully, put the data into the cache and return the data at the same time.
The expiration time of readWriteCacheMap cache is 180 seconds by default. When the service goes offline, expires, registers, and changes status, the data in this cache will be cleared.
When Eureka Client obtains full or incremental data, it will first obtain it from the L1 cache; If it does not exist in the L1 cache, it is obtained from the L2 cache; If the L2 cache does not exist, the data of the storage layer is synchronized to the cache and then obtained from the cache.
Through the two-layer cache mechanism of Eureka Server, the response time of Eureka Server can be effectively improved. Through the data cutting of data storage layer and cache layer, different data support can be provided according to the use scenario.
Significance of multi-level cache
Why design multi-level cache here? The reason is very simple. When there is a large-scale service registration and update, if only one ConcurrentHashMap data is modified, it is bound to lead to competition and performance impact due to the existence of locks.
Eureka is an AP model, which only needs to meet the final availability. Therefore, it uses multi-level cache to realize read-write separation. When the registration method writes, it directly writes to the memory registry. After writing the table, it actively invalidates the read-write cache.
The interface for obtaining registration information first fetches from the read-only cache. The read-only cache does not fetch from the read-write cache, and the read-write cache does not fetch from the memory Registry (not just fetching, it is more complex here). In addition, the read-write cache will update and write back to the read-only cache
- responseCacheUpdateIntervalMs: the timer interval of readonlycahemap cache update. The default is 30 seconds
- Responsecacheautoexpirationinseconds: readwritecachemap cache expiration time. The default is 180 seconds.
Cache initialization
readWriteCacheMap uses the LoadingCache object, which is an api provided in guava to implement memory caching. The creation method is as follows
LoadingCache<Long, String> cache = CacheBuilder.newBuilder()
//The size of the cache pool. When the cache items are close to this size, Guava begins to recycle the old cache items
.maximumSize(10000)
//If the set time object is not read / write accessed, the object will be deleted from memory (maintained irregularly in another thread)
.expireAfterAccess(10, TimeUnit.MINUTES)
//When the listener is removed, it will be triggered when the cache item is removed
.removalListener(new RemovalListener <Long, String>() {
@Override
public void onRemoval(RemovalNotification<Long, String> rn) {
//Perform logical operations
}
})
.recordStats()//Enable the statistics function of Guava Cache
.build(new CacheLoader<String, Object>() {
@Override
public Object load(String key) {
//Get objects from SQL or NoSql
}
});//The CacheLoader class implements automatic loadingAmong them, CacheLoader is used to realize the function of automatic cache loading. When readwritecachemap is triggered When using the get (key) method, the CacheLoader will be called back The load method searches the instance data in the service registration information according to the key for caching
ResponseCacheImpl(EurekaServerConfig serverConfig, ServerCodecs serverCodecs, AbstractInstanceRegistry registry) {
this.serverConfig = serverConfig;
this.serverCodecs = serverCodecs;
this.shouldUseReadOnlyResponseCache = serverConfig.shouldUseReadOnlyResponseCache();
this.registry = registry;
long responseCacheUpdateIntervalMs = serverConfig.getResponseCacheUpdateIntervalMs();
this.readWriteCacheMap =
CacheBuilder.newBuilder().initialCapacity(serverConfig.getInitialCapacityOfResponseCache())
.expireAfterWrite(serverConfig.getResponseCacheAutoExpirationInSeconds(), TimeUnit.SECONDS)
.removalListener(new RemovalListener<Key, Value>() {
@Override
public void onRemoval(RemovalNotification<Key, Value> notification) {
Key removedKey = notification.getKey();
if (removedKey.hasRegions()) {
Key cloneWithNoRegions = removedKey.cloneWithoutRegions();
regionSpecificKeys.remove(cloneWithNoRegions, removedKey);
}
}
})
.build(new CacheLoader<Key, Value>() {
@Override
public Value load(Key key) throws Exception {
if (key.hasRegions()) {
Key cloneWithNoRegions = key.cloneWithoutRegions();
regionSpecificKeys.put(cloneWithNoRegions, key);
}
Value value = generatePayload(key); //Pay attention here
return value;
}
});The cache loading is completed based on the generatePayload method. The code is as follows.
private Value generatePayload(Key key) {
Stopwatch tracer = null;
try {
String payload;
switch (key.getEntityType()) {
case Application:
boolean isRemoteRegionRequested = key.hasRegions();
if (ALL_APPS.equals(key.getName())) {
if (isRemoteRegionRequested) {
tracer = serializeAllAppsWithRemoteRegionTimer.start();
payload = getPayLoad(key, registry.getApplicationsFromMultipleRegions(key.getRegions()));
} else {
tracer = serializeAllAppsTimer.start();
payload = getPayLoad(key, registry.getApplications());
}
} else if (ALL_APPS_DELTA.equals(key.getName())) {
if (isRemoteRegionRequested) {
tracer = serializeDeltaAppsWithRemoteRegionTimer.start();
versionDeltaWithRegions.incrementAndGet();
versionDeltaWithRegionsLegacy.incrementAndGet();
payload = getPayLoad(key,
registry.getApplicationDeltasFromMultipleRegions(key.getRegions()));
} else {
tracer = serializeDeltaAppsTimer.start();
versionDelta.incrementAndGet();
versionDeltaLegacy.incrementAndGet();
payload = getPayLoad(key, registry.getApplicationDeltas());
}
} else {
tracer = serializeOneApptimer.start();
payload = getPayLoad(key, registry.getApplication(key.getName()));
}
break;
case VIP:
case SVIP:
tracer = serializeViptimer.start();
payload = getPayLoad(key, getApplicationsForVip(key, registry));
break;
default:
logger.error("Unidentified entity type: {} found in the cache key.", key.getEntityType());
payload = "";
break;
}
return new Value(payload);
} finally {
if (tracer != null) {
tracer.stop();
}
}
}This method accepts a Key type parameter and returns a Value type. The important fields in the Key include:
- KeyType, indicating payload text format, with JSON and XML Two values.
- EntityType indicates the type of cache. There are three values: application, VIP and SVIP.
- entityName indicates the name of the cache, which may be a single application name or ALL_APPS or ALL_APPS_DELTA .
Value has a payload of String type and a byte array, representing gzip compressed bytes.
Cache synchronization
In the construction and implementation of ResponseCacheImpl class, a scheduled task is initialized, and each scheduled task
ResponseCacheImpl(EurekaServerConfig serverConfig, ServerCodecs serverCodecs, AbstractInstanceRegistry registry) {
//Omit
if (shouldUseReadOnlyResponseCache) {
timer.schedule(getCacheUpdateTask(),
new Date(((System.currentTimeMillis() / responseCacheUpdateIntervalMs) * responseCacheUpdateIntervalMs)
+ responseCacheUpdateIntervalMs),
responseCacheUpdateIntervalMs);
}
}By default, different data is updated from readWriteCacheMap every 30s and synchronized to readOnlyCacheMap
private TimerTask getCacheUpdateTask() {
return new TimerTask() {
@Override
public void run() {
logger.debug("Updating the client cache from response cache");
for (Key key : readOnlyCacheMap.keySet()) { //Traversing a read-only collection
if (logger.isDebugEnabled()) {
logger.debug("Updating the client cache from response cache for key : {} {} {} {}",
key.getEntityType(), key.getName(), key.getVersion(), key.getType());
}
try {
CurrentRequestVersion.set(key.getVersion());
Value cacheValue = readWriteCacheMap.get(key);
Value currentCacheValue = readOnlyCacheMap.get(key);
if (cacheValue != currentCacheValue) { //Judge the difference information. If there is a difference, update it
readOnlyCacheMap.put(key, cacheValue);
}
} catch (Throwable th) {
logger.error("Error while updating the client cache from response cache for key {}", key.toStringCompact(), th);
} finally {
CurrentRequestVersion.remove();
}
}
}
};
}Cache invalidation
In abstractinstanceregistry In the register method, when the service information is saved, invalidecache will be called to invalidate the cache
public void register(InstanceInfo registrant, int leaseDuration, boolean isReplication) {
//....
invalidateCache(registrant.getAppName(), registrant.getVIPAddress(), registrant.getSecureVipAddress());
//....
}Finally, responsecacheimpl. Is called The invalidate method completes the cache invalidation mechanism
public void invalidate(Key... keys) {
for (Key key : keys) {
logger.debug("Invalidating the response cache key : {} {} {} {}, {}",
key.getEntityType(), key.getName(), key.getVersion(), key.getType(), key.getEurekaAccept());
readWriteCacheMap.invalidate(key);
Collection<Key> keysWithRegions = regionSpecificKeys.get(key);
if (null != keysWithRegions && !keysWithRegions.isEmpty()) {
for (Key keysWithRegion : keysWithRegions) {
logger.debug("Invalidating the response cache key : {} {} {} {} {}",
key.getEntityType(), key.getName(), key.getVersion(), key.getType(), key.getEurekaAccept());
readWriteCacheMap.invalidate(keysWithRegion);
}
}
}
}Copyright notice: unless otherwise stated, all articles on this blog adopt CC BY-NC-SA 4.0 license agreement. Reprint please indicate from Mic to take you to learn architecture!
If this article is helpful to you, please pay attention and praise. Your persistence is the driving force of my continuous creation. Welcome to WeChat public official account for more dry cargo.