SPI mechanism of JDK, Dubbo and spring
The full name of SPI is Service Provider Interface, which is a service discovery mechanism. SPI
The essence of is to configure the fully qualified name of the interface implementation class in the file, and the service loader reads the configuration file and loads the implementation class. In this way, you can dynamically replace the implementation class for the interface at run time. Because of this feature, we can easily pass
SPI mechanism provides extended functions for our program.
Abstract from https://segmentfault.com/a/1190000039812642
What problems can SPI solve?
I hava a colorconfig, how can I get different colors according to different needs of different people?
public interface ColourConfig {
String myColour();
}
public class BlueConfig implements ColourConfig{
@Override
public String myColour() {
return "blue";
}
}
public class RedConfig implements ColourConfig {
@Override
public String myColour() {
return "red";
}
}
Ordinary programmers say: I care how you use it, and I'll open an interface for you
public class ColourFactory {
ColourConfig redColor = new RedConfig();
ColourConfig blueColor = new BlueConfig();
public String getBlueColor() {
return blueColor.myColour();
}
public String getRedColor() {
return redColor.myColour();
}
}
The product manager said: I expect to open a colorful color factory next to meet the needs of all users.
Ordinary programmer.
For the business itself that needs to be expanded, we can accept horizontal expansion. However, in object-oriented design, we generally recommend interface programming between modules, and the horizontal expansion in the business is not perceived by the caller.
The preliminary modification scheme is to provide only one external interface to customize colorfactory for different callers.
public class ColourFactory {
ColourConfig colorConfig= new RedConfig();
public String getColor() {
return colorConfig.myColour();
}
}
But this is still very troublesome. Can we have a scheme? I have a set of code externally, but can we assemble different configs for different callers?
Yes, based on configuration. Whether the SPI of JDK, DUBBO or Spring, the core is based on configuration. Let's see how they do it first.
Use of SPI
JDK SPI
1. Code
public class JavaColorColourFactory {
public static String getColor(){
ServiceLoader<ColourConfig> colourLoader = ServiceLoader.load(ColourConfig.class);
Iterator<ColourConfig> colourIterator = colourLoader.iterator();
ColourConfig colourConfig = null;
while (colourIterator.hasNext()){
colourConfig = colourIterator.next();
}
return colourConfig == null ? null : colourConfig.myColour();
}
}
2. Configuration:
In the META-INF\services directory, create a new file named after the full class name of colorconfig. The configured content is the full class name of the required implementation class.
3. Precautions:
3.1 for Maven project, the configuration file needs to be placed in the resources directory. Many old materials on the Internet show that they are placed in the java directory and cannot take effect~
3.2 the above implementation classes can only be scanned to normal classes. When writing documents, for convenience, write the implementation class as the internal class of the interface, and it is found that it cannot be scanned!
3.3 the configuration file can be configured with multiple implementation classes. From the above code, we can see that the SPI mechanism of JDK is to traverse the files and take out all the configured implementation classes. Therefore, in the actual use scenario, the loading order cannot be determined between the files configured by the jar package itself and the files configured by the caller! Therefore, when using the spi of JDK, the way to ensure that the specified configuration is loaded is to ensure that only one configuration file will be introduced and no default configuration will be specified. Database driver is used in this way, Java sql. Driver just defines a specification and does not have any default implementation, Java sql. All configurations will be loaded in drivermanager.
The code is as follows: it is also mentioned in the comments that Get all the drivers through the classloader. Therefore, the SPI mechanism of JDK is more suitable for loading all configurations than the specified configuration! (if there is only one configuration, of course, only one will be loaded)
private static void loadInitialDrivers() {
String drivers;
try {
drivers = AccessController.doPrivileged(new PrivilegedAction<String>() {
public String run() {
return System.getProperty("jdbc.drivers");
}
});
} catch (Exception ex) {
drivers = null;
}
// If the driver is packaged as a Service Provider, load it.
//Pay attention to the following line of comments and load all drivers!
// Get all the drivers through the classloader
// exposed as a java.sql.Driver.class service.
// ServiceLoader.load() replaces the sun.misc.Providers()
AccessController.doPrivileged(new PrivilegedAction<Void>() {
public Void run() {
ServiceLoader<Driver> loadedDrivers = ServiceLoader.load(Driver.class);
Iterator<Driver> driversIterator = loadedDrivers.iterator();
try{
while(driversIterator.hasNext()) {
driversIterator.next();
}
} catch(Throwable t) {
// Do nothing
}
return null;
}
});
println("DriverManager.initialize: jdbc.drivers = " + drivers);
if (drivers == null || drivers.equals("")) {
return;
}
String[] driversList = drivers.split(":");
println("number of Drivers:" + driversList.length);
for (String aDriver : driversList) {
try {
println("DriverManager.Initialize: loading " + aDriver);
Class.forName(aDriver, true,
ClassLoader.getSystemClassLoader());
} catch (Exception ex) {
println("DriverManager.Initialize: load failed: " + ex);
}
}
}
DUBBO SPI
1. Common usage
@SPI("blue")
public interface ColourConfig {
String myColour();
}
public class BlueConfig implements ColourConfig {
@Override
public String myColour() {
return "blue";
}
}
public class RedConfig implements ColourConfig {
@Override
public String myColour() {
return "red";
}
}
use:
public class DubboColourFactory {
@Test
public void testColor(){
String color = getColor();
System.out.println(color);
}
public String getColor(){
//red
String red = ExtensionLoader.getExtensionLoader(ColourConfig.class).getExtension("red").myColour();
//The default value specified by SPI in the interface is blue
String defaultColour = ExtensionLoader.getExtensionLoader(ColourConfig.class).getDefaultExtension().myColour();
return red;
}
}
2. Configuration
META-INF/dubbo/internal/ META-INF/dubbo/ META-INF/services/
The above three folders are OK. dubbo will be loaded in order. It should be noted that the same name will be skipped! That is, if both META-INF/dubbo / and META-INF/services / have the same configuration, the one in META-INF/dubbo / will finally take effect.
The file name is the full class name of the interface, and the content is the key = full class name of all implementation classes
For example:
red=cn.com.test.dubbo.RedConfig
3. Advanced version (configuration file unchanged)
@SPI("blue")
public interface ColourConfig {
//The method modified by Adaptive annotation must use the URL as the transmission parameter. When the method modified by Adaptive annotation, the Dubbo framework will generate the default adapter.
//This annotation can also modify classes. When modifying classes, you need to customize the Adaptive adapter
@Adaptive
String myColour(URL url);
}
public class BlueConfig implements ColourConfig {
@Override
public String myColour(URL url) {
return "blue";
}
}
public class RedConfig implements ColourConfig {
@Override
public String myColour(URL url) {
return "red";
}
}
use
public class DubboColourFactory {
@Test
public void testColor(){
String color = getColor();
System.out.println(color);
}
public String getColor(){
//This map is used for routing. The key is the value specified by Adaptive. If it is not specified, it defaults to the modified class name, and the hump is changed to point connection
Map<String,String> paramMap = new HashMap<>();
paramMap.put("colour.config","red");
URL red = new URL("", null, 0,paramMap);
String adaptiveColour = ExtensionLoader.getExtensionLoader(ColourConfig.class).getAdaptiveExtension().myColour(red);
return adaptiveColour;
}
}
4. Ultimate Advanced Edition
@SPI("blue")
public interface ColourConfig {
String myColour();
}
@Adaptive
public class AdaptiveColourConfig implements ColourConfig{
private static volatile String DEFAULT_COLOUR;
public static void setDefaultColour(String colour) {
DEFAULT_COLOUR = colour;
}
@Override
public String myColour() {
if (StringUtils.isBlank(DEFAULT_COLOUR)){
return "blank";
}
return ExtensionLoader.getExtensionLoader(ColourConfig.class).getExtension(DEFAULT_COLOUR).myColour();
}
}
public class BlueConfig implements ColourConfig {
@Override
public String myColour() {
return "blue";
}
}
public class RedConfig implements ColourConfig {
@Override
public String myColour() {
return "red";
}
}
public class DubboColourFactory {
@Test
public void testColor(){
String color = getColor();
System.out.println(color);
}
public String getColor(){
AdaptiveColourConfig.setDefaultColour("red");
String adaptiveColour = ExtensionLoader.getExtensionLoader(ColourConfig.class).getAdaptiveExtension().myColour();
return adaptiveColour;
}
}
Configuration file needs to be added:
//This key can be written casually and will take effect. Be careful not to be the same as other keys
adaptive=cn.com.test.dubbo.AdaptiveColourConfig
SPRING SPI
1. Code: I won't post the code related to colorconfig, just a simple interface + implementation class.
public class SpringColourFactory extends BaseTest {
@Test
public void test(){
List<ColourConfig> colourConfigs = SpringFactoriesLoader.loadFactories(ColourConfig.class, this.getClass().getClassLoader());
colourConfigs.stream().forEach(colourConfig -> System.out.println(colourConfig.myColour()));
}
}
2. Configuration file
META-INF Under folder: File name: spring.factories Document content:(\Represents a line break or can be deleted\，Only one line) cn.com.test.spring.ColourConfig=\ cn.com.test.spring.BlueConfig,\ cn.com.test.spring.RedConfig
Comparison of SPI implementations of JDK, Dubbo and spring
| Use difficulty & learning difficulty | Multiple profiles | configuration file | Applicable scenario | |
|---|---|---|---|---|
| JDK | simple | load all | One interface, one | |
| SPRING | simple | load all | There is only one profile | |
| DUBBO | secondary | Decide who to load according to the alias, and the same name will be deleted | One interface, one |
Principle of SPI
Start looking at the source code ~ some of the following are omitted, and only the key steps (steps I can understand)
JDK SPI
1.ServiceLoader colourLoader = ServiceLoader.load(ColourConfig.class);
To put it simply, we created a ServiceLoader and lazyitterer.
public static <S> ServiceLoader<S> load(Class<S> service) {
//Note that the ClassLoader here is AppClasLoader. We usually use getClassLoader() to get the class loader. Why not here? Look at 3
ClassLoader cl = Thread.currentThread().getContextClassLoader();
return ServiceLoader.load(service, cl);
}
public static <S> ServiceLoader<S> load(Class<S> service,ClassLoader loader)
{
return new ServiceLoader<>(service, loader);
}
private ServiceLoader(Class<S> svc, ClassLoader cl) {
service = Objects.requireNonNull(svc, "Service interface cannot be null");
//Save class loader
loader = (cl == null) ? ClassLoader.getSystemClassLoader() : cl;
reload();
}
public void reload() {
providers.clear();
//An iterator is created here
lookupIterator = new LazyIterator(service, loader);
}
2.iterator
In fact, the following logic is very simple: read the configuration file, and then instantiate the object according to the configured full class name,
public boolean hasNext() {
return hasNextService();
}
private boolean hasNextService() {
//Spell out the classpath
String fullName = PREFIX + service.getName();
configs = loader.getResources(fullName);
//This is the byte stream to read the file
pending = parse(service, configs.nextElement());
//Read the full class name of the implementation class
nextName = pending.next();
return true;
}
public S next() {
return nextService();
}
private S nextService() {
String cn = nextName;
nextName = null;
Class<?> c = null;
c = Class.forName(cn, false, loader);
S p = service.cast(c.newInstance());
providers.put(cn, p);
return p;
}
3. Small knowledge
Class loading is familiar to everyone, so I won't repeat it here. As we all know, the two parent delegation mode, that is, Bootstrap ClassLoader, Extension ClassLoader and App ClassLoader are loaded in order, and the child ClassLoader entrusts the parent ClassLoader to load (of course, the relationship between child and parent is not so clear, just for convenience).
Obviously, ServiceLoader is loaded by App ClassLoader.
Assuming that the above javacolorcolorcolorfactory and ColorConfig are loaded by the user-defined class loader instead of AppClassLoader, based on the parental delegation mode, ServiceLoader obviously cannot be loaded into the classes loaded by the user-defined class loader. So there is classLoader CL = thread currentThread(). getContextClassLoader(); The classLoader is passed in the thread context, which breaks the limitation of the parental mode.
(whisper BB: I'm going to make a DEMO, but I didn't make it. I'll make it up later)
DUBBO SPI
1.ExtensionLoader.getExtensionLoader(ColourConfig.class)
It is very simple to obtain the ExtensionLoader. Here, the lazy loading mode is adopted. It is created only when it is found that there is no one in the call, and it is put into the cache
public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type) {
ExtensionLoader<T> loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
if (loader == null) {
EXTENSION_LOADERS.putIfAbsent(type, new ExtensionLoader<T>(type));
loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
}
return loader;
}
2.ExtensionLoader#getExtension
public T getExtension(String name) {
if ("true".equals(name)) {
return getDefaultExtension();
}
Holder<Object> holder = cachedInstances.get(name);
...
Object instance = holder.get();
...
instance = createExtension(name);
...
return (T) instance;
}
private T createExtension(String name) {
//This is the process of loading the configuration file,
Class<?> clazz = getExtensionClasses().get(name);
try {
T instance = (T) EXTENSION_INSTANCES.get(clazz);
...
//Process of instantiating extension points
injectExtension(instance);
//The following text is not the key point of the article
//Take DubboProtocol as an example. We need to monitor and filter it. The implementation mechanism of Dubbo is based on the packaging class
//Dubbo will load the Wrapper class into cachedWrapperClasses. Here is a Set, so the final call is a chain call that does not guarantee the order The alias of Wrapper is useless (maybe I didn't find it useful), just a simple logo
//dubbo=com.alibaba.dubbo.rpc.protocol.dubbo.DubboProtocol
//filter=com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper
//listener=com.alibaba.dubbo.rpc.protocol.ProtocolListenerWrapper
Set<Class<?>> wrapperClasses = cachedWrapperClasses;
if (wrapperClasses != null && wrapperClasses.size() > 0) {
for (Class<?> wrapperClass : wrapperClasses) {
instance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance));
}
}
return instance;
} catch (Throwable t) {
...
}
}
private Map<String, Class<?>> getExtensionClasses() {
cachedClasses.set( loadExtensionClasses());
}
private Map<String, Class<?>> loadExtensionClasses() {
final SPI defaultAnnotation = type.getAnnotation(SPI.class);
...
Map<String, Class<?>> extensionClasses = new HashMap<String, Class<?>>();
//The following is how to load the configuration files under the three directories in sequence, which is lazy loading, and skip the key conflict, so the conflict shall prevail
loadFile(extensionClasses, DUBBO_INTERNAL_DIRECTORY);
loadFile(extensionClasses, DUBBO_DIRECTORY);
loadFile(extensionClasses, SERVICES_DIRECTORY);
return extensionClasses;
}
private void loadFile(Map<String, Class<?>> extensionClasses, String dir) {
//The IO stream is omitted
//Cacheddadaptive class, which corresponds to the scene of Adaptive annotation on the class
if (clazz.isAnnotationPresent(Adaptive.class)) {
if(cachedAdaptiveClass == null) {
cachedAdaptiveClass = clazz;
}
}else{
try {
//Try to get the constructor of the current class. If you can get the constructor with the current type parameter, it is considered to be a wrapper class. Otherwise, enter Catch. So Catch makes sense here
clazz.getConstructor(type);
Set<Class<?>> wrappers = cachedWrapperClasses;
if (wrappers == null) {
cachedWrapperClasses = new ConcurrentHashSet<Class<?>>();
wrappers = cachedWrapperClasses;
}
wrappers.add(clazz);
} catch (NoSuchMethodException e) {
clazz.getConstructor();
String[] names = NAME_SEPARATOR.split(name);
if (names != null && names.length > 0) {
//This is a more complex extension mechanism, which supports grouping, sorting, etc. I won't elaborate. I'll go further when I have time
Activate activate = clazz.getAnnotation(Activate.class);
...
for (String n : names) {
//Put the current name and class into the cache
//Extension classes will eventually be placed in cached classes
//cachedClasses and cachedNames save two different pieces of data with name and Class as key s, respectively, for faster query
if (! cachedNames.containsKey(clazz)) {
cachedNames.put(clazz, n);
}
Class<?> c = extensionClasses.get(n);
if (c == null) {
extensionClasses.put(n, clazz);
}
}
}
}
}
}
private T injectExtension(T instance) {
//This is reflection, obtaining the attribute of instance, and then injecting.
}
3.ExtensionLoader#getAdaptiveExtension
public T getAdaptiveExtension() {
instance = createAdaptiveExtension();
}
private T createAdaptiveExtension() {
//1. Get adaptiveExtensionClass
//2. Instantiate and initialize
return injectExtension((T) getAdaptiveExtensionClass().newInstance());
}
private Class<?> getAdaptiveExtensionClass() {
//As mentioned earlier,
//1. load all class es in the configuration file
//2. Initialize the three caches: cacheddadaptive class, cachedClasses and cachedWrapperClasses
//It should be noted that the cacheddadaptiveclass used here is a class with Adaptive annotation on the class. The annotation marked on the above-mentioned method cannot be scanned here when it comes in for the first time
getExtensionClasses();
if (cachedAdaptiveClass != null) {
return cachedAdaptiveClass;
}
//The method annotation is first parsed here
return cachedAdaptiveClass = createAdaptiveExtensionClass();
}
private Class<?> createAdaptiveExtensionClass() {
//Here is to create the default adaptive extension. The specific method is string splicing. Don't look carefully, write down the text flow
//1. Poll the methods of the current class. If no method has Adaptive annotation, an error will be reported
//2. String splicing package name, introduce the full Class name of ExtensionLoader, splice a Class with $Adaptive suffix, polling method, and splice the methods with Adaptive annotation into code in the form of string splicing.
//3. It should be noted that the interface modified by Adaptive annotation must have URL type parameters, or the parameters contain URL objects, otherwise an error will be reported. The value of Adaptive annotation is the key used for the final route. Official annotation: (if the key is not set, the point separation of "extension point interface name" is used as the key)
String code = createAdaptiveExtensionClassCode();
ClassLoader classLoader = findClassLoader();
//I haven't seen this either...
com.alibaba.dubbo.common.compiler.Compiler compiler = ExtensionLoader.getExtensionLoader(com.alibaba.dubbo.common.compiler.Compiler.class).getAdaptiveExtension();
return compiler.compile(code, classLoader);
}
SPRING SPI
Looking at...