Dubbo 3 concept and architecture
1. Service discovery
Compared with Dubbo 2, Dubbo 3 changes from interface level service registration discovery to application level. These two methods are not compatible with each other.
The benefit of this transformation is that it greatly releases the storage pressure of the service registry. For example, if the original service provides 100 dubbo interfaces and deploys 100 machines, the registry needs to store 10000 pieces of data, while it only needs 100 at the application level.
Therefore, for Dubbo 3, the biggest advantage of application level service registration and discovery is to save memory and reduce the pressure of address push.
For example, suppose we have 2000 instances and 50 applications (500 dubbo interfaces, with an average of 10 for each application)
Assume that the average size of each interface level URL is 5KB, and the average size of each application level URL is 0.5kb
Interface level address quantity: 2000 * 500 * 5kb ≈ 4.8G
Application level address quantity: 2000 * 50 * 0.5kb ≈ 48M
The savings are considerable.
At the same time, the official also gave 2.5% of the double address registration discovery X smooth migration to 3 X's Guide Application level address discovery migration guide
Here, mysterious and mysterious, it is said that it is to "adapt to the change of cloud native microservice". Another term "cloud native" has been introduced. I'm afraid I'm ignorant of it. This word is really pure and disgusting. I don't know what the word means. How tall the name is, how difficult it is to understand. You can refer to this article What is cloud primordial? This time someone finally made it clear
Here are some vague questions that I want to talk about. I think they are the key points that hinder me from understanding dubbo design:
Several questions
Service registration and discovery. What does the provider register? What did consumer find? How to register and discover?
What information does the consumer need to know when calling to locate the interface? How to locate? How to act as agent? What is the difference between dubbo2 and dubbo3?
This may involve an essential problem, that is, how to build RPC services and what is the design architecture of dubbo?
In fact, my biggest feeling in learning the framework is that it is complex, large and complete. There are too many documents. I don't know what to look at and read everything, but I don't seem to read anything. I'm not interested in looking at it. I'm completely relying on my willpower to look and learn. It's inefficient and against one's will.
So I wonder if there is a better way to understand and learn knowledge and principles? I thought of the word "knowledge". Learning is inseparable from asking. In fact, when I study again, I very much hope to have a teacher who can ask me a question and answer a question. In this way, after my question is asked and answered, the teacher will lead to new questions from the answers until the end. I think even if I master this knowledge, it's like I discuss technical problems with my colleagues and friends. It's just that self-study is to discuss with myself, with browsers and documents.
Therefore, I want to reorganize this article with a new situation.
Dubbo studies and questions
Q1: what is the dubbo framework?
dubbo is a microservice development framework with two key capabilities: RPC communication, Remote Procedure Call and microservice governance
Q2: what is RPC communication? What is micro service governance?
The full name of RPC is Remote Procedure Call. RPC communication allows remote services to be called like local services. A simple understanding is that I can call the methods written by other services, just like the methods written in my own class. I think the advantage is convenient and less cumbersome than HTTP.
Microservice governance should be a relatively broad term. From the official perspective, it refers to service discovery, load balancing, traffic scheduling, etc.
Q3: composition and main process of dubbo framework
In fact, the above official figure also illustrates the general process. There are three indispensable roles: provider, service provider, consumer, service consumer and registry registry. These necessary roles, but also some optional extended roles, metadata center and monitoring center. (of course, if you are a single service provider, you can also omit the registration center)
The provider registers its address in the registry registry, and the consumer reads the address list provided by the subscription provider in the registry. It can be seen here that the registry is essential and plays the role of an intermediary.
Q4: what exactly did the provider register with the registry? How did you register?
For example, as a provider, I provide the following interface services:
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:dubbo="http://dubbo.apache.org/schema/dubbo"
xmlns="http://www.springframework.org/schema/beans" xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://dubbo.apache.org/schema/dubbo http://dubbo.apache.org/schema/dubbo/dubbo.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
<context:property-placeholder/>
<dubbo:application name="demo-provider"/>
<dubbo:registry address="zookeeper://127.0.0.1:2181"/>
<dubbo:provider token="true"/>
<bean id="demoService" class="com.dyinggq.dubbo3simpledemo.basic.impl.DemoServiceImpl"/>
<dubbo:service interface="com.dyinggq.dubbo3simpledemo.basic.api.DemoService" ref="demoService"/>
</beans>
Start the local localhost instance with Zookeeper as the registration center
The following addresses will be stored in the interface level address registration discovery:
dubbo://172.20.10.2:20880/dubbo.samples.basic.api.DemoService?anyhost=true&application=demo-provider&background=false&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=dubbo.samples.basic.api.DemoService&metadata-type=remote&methods=testVoid,sayHello&pid=19348&release=3.0.5&service-name-mapping=true&side=provider×tamp=1644139208312&token=cf1942fb-267e-407c-98ef-cb1b06d853b8
When the service provider starts: to / Dubbo / Dubbo samples. basic. api. Write your own URL address in the demoservice / providers directory
Here, dubbo will think about the files added in zk
Fully qualified name file of the interface
Here is com dyinggq. dubbo3simpledemo. basic. api. Demoservice and the provider package under it, and the interface details and ip address when stored under it
mapping file
Interface full path and application name, here is com dyinggq. dubbo3simpledemo. basic. api. Demoservice and demo provider
Metadata metadata
It is divided into two sub directories: interface path and metadata service.
There are several interfaces, that is, there are several corresponding directories. Here we take com dyinggq. dubbo3simpledemo. basic. api. Demoservice as an example
The interface directory is divided into consumer and provider. The corresponding node name is the application name at the time of registration
The demo provider in the provider directory stores:
{
"parameters": {
"side": "provider",
"interface": "com.dyinggq.dubbo3simpledemo.basic.api.DemoService",
"metadata-type": "remote",
"application": "demo-provider",
"dubbo": "2.0.2",
"release": "3.0.5",
"anyhost": "true",
"methods": "testVoid,sayHello",
"background": "false",
"deprecated": "false",
"dynamic": "true",
"service-name-mapping": "true",
"generic": "false",
"token": "9451d1a3-7178-49c4-a271-63707b907c5f"
},
"canonicalName": "com.dyinggq.dubbo3simpledemo.basic.api.DemoService",
"codeSource": "file:/Users/admin/Documents/GitHub/dubbo3-simple-demo/target/classes/",
"methods": [
{
"name": "sayHello",
"parameterTypes": [
"java.lang.String"
],
"returnType": "java.lang.String",
"annotations": []
},
{
"name": "testVoid",
"parameterTypes": [],
"returnType": "void",
"annotations": []
}
],
"types": [
{
"type": "void"
},
{
"type": "java.lang.String"
}
],
"annotations": []
}
Interface information, methods, versions, and related configurations.
Corresponding demo consumer storage
{
"side": "consumer",
"interface": "com.dyinggq.dubbo3simpledemo.basic.api.DemoService",
"metadata-type": "remote",
"application": "demo-consumer",
"dubbo": "2.0.2",
"release": "3.0.5",
"sticky": "false",
"check": "true",
"methods": "testVoid,sayHello",
"background": "false"
}
For MetadataService, the corresponding meta information is registered for each group, and the data is consistent except for different groups
{
"parameters": {
"version": "1.0.0",
"side": "provider",
"interface": "org.apache.dubbo.metadata.MetadataService",
"group": "demo-consumer",
"metadata-type": "remote",
"application": "demo-consumer",
"dubbo": "2.0.2",
"release": "3.0.5",
"anyhost": "true",
"methods": "getExportedURLs,getAndListenInstanceMetadata,toURLs,serviceName,isMetadataServiceURL,getSubscribedURLs,version,getExportedServiceURLs,exportInstanceMetadata,getMetadataInfo,toSortedStrings,getMetadataInfos,getServiceDefinition,getInstanceMetadataChangedListenerMap",
"deprecated": "false",
"getAndListenInstanceMetadata.sent": "true",
"generic": "false",
"getAndListenInstanceMetadata.1.callback": "true",
"revision": "3.0.5",
"delay": "0",
"getAndListenInstanceMetadata.return": "true",
"background": "false",
"dynamic": "true",
"executes": "100",
"connections": "1"
},
"canonicalName": "org.apache.dubbo.metadata.MetadataService",
"codeSource": "dubbo-3.0.5.jar",
"methods": [
{
"name": "serviceName",
"parameterTypes": [],
"returnType": "java.lang.String",
"annotations": []
},
{
"name": "getSubscribedURLs",
"parameterTypes": [],
"returnType": "java.util.SortedSet\u003cjava.lang.String\u003e",
"annotations": []
},
{
"name": "getExportedURLs",
"parameterTypes": [
"java.lang.String",
"java.lang.String",
"java.lang.String",
"java.lang.String"
],
"returnType": "java.util.SortedSet\u003cjava.lang.String\u003e",
"annotations": []
},
{
"name": "getExportedURLs",
"parameterTypes": [
"java.lang.String",
"java.lang.String",
"java.lang.String"
],
"returnType": "java.util.SortedSet\u003cjava.lang.String\u003e",
"annotations": []
},
{
"name": "getExportedURLs",
"parameterTypes": [
"java.lang.String",
"java.lang.String"
],
"returnType": "java.util.SortedSet\u003cjava.lang.String\u003e",
"annotations": []
},
{
"name": "getExportedURLs",
"parameterTypes": [
"java.lang.String"
],
"returnType": "java.util.SortedSet\u003cjava.lang.String\u003e",
"annotations": []
},
{
"name": "getExportedURLs",
"parameterTypes": [],
"returnType": "java.util.SortedSet\u003cjava.lang.String\u003e",
"annotations": []
},
{
"name": "getExportedServiceURLs",
"parameterTypes": [],
"returnType": "java.util.Set\u003corg.apache.dubbo.common.URL\u003e",
"annotations": []
},
{
"name": "getServiceDefinition",
"parameterTypes": [
"java.lang.String"
],
"returnType": "java.lang.String",
"annotations": []
},
{
"name": "getServiceDefinition",
"parameterTypes": [
"java.lang.String",
"java.lang.String",
"java.lang.String"
],
"returnType": "java.lang.String",
"annotations": []
},
{
"name": "getMetadataInfo",
"parameterTypes": [
"java.lang.String"
],
"returnType": "org.apache.dubbo.metadata.MetadataInfo",
"annotations": []
},
{
"name": "getMetadataInfos",
"parameterTypes": [],
"returnType": "java.util.Map\u003cjava.lang.String,org.apache.dubbo.metadata.MetadataInfo\u003e",
"annotations": []
},
{
"name": "getInstanceMetadataChangedListenerMap",
"parameterTypes": [],
"returnType": "java.util.Map\u003cjava.lang.String,org.apache.dubbo.metadata.InstanceMetadataChangedListener\u003e",
"annotations": []
},
{
"name": "getAndListenInstanceMetadata",
"parameterTypes": [
"java.lang.String",
"org.apache.dubbo.metadata.InstanceMetadataChangedListener"
],
"returnType": "java.lang.String",
"annotations": []
},
{
"name": "exportInstanceMetadata",
"parameterTypes": [
"java.lang.String"
],
"returnType": "void",
"annotations": []
},
{
"name": "version",
"parameterTypes": [],
"returnType": "java.lang.String",
"annotations": []
}
],
"types": [
{
"type": "void"
},
{
"type": "org.apache.dubbo.metadata.MetadataInfo",
"properties": {
"app": "java.lang.String",
"services": "java.util.Map\u003cjava.lang.String,org.apache.dubbo.metadata.MetadataInfo.ServiceInfo\u003e",
"revision": "java.lang.String"
}
},
{
"type": "long[]",
"items": [
"long"
]
},
{
"type": "java.util.Map\u003cjava.lang.String,java.util.Map\u003cjava.lang.String,java.lang.String\u003e\u003e",
"items": [
"java.lang.String",
"java.util.Map\u003cjava.lang.String,java.lang.String\u003e"
]
},
{
"type": "org.apache.dubbo.metadata.InstanceMetadataChangedListener"
},
{
"type": "java.util.Map\u003cjava.lang.String,java.lang.Object\u003e",
"items": [
"java.lang.String",
"java.lang.Object"
]
},
{
"type": "org.apache.dubbo.metadata.MetadataInfo.ServiceInfo",
"properties": {
"path": "java.lang.String",
"protocol": "java.lang.String",
"name": "java.lang.String",
"params": "java.util.Map\u003cjava.lang.String,java.lang.String\u003e",
"version": "java.lang.String",
"group": "java.lang.String"
}
},
{
"type": "java.util.Map\u003cjava.lang.String,org.apache.dubbo.metadata.InstanceMetadataChangedListener\u003e",
"items": [
"java.lang.String",
"org.apache.dubbo.metadata.InstanceMetadataChangedListener"
]
},
{
"type": "java.lang.Integer"
},
{
"type": "int"
},
{
"type": "long"
},
{
"type": "java.util.SortedSet\u003cjava.lang.String\u003e",
"items": [
"java.lang.String"
]
},
{
"type": "java.util.Set\u003corg.apache.dubbo.common.URL\u003e",
"items": [
"org.apache.dubbo.common.URL"
]
},
{
"type": "java.util.Map\u003cjava.lang.String,org.apache.dubbo.metadata.MetadataInfo\u003e",
"items": [
"java.lang.String",
"org.apache.dubbo.metadata.MetadataInfo"
]
},
{
"type": "org.apache.dubbo.common.url.component.URLParam",
"properties": {
"hashCodeCache": "int",
"DEFAULT_KEY": "java.util.BitSet",
"enableCompressed": "boolean",
"VALUE": "java.lang.Integer[]",
"EXTRA_PARAMS": "java.util.Map\u003cjava.lang.String,java.lang.String\u003e",
"rawParam": "java.lang.String",
"KEY": "java.util.BitSet",
"METHOD_PARAMETERS": "java.util.Map\u003cjava.lang.String,java.util.Map\u003cjava.lang.String,java.lang.String\u003e\u003e"
}
},
{
"type": "org.apache.dubbo.common.URL",
"properties": {
"hashCodeCache": "int",
"attributes": "java.util.Map\u003cjava.lang.String,java.lang.Object\u003e",
"urlAddress": "org.apache.dubbo.common.url.component.URLAddress",
"urlParam": "org.apache.dubbo.common.url.component.URLParam"
}
},
{
"type": "java.util.Map\u003cjava.lang.String,org.apache.dubbo.metadata.MetadataInfo.ServiceInfo\u003e",
"items": [
"java.lang.String",
"org.apache.dubbo.metadata.MetadataInfo.ServiceInfo"
]
},
{
"type": "java.lang.Integer[]",
"items": [
"java.lang.Integer"
]
},
{
"type": "boolean"
},
{
"type": "org.apache.dubbo.common.url.component.URLAddress",
"properties": {
"port": "int",
"host": "java.lang.String"
}
},
{
"type": "java.lang.Object"
},
{
"type": "java.lang.String"
},
{
"type": "java.util.BitSet",
"properties": {
"words": "long[]"
}
},
{
"type": "java.util.Map\u003cjava.lang.String,java.lang.String\u003e",
"items": [
"java.lang.String",
"java.lang.String"
]
}
],
"annotations": []
}
It can be seen that interface level address discovery stores a large amount of interface metadata information
Q5: what is the difference between interface level address discovery and application level address discovery?
In the above words: compared with interface level address discovery, application level address discovery has less stored data, faster push speed and less pressure.
How to say, application level address discovery is not a new thing, but dubbo chose to embrace the improvements made by the mainstream. For example, the SpringCloud system and K8S system are service registration and discovery based on the dimension of application. Only by aligning with these systems can we better connect with them.
How to enable application level address discovery? Here's a chestnut
The parameter "registry: Address" is added=“ zookeeper://127.0.0.1:2181 " register-mode="instance"/>
The optional values are interface level, instance application level and all. The default value is all, that is, both interface level address and application level address are registered
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:dubbo="http://dubbo.apache.org/schema/dubbo"
xmlns="http://www.springframework.org/schema/beans" xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://dubbo.apache.org/schema/dubbo http://dubbo.apache.org/schema/dubbo/dubbo.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
<context:property-placeholder/>
<dubbo:application name="demo-provider-2" version="1.0.1" register-mode="instance" />
<dubbo:registry address="zookeeper://127.0.0.1:2181" check="false" version="1.0.1" />
<dubbo:provider token="true"/>
<bean id="demoService" class="com.dyinggq.dubbo3simpledemo.basic.impl.DemoServiceImpl" />
<bean id="demoService2" class="com.dyinggq.dubbo3simpledemo.basic.impl.DemoService2Impl"/>
<dubbo:service interface="com.dyinggq.dubbo3simpledemo.basic.api.DemoService" ref="demoService"/>
<dubbo:service interface="com.dyinggq.dubbo3simpledemo.basic.api.DemoService2" ref="demoService2"/>
</beans>
At this time, the consumer needs to specify services to specify the application corresponding to the interface, that is: services="demo-provider-2"
Or provide a metadata Center for service introspection (but it will undoubtedly increase the complexity of the service)
The service introspection name is awesome. In fact, it is to let dubbo match itself. When the provider registers, the mapping relationship between the interface and the application name is stored. When consumers consume, they get the deployed application name according to the interface name, and then do service discovery. This needs to be supported by a metadata center
<beans xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:dubbo="http://dubbo.apache.org/schema/dubbo"
xmlns="http://www.springframework.org/schema/beans" xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://dubbo.apache.org/schema/dubbo http://dubbo.apache.org/schema/dubbo/dubbo.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd">
<context:property-placeholder/>
<dubbo:application name="demo-consumer"/>
<dubbo:registry address="zookeeper://127.0.0.1:2181"/>
<dubbo:reference id="demoService" services="demo-provider-2" check="true" interface="com.dyinggq.dubbo3simpledemo.basic.api.DemoService"/>
</beans>
Take a look at the registry data
Compare the application level address discovery in the figure below
It can be found that the url interface data is saved, i.e
dubbo://172.20.10.2:20880/dubbo.samples.basic.api.DemoService?anyhost=true&application=demo-provider&background=false&deprecated=false&dubbo=2.0.2&dynamic=true&generic=false&interface=dubbo.samples.basic.api.DemoService&metadata-type=remote&methods=testVoid,sayHello&pid=19348&release=3.0.5&service-name-mapping=true&side=provider×tamp=1644139208312&token=cf1942fb-267e-407c-98ef-cb1b06d853b8
This data is not a small expense for giant applications.
Because dubbo used to be interface level address discovery, and service level address discovery is not compatible with it, it is difficult to make a smooth transition for users. Here, dubbo gives a solution: Application level address discovery migration guide