Implement an RPC framework by hand

1, Introduction to RPC pre knowledge

1. What is RPC?

RPC is an abbreviation for Remote Procedure Call.

Suppose we have two servers A and B. an application deployed on server A wants to call the functions and methods of the application deployed on server B. because they are not in the same memory space, they cannot be called directly. Therefore, we need to express the semantics of the call and convey the data of the call through the network. In Java, the classes, methods and parameters to be called are serialized and transmitted to the target application through the network, which is called and executed by reflection.

2. Data exchange mode

Data exchange using middleware.

Direct data exchange.

3. Comparison of existing RPC frameworks

The current mainstream RPC framework is shown in the table below:

4. Core principles

4.1 calling process

• Server: Provider, service provider
• Client: Consumer, service consumer
• Stub: stub, service description

The flow of a function call is as follows:

• First, the client needs to tell the server about the function to be called. There is a mapping between the function and the process. When the client calls remotely, it needs to check the function, find the corresponding ID, and then execute the code of the function.
• The client needs to transfer the local parameters to the remote function. In the process of local call, it can directly press the stack. However, in the process of remote call, it is not in the same memory and cannot directly transfer the parameters of the function. Therefore, the client needs to convert the parameters into a byte stream and transfer it to the service end, and then the service end converts the byte stream into a format that can be read by itself, Is a process of serialization and deserialization.
• How to transmit the data when it is ready? The network transport layer needs to transmit the calling ID and serialized parameters to the server, and then serialize the calculated results to the client. Therefore, the TCP layer can complete the above process.

4.2 architecture

my-rpc
├── my-rpc-client    -- client
├── my-rpc-codec     -- Serializable and Deserialize 
├── my-rpc-common    -- Provide some reflection tools
├── my-rpc-protocol  -- Specified data transmission protocol
├── my-rpc-server    -- Server
├── my-rpc-transport -- be used for client And server of http communication processing 
└── my-rpc-example   -- Call example

Module dependencies are shown in the figure below. My RPC server and my RPC client have the same dependencies.

4.3 detailed introduction to functions of each functional module

(1) Protocol module

Used to specify data transmission protocols and rules;

(2) transport module

• This module is mainly used to handle the http communication between the client and the server. The client request content is encapsulated and transmitted in the form of request class, and the server response content is encapsulated and returned in the form of response class;
• Use jetty container to complete init, start and stop functions;
• The most important thing is the initialization of the RequestHandler instance. This abstract class is defined in the Transport module and is mainly used by the server to process requests from client s. Its abstract method implementation will be explained in detail in RpcServer class.

(3) common module

• The common module is mainly some reflection tools, and its specific implementation is as follows:
• One purpose of the getPublicMethods() method is to store the servciescript of all methods during Server registration.
• The invoke() method is used to execute the method of the specified instance object.

(4) codec module

• Encoder
• Decoder decoder

(5) server module

• One of the two core modules of the project is to define the method of processing client requests.
• The register() method is mainly used to register all common methods of the class, and obtain the ServiceDescriptor instance and ServiceInstance described earlier and store them as key value pairs.
• It mainly defines several variables, one is the target object of a method to be executed, and the other is the method to be executed.
• Its onRequest() method obtains the data from the Client through the inputStream and OutputStream parameters of the Servlet, and get s the instance object and method from the ServiceManager through the obtained Request instance parameters.
• Because the Request object contains the actual parameters obtained by the Client, the above parameters are passed to the ServiceInvoker object for execution.

(6) client module

• The main functions of this module are to connect one dynamic agent to obtain arguments, and the other is to request the Server to make procedure calls.
• Its RpcClient class is mainly used to handle the connection between the Client and the Server. It is equivalent to a connection pool, which returns the connection randomly when there is a demand.
• The getProxy() method of RpcClient class is a dynamic proxy, so we need to focus on the RemoteInvoker class.
• In the invoke() method, the parameters of the proxy method are stored, encapsulated into the Request object, and finally serialized and passed to the Server.

(7) Example example

• An example of the use of addition and subtraction

4.4 technology stack involved

• Fundamentals: Java, Maven, reflection, JDK dynamic agent
• Serialization: fastjason
• Network: Jetty, URLConnection

2, Realize

1. Class dependency graph

2. Implementation process

Additional question: what is the difference between dependencies and dependency management

(1) dependencies even if the dependency is not written in the child project, the child project will still inherit the dependency from the parent project (inherit all);

(2) Dependency management only declares dependencies and does not implement the introduction. Therefore, the dependencies required for the declarations to be displayed in the subproject. If the dependency is not declared in the child project, it will not be inherited from the parent project; Only when the dependency is written in the child project and no specific version is specified, the item will be inherited from the parent project, and both version and scope are read from the parent pom; In addition, if the version number is specified in the subproject, the jar version specified in the subproject is used.

Additional questions: Class, T and Class in Java generics <? >

A separate T represents a type, and class represents the class corresponding to this type. Class <? > Represents a class of uncertain type/

How to create an instance of Class type?

Just like using non generic code, there are two ways: calling method class Forname() or use the class constant X.class. Class.forName() is defined to return class <? >. On the other hand, the class constant X.class is defined as having the type class, so string Class is of type class.

Why do you need T modification in the method

The declaration of a generic type must be after the modifier of the method (public,static,final,abstract, etc.) and before the return value declaration.

public static <T> T request2Bean(HttpServletRequest request,Class<T> clazz){}

The first one is corresponding to the passed in parameter Class, which is equivalent to a generic type of return value. The following T is the return value type, which represents that the method must return T type (determined by the passed in Class)

Implementation process:

1.Construction project
2.Implementation of general module
3.Implement serialization module
4.Implement network module
5.realization server
6.realization client
7.gk-rpc Use case

rpc-proto :  Basic protocol encapsulation
    Peer: host+port
    ServiceDescriptor: The service description will be the corresponding [service description] in the registry key Value] contains [class, method, return value type, parameter type array], which uniquely determines a method
    Request: Request body,Hold [service description]+[request parameter array]
    Response: Default response body encapsulation
rpc_transport: Network service encapsulation
    TransportClient: Client encapsulation interface+[implementation]
        1.Create connection
        2.Send data waiting for response: inputstream，wait for outputstream
        3.Close connection
    TransportServer:Server encapsulation interface+[implementation]
        1.lsnrctl start : servlet Administration
        2.Receive request: receive the request, deserialize the object, process the call, and return data
        3.lsnrctl stop 
rpc-common: Tool class encapsulation
    ReflectionUtils: 
        according to class create object
        according to class Gets all public methods of this class
        invoke Method call: public static Object invoke(Object object, Method method, Object... args)
rpc-codec: Serialization encapsulation interface+[implementation]
    Encoder To binary
    Decoder Transfer object
rpc-server: Server encapsulation
    RpcServerConfig: Service configuration class
        HTTPTransportServer: Default service instance class
        JSONEncoder: Serialize instance class
        JSONDecoder: Deserialize instance class
        port: Listening port
    ServiceInstance: Instance of service --> Which object exposes which method
        target: object
        method: method
    ServiceManager: Administration rpc All services
        Map<ServiceDescriptor, ServiceInstance> services: To describe a service, use the service instance as key-value Storage, so that when the client sends it, it can find an accurate instance and call the correct method
        register: Service registration[ register(Class<T> interfaceClass, T bean) ]
            Interface class + object bean: Treat each method in the object as a ServiceInstance Register into map in
        lookup: Service lookup[ lookUp(Request request)]
            Get in request ServiceDescriptor，go map Remove from
    ServiceInvoke: [Service invocation]
        invoke(ServiceInstance serviceInstance, Request request):
            adopt request of ServiceDescriptor An instance of the service was found
            Call the method through reflection and pass in parameters
            ReflectionUtils.invoke(serviceInstance.getTarget(), serviceInstance.getMethod(),request.getParameters())
    RPCServer: [Encapsulation of services]
        1.set up RpcServerConfig config
        2.Get network instance by reflection     ReflectionUtils.newInstance(config.getTransportClass());
        3.Reflection get serialized instance   ReflectionUtils.newInstance(config.getEncoderClass());
        4.Reflection get deserialization instance ReflectionUtils.newInstance(config.getDecoderClass());
        5.Create service call object: this.serviceInvoke = new ServiceInvoke();
        6.To create a service management object: this.serviceManager = new ServiceManager();
        7.Initialize network instance: this.net.init(config.getPort(), this.handler); At this time, only the service information is ready, and listening is not enabled
        
        8.register: Service registration register(Class<T> interfaceClass, T bean) {serviceManager.register(interfaceClass, bean); }
        9.start: this.net.start open
        10.stop: this.net.stop  close
        11.handler Request processing:
            1.receive inputStream
            2.Deserialization get Request
            3.according to ServiceManager.lookup(request)Find instance ServiceInstance
            4.adopt Object invoke = serviceInvoke.invoke(sis, request);Get the response results and package them into response.setData(invoke);
            5.serialize Response
            6.write
rpc-client: Client encapsulation

Result display diagram: