Distributed transaction practice in microservices

background

With the rapid development of business and the increasing complexity of business, microservice, as one of the best solutions, decouples services, reduces complexity and increases maintainability, but also brings some new problems.

When we need to ensure data consistency across services, the original database transaction is unable to put multiple operations across databases and services into one transaction. There are many such application scenarios, and we can list many:

In the inter-bank transfer scenario, the data is not in the same database, but it is necessary to ensure that balance deduction and balance increase either succeed or fail at the same time
After publishing articles, update statistics such as the total number of articles. Publishing articles and updating statistics are usually in different microservices
Order system after microservicing
Travel needs to book several tickets in the third-party system at the same time

In the face of these scenarios where local transactions cannot be solved, we need a distributed transaction solution to ensure the consistency of updated data across services and databases.

Go zero combines with dtm to launch a minimalist scheme for seamless access to dtm in go zero, making the use of distributed transactions never so simple.

Run an example

Let's look at a runnable example, and then see how to develop and complete a complete distributed transaction by ourselves

Below, take etcd as the registration service center. You can run an example of go zero according to the following steps:

Configure dtm

MicroService:
    Driver: 'dtm-driver-gozero' # Configure dtm to use go zero micro service protocol
    Target: 'etcd://localhost:2379/dtmservice '# register dtm to this address of etcd
    EndPoint: 'localhost:36790' # Local address of dtm

Start etcd

# Prerequisite: etcd is installed
etcd

Start dtm

# Premise: the database link of dtm has been configured
go run app/main.go dev

Run a go zero service

git clone github.com/yedf/dtmdriver-clients &amp;&amp; cd dtmdriver-clients
cd gozero/trans &amp;&amp; go run trans.go

Initiate a dtm transaction with go zero

# In the directory of dtmdriver clients
cd gozero/app &amp;&amp; go run main.go

When you see in trans's log

2021/12/03 15:44:05 transfer out 30 cents from 1
2021/12/03 15:44:05 transfer in 30 cents to 2
2021/12/03 15:44:05 transfer out 30 cents from 1
2021/12/03 15:44:05 transfer out 30 cents from 1

That is, the transaction is completed normally

Development access

reference resources yedf/dtmdriver-clients Code of

// The following line imports the dtm driver of gozero
import _ "github.com/yedf/dtmdriver-gozero"

// Before using dtm client dtmgrpc, you need to execute the following line of calls to tell dtmgrpc how to use gozero's driver to handle gozero's url
err := dtmdriver.Use("dtm-driver-gozero")
// check err

// dtm has registered to the following address through the previous configuration, so it is used in dtmgrpc
var dtmServer = "etcd://localhost:2379/dtmservice"

// Next, Load the configuration from the configuration file, and then obtain the address of the business service through BuildTarget
var c zrpc.RpcClientConf
conf.MustLoad(*configFile, &amp;c)
busiServer, err := c.BuildTarget()

  // Generate a message type distributed transaction using dtmgrpc and commit it
	gid := dtmgrpc.MustGenGid(dtmServer)
	msg := dtmgrpc.NewMsgGrpc(dtmServer, gid).
    // The first step in the transaction is to call trans TransSvcClient. TransOut
    // From trans pb. The Method name corresponding to the above Method found in go is "/ trans.TransSvc/TransOut"
    // dtm needs to call this method from the dtm server, so instead of strong type, it uses the dynamic url: busiServer+"/trans.TransSvc/TransOut"
		Add(busiServer+"/trans.TransSvc/TransOut", &amp;busi.BusiReq{Amount: 30, UserId: 1}).
		Add(busiServer+"/trans.TransSvc/TransIn", &amp;busi.BusiReq{Amount: 30, UserId: 2})
	err := msg.Submit()

The whole development access process is very few. The previous notes are very clear and will not be repeated.

matters needing attention

In the process of developing access, find * pb. When accessing the method path of grpc in the go file, you must find the path of invoke

Deep understanding of dynamic invocation

When go zero uses dtm distributed transactions, many calls are initiated from the dtm server, such as the Confirm/Cancel of TCC and all calls of SAGA/MSG.

dtm does not need to know the strong types of business APIs that make up distributed transactions. It calls these APIs dynamically.

The call of grpc can be similar to HTTP POST, where:

c. The target generated by buildtarget() is similar to the Host in the URL
/trans.TransSvc/TransOut is equivalent to Path in URL
& busi. Busireq {amount: 30, userid: 1} is equivalent to the Body in Post
pb.Response is equivalent to the response of an HTTP request

Through the following code, dtm will get the complete information and be able to initiate a complete call

Add(busiServer+"/trans.TransSvc/TransOut", &busi.BusiReq{Amount: 30, UserId: 1})

More complete examples

Mikael, a warm-hearted community student, helped write a richer example. Combined with practical applications and sub transaction barriers, he completely demonstrated a distributed transaction actually running online. Interested students can refer to:

https://github.com/Mikaelemmmm/gozerodtm

Access by other means

There are other non etcd access methods for go zero microservices. We will explain their access methods in turn

Direct connection

For direct connection, you only need to set the Target to an empty string based on the etcd configuration of dtm above.

In the case of direct connection, dtm does not need to be registered in the registry.

K8S

For K8S, you only need to set the Target to an empty string based on the etcd configuration of dtm above.

In K8S, the service is registered in K8S by deployment Yaml is completed. The application is internal and does not need to be registered.

Live sharing Preview

Go zero author and I (dtm author) will read at go night at 21 p.m. on December 22 to jointly share a live broadcast of go zero's distributed transaction practice, which will bring more and more in-depth discussions. Welcome to participate at that time.

The live address is: https://live.bilibili.com/11171965

Summary

This time, the cooperation between go zero and dtm has created the first microservice solution supporting distributed transactions in go ecology, which is of great significance.

Go zero project address: https://github.com/zeromicro/go-zero
Go zero project address: https://gitee.com/kevwan/go-zero
dtm project address: https://github.com/yedf/dtm

Welcome to go zero and dtm, use our native distributed transaction microservice solution, and star support us!

Wechat communication group

Focus on the "micro service practice" official account and click on the exchange group to get the community community's two-dimensional code.

Keywords: Go github etcd

Added by dserf on Tue, 21 Dec 2021 05:56:00 +0200

Programming VIP