LCN
TxLCN distributed transaction framework,
Multiple service modules manipulate the same data source,
It is also possible for a service to span multiple data source nodes
Either all succeed or all fail.
Solved by TxLCN.
Common solutions
-
Global transaction 2pc -- relational database
Disadvantages: low efficiency. During the execution of the algorithm, all nodes are blocked and the resources held by all nodes are blocked.
-
3pc (three stage submission agreement)
-
Message Oriented Middleware
-
TCC GTC
-
Provide rollback interface
-
Distributed database
CAP theorem: in a distributed system, consistency, availability and partition tolerance can satisfy at most two of them at the same time, and three can't have both.
Availability and consistency are contradictory (if one node is broken, another node has to bear it).
Partition fault tolerance means that the system cannot achieve data consistency within A certain time limit, which means that partition occurs. At this time, we must choose between A and C.
BASE theory: optimization of CAP theory. Basically available, soft state and finally consistent.
Even if strong consistency cannot be achieved, each application can adopt appropriate methods to achieve final consistency according to its own business characteristics.
BA basic availability: when an unknown fault occurs in the distributed system, it is allowed to lose some availability, but it does not mean that the whole system is unavailable. (e.g. concurrent degradation page of second kill activity)
S soft state: the data in the system is allowed to have an intermediate state (soft state), which does not affect the availability and allows synchronization delay between distributed nodes. (such as Eureka cluster synchronization data)
Final consistency: it allows the data of the whole system to reach the consistency of the whole system after a certain time. For weak consistency, the whole system does not achieve consistency when responding to the user results, but it will eventually achieve consistency.
Strong consistency: after the system accepts the request, the whole system must achieve consistent results before responding.
LCN source
-
Lock transaction unit (lock)
-
Determine transaction module status (Confirm)
-
Notification transaction (Notify)
Three transaction modes
-
LCN @LcnTransaction
The operation of local transactions is realized based on jdbc relational database.
-
TCC @TccTransaction
Based on the non relational database (non relational is non transactional, which also supports relational database), the distributed transaction is realized through the scheduling of business logic. There are three steps: try (try to execute the business), confirm (confirm the execution of the business) and cancel (cancel the execution of the business). Developers need to write their own operations on the database.
XA two phase commit protocol
The interface specification between transaction middleware and database defined by X/Open DTP. It is used to notify the start, end, commit and rollback of database transactions. XA interface functions are provided by database manufacturers.
X/Open DTP includes application program (AP), transaction manager (TM), resource manager (RM, general database) and communication resource manager (CRM).
In the second stage, the transaction manager is responsible for coordinating the service AP to ask for data from different RM S, and submitting the transaction as late as possible (completing all work before submission), but also waiting for TM response, which is easy to block. Throughput is preferred and is not suitable for the Internet.
-
TXC @TxcTransaction
The name of TXC mode comes from Taobao. Before executing sql, query the data of sql, then save the sql and create locks. When rollback is needed, rollback the database according to these recorded data (currently, the implementation of locks depends on redis distributed lock control)
-
Features: low embedding, will not occupy the connection resources of the database, and consumes more resources.
-
TxLCN principle
It is composed of two modules, txclient and txmanager;
TxClient is the dependency framework of the module and provides standard support for TX LCN. Both the initiator and participant of the transaction belong to TxClient
TxManager is responsible for controlling the whole transaction.
123 publishes their own transactions to txmanager on behalf of three txclients. 4. The initiator tells txmanager whether the data acquisition is successful or failed. If it is successful (one of the failures is also considered as failure), txmanager returns notification AB to let them submit their own transactions. 5. Txmanager notifies the initiator to submit their own transactions.
Create transaction group: the transaction initiator calls txManager to create a transaction group object before executing the business code, and gets the transaction representation GroupId.
Join transaction groups: 2 and 3. Participants report their transaction execution to txManager
Notify transaction group: after the initiator executes the business code, it notifies txManager of the execution result status
Notification transaction unit: txManager asks participants about the transaction execution results
Response notification transaction group: txManager notifies the initiator of the final result of the transaction
Create a new TxManager project and act as a manager
Execute the tx-manager.sql script file with the dependency
Configure application.properties
spring.application.name=TransactionManager
server.port=7970
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost:3306/tx-manager?characterEncoding=UTF-8
spring.datasource.username=root
spring.datasource.password=123456
spring.redis.host=192.168.234.100
tx-lcn.logger.enabled=false
Startup class plus @ enabletransactionmanagerserver
In case of servertimezone problem, add serverTimezone=GMT
visit
If the transaction is open: change tx-lcn.logger.enabled=true
Dependency:
<dependencies> <dependency> <groupId>redis.clients</groupId> <artifactId>jedis</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>com.codingapi.txlcn</groupId> <artifactId>txlcn-tm</artifactId> <version>5.0.2.RELEASE</version> </dependency> </dependencies> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>${spring-cloud.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build>
The tid of the newly created teacher and student student depends on the teacher. Only when the teacher is successfully inserted, s can it be successfully inserted. Do not add foreign keys first.
Use two different services to add different table data
Now, in the process of adding students, if an error (South (0)) occurs after adding teacher data, it will appear that the teacher's addition has been submitted, but the student's addition has not been submitted
At this time, @ transactional is not enough only in the addition method of students, @ transactional is only limited to local transactions, which is a distributed transaction.
Participant configuration
Introduce dependency
<dependency> <groupId>com.codingapi.txlcn</groupId> <artifactId>txlcn-tc</artifactId> <version>5.0.2.RELEASE</version> </dependency> <dependency> <groupId>com.codingapi.txlcn</groupId> <artifactId>txlcn-txmsg-netty</artifactId> <version>5.0.2.RELEASE</version> </dependency>
Above the insertion methods of teacher and student
@Transavtional @LcnTransactional//(propogation Isolation level)
Add above the startup classes of teacher and student
@EnableDistributedTransaction
Set the address in the application.properties file of teacher and student
#Configure the transaction port of the transaction manager TxManager
tx-lcn.client.manager-address: 127.0.0.1:8070
TCC (example of non transactional database)
docker container ls -a
docker start mango
Suppose the teacher wants to store the data in mangodb
Add the following dependencies
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-mongodb</artifactId> </dependency> <!-- data source--> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-jdbc</artifactId> </dependency> <!--data source--> <dependency> <groupId>com.alibaba</groupId> <artifactId>druid-spring-boot-starter</artifactId> <version>1.2.1</version> </dependency> <!--Connection drive --> <dependency> <groupId>mysql</groupId></groupId> <artifactId>mysql-connector-java</artifactId> <version>5.1.49</version> </dependency>
Modify connection mode configuration
spring.data.mongodb.host=192.168.234.100
spring.data.mongodb.authentication-database=admin
spring.data.mongodb.username=lc
spring.data.mongodb.password=123456
spring.data.mongodb.database=ego
Modify pojo class
Auto assemble above insertion method
@MongoTemplate mongoTemplate
The insertion method is @ Transactional
TCC mode requires manual processing by developers, so TCC is also called code compensation business. mongodb has no transactions and needs to be deleted manually. There are three methods:
-
xxx()
-
confirmXxx()
-
cancelXxx()
@Autowired @MongoTemplate mongoTemplate @TccTransaction public int insert(Teacher teacher){ System.out.println("Trying to add data") Teacher result = mongoTemplate.insert(teache); if(result!=null){return 1;} return 0; } public void confirmInsert(Teacher teacher){ System.out.println("teacher="+teacher) //It just has the function of confirmation } public void cancelInsert(Teacher teacher){ //this teacher With data Criteria criteria = Criteria.where("id").is(teacher.getId()); Query query=new Query(criteria); DeleteResult deleteResut=mongoTemplate.remove(query,Teacher.class); System.out.println("deleteResut.getDeletedCount="+deleteResut.getDeletedCount()) }
The student side does not need to add @ TccTransaction, but it is still lcn (the database has not been changed)
Seata
Data is stored on different data servers.
Software: components
Transaction Coordinator(TC): a transaction coordinator that maintains the running state of global transactions, which is equivalent to Seata's server
Transaction Manager (TM): Transaction Manager. It is responsible for starting a global transaction and finally initiating a global commit or global rollback
The Resource Manger(RM) resource manager registers branch transactions with the transaction manager, reports the status of branch transactions, and is responsible for branch transaction submission or rollback
Transaction control process:
-
TM applies to TC to start global transaction (@ GlobalTransaction)
-
TC returns the unique XID of the global transaction to TM
-
Branch service ① applies to TC to register branch service, and TC returns BranchID to branch service ①
-
Branch service ① performs operation (RM), records operation log undo_log, submits local branch transactions, and reports transaction processing results to TC
-
Branch service ① remote call branch service ②
-
The branch service performs the same operation (RM) as ① to obtain the unique BranchID,undo_log and report
-
After the branch service is processed, TM submits the global transaction resolution to TC and asks whether all of them are executed successfully.
-
If there is a problem with the transaction, reverse the undo_log operation to rollback the local transaction.
Add dependency:
<seata.version>1.4.2</seata.version> <dependency> <groupId>io.seata</groupId> <artifactId>seata-all</artifactId> <version>${seata.version}</version> </dependency>
https://seata.io/en-us/blog/download.html Download the corresponding version of seata server
After decompression, get file.conf and registry.conf in the \ seata\conf directory
Modify the mod and the corresponding URL under file.conf. Driverclassname currently has three options: file, db and redis
After selecting the db mode, you need to create three tables in the corresponding database: globalTable (persistent global transaction), branchTable (persistent transactions of each submitted branch) and lockTable (persistent resource locked transactions of each branch)
-- the table to store GlobalSession data
-- Persistent global transactions
CREATE TABLE IF NOT EXISTS `global_table`
(
`xid` VARCHAR(128) NOT NULL,
`transaction_id` BIGINT,
`status` TINYINT NOT NULL,
`application_id` VARCHAR(32),
`transaction_service_group` VARCHAR(32),
`transaction_name` VARCHAR(128),
`timeout` INT,
`begin_time` BIGINT,
`application_data` VARCHAR(2000),
`gmt_create` DATETIME,
`gmt_modified` DATETIME,
PRIMARY KEY (`xid`),
KEY `idx_gmt_modified_status` (`gmt_modified`, `status`),
KEY `idx_transaction_id` (`transaction_id`)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8;
-- the table to store BranchSession data
-- Persistent transactions of each committed branch
CREATE TABLE IF NOT EXISTS `branch_table`
(
`branch_id` BIGINT NOT NULL,
`xid` VARCHAR(128) NOT NULL,
`transaction_id` BIGINT,
`resource_group_id` VARCHAR(32),
`resource_id` VARCHAR(256),
`branch_type` VARCHAR(8),
`status` TINYINT,
`client_id` VARCHAR(64),
`application_data` VARCHAR(2000),
`gmt_create` DATETIME(6),
`gmt_modified` DATETIME(6),
PRIMARY KEY (`branch_id`),
KEY `idx_xid` (`xid`)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8;
-- the table to store lock data
-- Persist each branch lock table transaction
CREATE TABLE IF NOT EXISTS `lock_table`
(
`row_key` VARCHAR(128) NOT NULL,
`xid` VARCHAR(96),
`transaction_id` BIGINT,
`branch_id` BIGINT NOT NULL,
`resource_id` VARCHAR(256),
`table_name` VARCHAR(32),
`pk` VARCHAR(36),
`gmt_create` DATETIME,
`gmt_modified` DATETIME,
PRIMARY KEY (`row_key`),
KEY `idx_branch_id` (`branch_id`)
) ENGINE = InnoDB
DEFAULT CHARSET = utf8;
The registry.conf file sets the registry and configuration center. I chose to put them all in Nacos (additional steps are required, see https://blog.csdn.net/qq853632587/article/details/111644295)
The downloaded nacos-config.sh is placed in the conf directory
https://github.com/seata/seata/blob/develop/script/config-center/config.txt
config.txt refers to seata's various detailed configurations. You can import these configurations into Nacos by executing nacos-config.sh, so you don't need to put file.conf and registry.conf into our project. You can read the required configurations directly from Nacos.
In the conf file git bash
sh nacos-config.sh -h localhost -p 8848 -g SEATA_GROUP -t 0af6e97b-a684-4647-b696-7c6d42aecce7 -u nacos -w nacos
Each service is configured
spring:
application:
name: application_name
datasource:
driver-class-name: com.mysql.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/seata
username: root
password: 123456
seata:
enabled: true
enable-auto-data-source-proxy: true
tx-service-group: my_test_tx_group
registry:
type: nacos
nacos:
application: seata-server
server-addr: 127.0.0.1:8848
username: nacos
password: nacos
config:
type: nacos
nacos:
server-addr: 127.0.0.1:8848
group: SEATA_GROUP
username: nacos
password: nacos
namespace: own namespace
service:
vgroup-mapping:
my_test_tx_group: default
disable-global-transaction: false
client:
rm:
report-success-enable: false
Add @ GlobalTransactional above the methods (different methods) of the business logic layer
MQ transaction message
Some third-party MQ supports transaction messages, such as RocketMQ, but neither RabbitMQ nor Kafka supports transaction messages. The way to support transaction messages is also similar to two-phase commit.
For example: let Alipay account deduct ten thousand, we generate a credential (message), this message is written to let Yu Ebao account increase ten thousand. We can take this message to achieve final consistency.
Message decoupling
-
Before sending the payment transaction, Alipay sends messages to the real-time message service request. The real time message service only records the message data, but does not actually send (only know that this message). Only when the message is sent successfully, will the transaction be submitted.
-
First encapsulate (Alipay -10000) into a message (New Message ()), and then submit the message to the MQ server. When the local transaction is completed, it will be successful: (give MQ a logo: COMMIT) or fail: (give MQ a logo: ROLLBACK).
send(producer.send(new Message(),callback(It handles local affairs))) //stay callback Processing local transactions: in callback In the method: update A set amount = amount - 10000 where userId = 1;
-
When the Alipay payment transaction is successfully submitted, it sends confirmation to the real-time message service. Only after the confirmation of the sending instruction is received, the real-time message service really sends the message.
-
When the Alipay payment transaction fails to roll back, the message is cancelled to the real time message service. The message will not be sent.
-
For those unconfirmed messages or cancelled messages, a message state confirmation system is needed to update the status of Alipay message query.
Advantages: message data is stored independently, reducing the coupling between business system and message system
Disadvantages: one message sending requires two requests; the business processing service needs to implement the message status check interface.
