This paper introduces the implementation principle of PooledDataSource.
Data source configuration
In mybatis config In the XML configuration file, you can configure the data source by setting the dataSource tag.
<environments default="development">
<environment id="development">
<transactionManager type="JDBC">
<property name="..." value="..."/>
</transactionManager>
<dataSource type="POOLED">
<property name="driver" value="${driver}"/>
<property name="url" value="${url}"/>
<property name="username" value="${username}"/>
<property name="password" value="${password}"/>
</dataSource>
</environment>
</environments>
The following configuration description is intercepted from Official document - XML configuration - environments
The dataSource element uses the standard JDBC data source interface to configure the resources of the JDBC connection object.
There are three built-in data source types (that is, type="[UNPOOLED|POOLED|JNDI]):
UNPOOLED
The implementation of this data source opens and closes the connection each time it is requested. Although a little slow, it is a good choice for simple applications that do not require high database connection availability.
Performance depends on the database used. For some databases, using connection pool is not important. This configuration is very suitable for this situation. UNPOOLED data sources only need to configure the following five attributes:
- driver – this is the fully qualified name of a JDBC driven Java class (not a data source class that may be included in a JDBC driver).
- url – this is the JDBC URL address of the database.
- username – the user name to log in to the database.
- Password – password to log in to the database.
- defaultTransactionIsolationLevel – the default connection transaction isolation level.
- defaultNetworkTimeout – the default network timeout in milliseconds to wait for a database operation to complete. Check the API documentation for java.sql.Connection#setNetworkTimeout() for more information.
Optionally, you can also pass properties to the database driver. Just add "driver." to the property name Prefix, for example:
- driver.encoding=UTF8
This will be via drivermanager The getconnection (URL, driverproperties) method passes the encoding property with the value of UTF8 to the database driver.
POOLED
The implementation of this data source uses the concept of "pool" to organize JDBC connection objects, avoiding the necessary initialization and authentication time when creating new connection instances.
This processing method is very popular and can enable concurrent Web applications to respond to requests quickly.
In addition to the above mentioned attributes under UNPOOLED, there are more attributes to configure the data source of POOLED:
- poolMaximumActiveConnections – the number of active (in use) connections that can exist at any time. Default: 10
- poolMaximumIdleConnections – the number of idle connections that may exist at any time.
- poolMaximumCheckoutTime – the time that connections in the pool are checked out before being forcibly returned. The default value is 20000 milliseconds (i.e. 20 seconds)
- poolTimeToWait – this is an underlying setting. If it takes a long time to obtain a connection, the connection pool will print the status log and try to obtain a connection again (to avoid continuous failure in case of misconfiguration and no log printing). The default value is 20000 milliseconds (i.e. 20 seconds).
- poolMaximumLocalBadConnectionTolerance – this is a low-level setting for bad connection tolerance that applies to every thread trying to get a connection from the cache pool. If the thread gets a bad connection, the data source allows the thread to try to get a new connection again, but the number of retries should not exceed the sum of poolMaximumIdleConnections and poolMaximumLocalBadConnectionTolerance. Default value: 3 (added to 3.4.5)
- poolPingQuery – a probe query sent to the database to verify that the connection is working properly and ready to accept the request. The default is "NO PING QUERY SET", which will cause most database drivers to return appropriate error messages when errors occur.
- poolPingEnabled – whether to enable detection query. If enabled, you need to set the poolPingQuery property to an executable SQL statement (preferably a very fast SQL statement). The default value is false.
- poolPingConnectionsNotUsedFor – configure the frequency of poolPingQuery. It can be set as the database connection timeout to avoid unnecessary detection. The default value is 0 (that is, all connections are detected at every moment - of course, it is only applicable when poolPingEnabled is true).
JNDI
This data source implementation is to be used in containers such as EJB or application server. The container can configure the data source centrally or externally, and then place a data source reference of JNDI context. This data source configuration requires only two properties:
- initial_ Context – this attribute is used to find the context in the InitialContext (that is, initialContext.lookup(initial_context)). This is an optional attribute. If it is ignored, data will be found directly from InitialContext_ Source property.
- data_source – this is the context path that references the location of the data source instance. Initial is provided_ When the context is configured, it will be found in the returned context. If it is not provided, it will be found directly in InitialContext.
Similar to other data source configurations, you can add the prefix "env." Pass the property directly to InitialContext. For example:
- env.encoding=UTF8
This will pass the encoding attribute with the value of UTF8 to its constructor when initializcontext is instantiated.
Third party data sources
You can implement the interface org apache. ibatis. datasource. Datasourcefactory to use third-party data sources to implement:
public interface DataSourceFactory {
void setProperties(Properties props);
DataSource getDataSource();
}
org.apache.ibatis.datasource.unpooled.UnpooledDataSourceFactory can be used as a parent class to build a new data source adapter, such as the following code necessary to insert C3P0 data source:
import org.apache.ibatis.datasource.unpooled.UnpooledDataSourceFactory;
import com.mchange.v2.c3p0.ComboPooledDataSource;
public class C3P0DataSourceFactory extends UnpooledDataSourceFactory {
public C3P0DataSourceFactory() {
this.dataSource = new ComboPooledDataSource();
}
}
In order to make it work, remember to add corresponding properties to each setter method you want MyBatis to call in the configuration file. The following is an example of connecting to a PostgreSQL database:
<dataSource type="org.myproject.C3P0DataSourceFactory"> <property name="driver" value="org.postgresql.Driver"/> <property name="url" value="jdbc:postgresql:mydb"/> <property name="username" value="postgres"/> <property name="password" value="root"/> </dataSource>
Source code analysis
This section takes the < datasource type = "pooled" > configuration as an example to explore the implementation principle of PooledDataSource.
Implementation principle of PooledDataSource
Constructor
The PooledDataSource constructor is as follows. You can see that when creating a PooledDataSource object, an UnpooledDataSource object will be created.
Meanwhile, when instantiating the PooledDataSource object, a PoolState instance will be created.
private final PoolState state = new PoolState(this); // Used to store database connection objects
private final UnpooledDataSource dataSource; // Used to create database connection objects
private int expectedConnectionTypeCode; // Database connection ID, hash value of url+username+password string
public PooledDataSource() {
dataSource = new UnpooledDataSource();
}
public PooledDataSource(UnpooledDataSource dataSource) {
this.dataSource = dataSource;
}
public PooledDataSource(String driver, String url, String username, String password) {
dataSource = new UnpooledDataSource(driver, url, username, password);
expectedConnectionTypeCode = assembleConnectionTypeCode(dataSource.getUrl(), dataSource.getUsername(), dataSource.getPassword());
}
public PooledDataSource(String driver, String url, Properties driverProperties) {
dataSource = new UnpooledDataSource(driver, url, driverProperties);
expectedConnectionTypeCode = assembleConnectionTypeCode(dataSource.getUrl(), dataSource.getUsername(), dataSource.getPassword());
}
public PooledDataSource(ClassLoader driverClassLoader, String driver, String url, String username, String password) {
dataSource = new UnpooledDataSource(driverClassLoader, driver, url, username, password);
expectedConnectionTypeCode = assembleConnectionTypeCode(dataSource.getUrl(), dataSource.getUsername(), dataSource.getPassword());
}
public PooledDataSource(ClassLoader driverClassLoader, String driver, String url, Properties driverProperties) {
dataSource = new UnpooledDataSource(driverClassLoader, driver, url, driverProperties);
expectedConnectionTypeCode = assembleConnectionTypeCode(dataSource.getUrl(), dataSource.getUsername(), dataSource.getPassword());
}
Database connection - original object
The connection address, user name and password information of the database are saved in the UnpooledDataSource object.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-dsLQtzNs-1629363020507)(./images/1629343066373.png)]
Why save an UnpooledDataSource object in a PooledDataSource object?
This is to use UnpooledDataSource to establish a connection to the database.
For example, in the case of using MySQL driver, a Socket connection will be established to the MySQL server and a com mysql. cj. jdbc. Connectionimpl connection object.
org.apache.ibatis.datasource.unpooled.UnpooledDataSource#getConnection()
org.apache.ibatis.datasource.unpooled.UnpooledDataSource#doGetConnection(java.lang.String, java.lang.String)
org.apache.ibatis.datasource.unpooled.UnpooledDataSource#doGetConnection(java.util.Properties)
private Connection doGetConnection(Properties properties) throws SQLException {
initializeDriver();
Connection connection = DriverManager.getConnection(url, properties); // Using the database driver package, create the connection object
configureConnection(connection);
return connection;
}
Database connection - proxy object
PoolState in PooledDataSource is an internal class used to store database connection objects and record statistics.
public class PoolState {
protected PooledDataSource dataSource;
protected final List<PooledConnection> idleConnections = new ArrayList<>(); // Idle connection
protected final List<PooledConnection> activeConnections = new ArrayList<>(); // Active connection
protected long requestCount = 0; // Number of requests
protected long accumulatedRequestTime = 0; // Total request time
protected long accumulatedCheckoutTime = 0; // Total check-out time (removing connections from the pool is called check-out)
protected long claimedOverdueConnectionCount = 0; // Number of connections declared expired
protected long accumulatedCheckoutTimeOfOverdueConnections = 0; // Total number of expired connections
protected long accumulatedWaitTime = 0; // Total waiting time
protected long hadToWaitCount = 0; // Number of times to wait
protected long badConnectionCount = 0; // Bad connections
public PoolState(PooledDataSource dataSource) {
this.dataSource = dataSource;
}
}
PoolState does not store the original connection object, such as com mysql. cj. jdbc. Instead of connectionimpl, the PooledConnection object is stored.
The dynamic proxy of JDK is used here. Every time a PooledConnection is created, a dynamic proxy will be created for the original connection object.
The purpose of using proxy is to change the behavior of Connection:
- Change the connection closing behavior of Connection#close to return the connection to the connection pool.
- Before each connection is used, the PooledConnection#checkConnection method is called to check whether the connection is valid.
What happens when TODO detects that the connection is invalid?
org.apache.ibatis.datasource.pooled.PooledConnection
class PooledConnection implements InvocationHandler { // Equivalent to a tool class
private static final String CLOSE = "close";
private static final Class<?>[] IFACES = new Class<?>[] { Connection.class };
private final int hashCode;
private final PooledDataSource dataSource;
private final Connection realConnection; // Original class - database connection
private final Connection proxyConnection; // Proxy class - database connection
private long checkoutTimestamp; // Timestamp checked out from connection pool
private long createdTimestamp; // Created timestamp
private long lastUsedTimestamp; // Last used timestamp
private int connectionTypeCode;
private boolean valid; // Is the connection valid
/**
* Constructor for SimplePooledConnection that uses the Connection and PooledDataSource passed in.
*
* @param connection
* - the connection that is to be presented as a pooled connection
* @param dataSource
* - the dataSource that the connection is from
*/
public PooledConnection(Connection connection, PooledDataSource dataSource) { // Pass in the original class and get the proxy class
this.hashCode = connection.hashCode();
this.realConnection = connection;
this.dataSource = dataSource;
this.createdTimestamp = System.currentTimeMillis();
this.lastUsedTimestamp = System.currentTimeMillis();
this.valid = true;
this.proxyConnection = (Connection) Proxy.newProxyInstance(Connection.class.getClassLoader(), IFACES, this); // JDK dynamic agent
}
/**
* Required for InvocationHandler implementation.
*
* @param proxy
* - not used
* @param method
* - the method to be executed
* @param args
* - the parameters to be passed to the method
* @see java.lang.reflect.InvocationHandler#invoke(Object, java.lang.reflect.Method, Object[])
*/
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { // Proxy method
String methodName = method.getName();
if (CLOSE.equals(methodName)) { // Change the behavior of closing the connection to putting it back into the connection pool
dataSource.pushConnection(this);
return null;
}
try {
if (!Object.class.equals(method.getDeclaringClass())) {
// issue #579 toString() should never fail
// throw an SQLException instead of a Runtime
checkConnection(); // Check the connection before using it
}
return method.invoke(realConnection, args);
} catch (Throwable t) {
throw ExceptionUtil.unwrapThrowable(t);
}
}
private void checkConnection() throws SQLException {
if (!valid) {
throw new SQLException("Error accessing PooledConnection. Connection is invalid.");
}
}
public boolean isValid() { // Verify that the connection is valid
return valid && realConnection != null && dataSource.pingConnection(this);
}
PooledDataSource#pingConnection
During the use of PooledDataSource, the PooledConnection#isValid method will be called to check whether the connection is valid.
Properties related to connection checking in PooledDataSource class:
// A probe query sent to the database to verify that the connection is working properly and ready to accept the request. protected String poolPingQuery = "NO PING QUERY SET"; // Whether to enable detection query. If enabled, you need to set the poolPingQuery property to an executable SQL statement (preferably a very fast SQL statement). The default value is false. protected boolean poolPingEnabled; // Configure the frequency of poolPingQuery. It can be set as the database connection timeout to avoid unnecessary detection. The default value is 0 (that is, all connections are detected at every moment - of course, it is only applicable when poolPingEnabled is true). protected int poolPingConnectionsNotUsedFor;
When the following conditions are met, the SQL statement configured by poolPingQuery will be sent to the database.
- The database connection was not closed.
- Configure poolPingEnabled to true in MyBatis XML.
- The time since the last use of the connection is greater than the connection check frequency.
org.apache.ibatis.datasource.pooled.PooledDataSource#pingConnection
/**
* Method to check to see if a connection is still usable
*
* @param conn
* - the connection to check
* @return True if the connection is still usable
*/
protected boolean pingConnection(PooledConnection conn) { // Verify that the connection is valid
boolean result = true;
try {
result = !conn.getRealConnection().isClosed(); // Verify that the database connection session is closed eg. com mysql. cj. jdbc. ConnectionImpl. isClosed
} catch (SQLException e) {
if (log.isDebugEnabled()) {
log.debug("Connection " + conn.getRealHashCode() + " is BAD: " + e.getMessage());
}
result = false;
}
if (result && poolPingEnabled && poolPingConnectionsNotUsedFor >= 0 // The connection that needs to be checked is configured
&& conn.getTimeElapsedSinceLastUse() > poolPingConnectionsNotUsedFor) { // The time since the last use of the connection is greater than the connection check frequency
try {
if (log.isDebugEnabled()) {
log.debug("Testing connection " + conn.getRealHashCode() + " ...");
}
Connection realConn = conn.getRealConnection();
try (Statement statement = realConn.createStatement()) {
statement.executeQuery(poolPingQuery).close(); // Send a simple statement to check whether the connection is valid
}
if (!realConn.getAutoCommit()) {
realConn.rollback();
}
result = true;
if (log.isDebugEnabled()) {
log.debug("Connection " + conn.getRealHashCode() + " is GOOD!");
}
} catch (Exception e) {
log.warn("Execution of ping query '" + poolPingQuery + "' failed: " + e.getMessage()); // Connection check failed
try {
conn.getRealConnection().close(); // Try closing the connection
} catch (Exception e2) {
// ignore
}
result = false;
if (log.isDebugEnabled()) {
log.debug("Connection " + conn.getRealHashCode() + " is BAD: " + e.getMessage());
}
}
}
return result;
}
The time stamp of the last connection used is recorded in the PooledConnection object.
org.apache.ibatis.datasource.pooled.PooledConnection#getTimeElapsedSinceLastUse
/**
* Getter for the time since this connection was last used.
*
* @return - the time since the last use
*/
public long getTimeElapsedSinceLastUse() {
return System.currentTimeMillis() - lastUsedTimestamp;
}
PooledDataSource#popConnection
Take out the database connection from the pool. The complete code is as follows:
Code flow:
- The while loop is used to fetch connections from the database connection pool (this operation is called check out). The PoolState state object lock needs to be obtained at the beginning of each loop.
- Detect the free connection set and active connection set in PoolState and obtain the connection object from them. There are several cases:
2.1 if the free connection set is not empty, take out a connection from it.
2.2 if the free connection set is empty and the active connection set is not full, a new connection is established by using the database driver package and packaged as a PooledConnection object (generating a dynamic agent).
2.3 if the free connection set is empty and the active connection set is full, you need to check the oldest connection:
2.3. 1 if the connection has timed out (the check-out time of the proxy object is greater than poolMaximumCheckoutTime, but the original connection may still exist), mark the proxy object PooledConnection as invalid and encapsulate the original connection as a new PooledConnection object.
2.3. 2 if the connection does not time out, the current check-out thread enters waiting. - Go to this step. If the PooledConnection connection is checked out from PoolState successfully, check whether the connection is valid:
3.1 if the connection is invalid, re-enter the while loop.
3.2 if the connection is valid, set the relevant timestamp and store it in the active connection set to end the while loop.
org.apache.ibatis.datasource.pooled.PooledDataSource#popConnection
private PooledConnection popConnection(String username, String password) throws SQLException {
boolean countedWait = false;
PooledConnection conn = null;
long t = System.currentTimeMillis();
int localBadConnectionCount = 0;
while (conn == null) { // Loop check out connections
synchronized (state) { // Every time you cycle, you have to re acquire the lock!
if (!state.idleConnections.isEmpty()) { // If the set of free connections is not empty, the connections (all valid) are taken from it
// Pool has available connection
conn = state.idleConnections.remove(0); // Remove and return
if (log.isDebugEnabled()) {
log.debug("Checked out connection " + conn.getRealHashCode() + " from pool.");
}
} else {
// Pool does not have available connection / / if the free connection collection is empty, check the active connection collection
if (state.activeConnections.size() < poolMaximumActiveConnections) { // If the active connection set is not full, a new database connection is established
// Can create new connection
conn = new PooledConnection(dataSource.getConnection(), this); // Using the database driver package, create the connection object, and then wrap it as a proxy object
if (log.isDebugEnabled()) {
log.debug("Created connection " + conn.getRealHashCode() + ".");
}
} else {
// Cannot create new connection / / if the free connection collection is empty and the active connection collection is full, you need to process expired active connections
PooledConnection oldestActiveConnection = state.activeConnections.get(0);
long longestCheckoutTime = oldestActiveConnection.getCheckoutTime();
if (longestCheckoutTime > poolMaximumCheckoutTime) { // For the first connection put into the active connection collection, if its check-out time has expired (that is, it has been out of the pool for too long)
// Can claim overdue connection
state.claimedOverdueConnectionCount++;
state.accumulatedCheckoutTimeOfOverdueConnections += longestCheckoutTime;
state.accumulatedCheckoutTime += longestCheckoutTime;
state.activeConnections.remove(oldestActiveConnection); // Remove from active connection collection
if (!oldestActiveConnection.getRealConnection().getAutoCommit()) {
try {
oldestActiveConnection.getRealConnection().rollback();
} catch (SQLException e) {
/*
Just log a message for debug and continue to execute the following // Rollback failed as if nothing had happened
statement like nothing happened.
Wrap the bad connection with a new PooledConnection, this will help // Wrap the bad connection as a new PooledConnection object
to not interrupt current executing thread and give current thread a // The thread currently executing the task will not be interrupted. The thread can take other valid connections from the connection pool later
chance to join the next competition for another valid/good database
connection. At the end of this loop, bad {@link @conn} will be set as null. // At the end of this cycle, the bad connection will be set to null
*/
log.debug("Bad connection. Could not roll back");
}
}
conn = new PooledConnection(oldestActiveConnection.getRealConnection(), this); // It needs to be identified as a bad connection later! How to identify? Via PooledConnection#isValid
conn.setCreatedTimestamp(oldestActiveConnection.getCreatedTimestamp());
conn.setLastUsedTimestamp(oldestActiveConnection.getLastUsedTimestamp());
oldestActiveConnection.invalidate(); // Set to invalid
if (log.isDebugEnabled()) {
log.debug("Claimed overdue connection " + conn.getRealHashCode() + ".");
}
} else {
// Must wait / / the active collection is full without timeout. You can only wait for other threads to return the active connection
try {
if (!countedWait) {
state.hadToWaitCount++;
countedWait = true;
}
if (log.isDebugEnabled()) {
log.debug("Waiting as long as " + poolTimeToWait + " milliseconds for connection.");
}
long wt = System.currentTimeMillis();
state.wait(poolTimeToWait); // Wait until it times out or is awakened by another thread (see PooledDataSource#pushConnection). Then enter the next while loop
state.accumulatedWaitTime += System.currentTimeMillis() - wt;
} catch (InterruptedException e) {
break;
}
}
}
}
if (conn != null) { // After getting the connection in various ways, you need to check whether the connection is valid
// ping to server and check the connection is valid or not
if (conn.isValid()) {
if (!conn.getRealConnection().getAutoCommit()) {
conn.getRealConnection().rollback();
}
conn.setConnectionTypeCode(assembleConnectionTypeCode(dataSource.getUrl(), username, password)); // Set the hash value of the connection ID: url+username+password string
conn.setCheckoutTimestamp(System.currentTimeMillis()); // Set the check-out time. Note that here is the timestamp taken from the database connection pool! Not the time to establish a connection with the database!
conn.setLastUsedTimestamp(System.currentTimeMillis()); // Set last use time
state.activeConnections.add(conn); // Join the active set (1. Move the original connection object from the idle set to the active set; 2. Take out the timeout connection from the active set and put it back into the active set)
state.requestCount++;
state.accumulatedRequestTime += System.currentTimeMillis() - t;
} else { // If the connection is invalid, enter the next cycle to obtain the connection again, or throw an exception
if (log.isDebugEnabled()) {
log.debug("A bad connection (" + conn.getRealHashCode() + ") was returned from the pool, getting another connection.");
}
state.badConnectionCount++;
localBadConnectionCount++;
conn = null;
if (localBadConnectionCount > (poolMaximumIdleConnections + poolMaximumLocalBadConnectionTolerance)) { // If the number of cycles is greater than (the number of free connections + the tolerance threshold of bad connections), throw an exception and no longer cycle
if (log.isDebugEnabled()) {
log.debug("PooledDataSource: Could not get a good connection to the database.");
}
throw new SQLException("PooledDataSource: Could not get a good connection to the database.");
}
}
}
}
}
if (conn == null) {
if (log.isDebugEnabled()) {
log.debug("PooledDataSource: Unknown severe error condition. The connection pool returned a null connection.");
}
throw new SQLException("PooledDataSource: Unknown severe error condition. The connection pool returned a null connection.");
}
return conn;
}
In the process of checking out the connection, PoolState will be used to record some total time consumption.
org.apache.ibatis.datasource.pooled.PoolState
protected long requestCount = 0; // Number of requests protected long accumulatedRequestTime = 0; // Total request time protected long accumulatedCheckoutTime = 0; // Total check-out time (removing connections from the pool is called check-out) protected long claimedOverdueConnectionCount = 0; // Number of connections declared expired protected long accumulatedCheckoutTimeOfOverdueConnections = 0; // Total number of expired connections protected long accumulatedWaitTime = 0; // Total waiting time protected long hadToWaitCount = 0; // Number of times to wait protected long badConnectionCount = 0; // Bad connections
In the PooledDataSource object, the capacity of the free connection collection, the active connection collection, and some maximum time limits are set.
org.apache.ibatis.datasource.pooled.PooledDataSource
protected int poolMaximumActiveConnections = 10; // Number of active (in use) connections that can exist at any time protected int poolMaximumIdleConnections = 5; // The number of idle connections that may exist at any time protected int poolMaximumCheckoutTime = 20000; // The time that connections in the pool were checked out before being forced back. Default: 20000 milliseconds (i.e. 20 seconds) protected int poolTimeToWait = 20000; // This is an underlying setting. If it takes a long time to obtain a connection, it will print a status log to the connection pool and try to obtain a connection again (to avoid continuous failure and no log printing in case of misconfiguration) protected int poolMaximumLocalBadConnectionTolerance = 3; // This is a low-level setting for bad connection tolerance, which applies to every thread trying to get a connection from the cache pool. If the thread gets a bad connection, the data source allows the thread to try to get a new connection again, but the number of retries should not exceed the sum of poolMaximumIdleConnections and poolMaximumLocalBadConnectionTolerance
PooledDataSource#pushConnection
Return the connection to the database connection pool.
Code flow:
- Gets the PoolState object lock.
- Remove PooledConnection from the active connection collection and check if the connection is valid.
- If the connection is valid, judge whether the idle collection is full:
3.1 if the free set is not full, encapsulate the original connection as a new PooledConnection object and add it to the free set.
3.2 if the free set is full, close the connection and reuse it no longer.
You can see that each time you return the database connection, you actually return the original com mysql. cj. jdbc. The connectionimpl connection object, while the proxy object generated by PooledConnection is used up and lost, and is set to the invalid state to avoid affecting other threads during reuse.
protected void pushConnection(PooledConnection conn) throws SQLException {
synchronized (state) {
state.activeConnections.remove(conn); // Remove from active connection collection
if (conn.isValid()) { // Verify whether the connection is valid. If it is valid, go to the next step
if (state.idleConnections.size() < poolMaximumIdleConnections && conn.getConnectionTypeCode() == expectedConnectionTypeCode) { // If the free connection set is not full and the connection ID is consistent (url+username+password), you need to join the free connection set
state.accumulatedCheckoutTime += conn.getCheckoutTime(); // Accumulate the total check-out time (record the total time of connections from out of the pool to in the pool) (when valid connections are taken out of the pool, the check-out time stamp will be recorded)
if (!conn.getRealConnection().getAutoCommit()) {
conn.getRealConnection().rollback(); // Roll back the previous transaction to avoid affecting the next use
}
PooledConnection newConn = new PooledConnection(conn.getRealConnection(), this); // Generate a new PooledConnection object for the original connection
state.idleConnections.add(newConn); // Join the free connection set (note that this is not to move the old PooledConnection from the active set to the free set)
newConn.setCreatedTimestamp(conn.getCreatedTimestamp());
newConn.setLastUsedTimestamp(conn.getLastUsedTimestamp());
conn.invalidate(); // Set the old PooledConnection object as invalid, because the user can get this instance directly to avoid using this instance to operate the database in the future
if (log.isDebugEnabled()) {
log.debug("Returned connection " + newConn.getRealHashCode() + " to pool.");
}
state.notifyAll(); // Wake up the thread waiting to get the database connection. See PooledDataSource#popConnection. After being awakened, only one thread will obtain the object lock.
} else { // If the free connection collection is full or the connection ID is inconsistent, close the connection
state.accumulatedCheckoutTime += conn.getCheckoutTime();
if (!conn.getRealConnection().getAutoCommit()) {
conn.getRealConnection().rollback();
}
conn.getRealConnection().close(); // Close the connection and no longer reuse
if (log.isDebugEnabled()) {
log.debug("Closed connection " + conn.getRealHashCode() + ".");
}
conn.invalidate();
}
} else { // Invalid connection, cumulative count
if (log.isDebugEnabled()) {
log.debug("A bad connection (" + conn.getRealHashCode() + ") attempted to return to the pool, discarding connection.");
}
state.badConnectionCount++;
}
}
}
Usage process of PooledDataSource
Data source configuration resolution
Resolve mybatis config. Using SqlSessionFactoryBuilder When the XML configuration file is, the environment tag will be parsed.
The call chain is as follows:
org.apache.ibatis.session.SqlSessionFactoryBuilder#build(Reader, String, java.util.Properties)
org.apache.ibatis.builder.xml.XMLConfigBuilder#parse()
org.apache.ibatis.builder.xml.XMLConfigBuilder#parseConfiguration(org.apache.ibatis.parsing.XNode)
private void environmentsElement(XNode context) throws Exception {
if (context != null) {
if (environment == null) {
environment = context.getStringAttribute("default");
}
for (XNode child : context.getChildren()) {
String id = child.getStringAttribute("id");
if (isSpecifiedEnvironment(id)) {
TransactionFactory txFactory = transactionManagerElement(child.evalNode("transactionManager")); // Instantiate transaction factory
DataSourceFactory dsFactory = dataSourceElement(child.evalNode("dataSource")); // Instantiate database connection pool factory
DataSource dataSource = dsFactory.getDataSource(); // From the database connection pool factory, get the data source object. An environment tag has only one data source!
Environment.Builder environmentBuilder = new Environment.Builder(id)
.transactionFactory(txFactory)
.dataSource(dataSource);
configuration.setEnvironment(environmentBuilder.build()); // Register the transaction factory and data source object into the Configuration object
break;
}
}
}
}
Data source initialization
Since < datasource type = "pooled" >, the PooledDataSourceFactory object is obtained by XML parsing.
org.apache.ibatis.builder.xml.XMLConfigBuilder#dataSourceElement
private DataSourceFactory dataSourceElement(XNode context) throws Exception {
if (context != null) {
String type = context.getStringAttribute("type"); // eg. "POOLED"
Properties props = context.getChildrenAsProperties();
DataSourceFactory factory = (DataSourceFactory) resolveClass(type).getDeclaredConstructor().newInstance();
factory.setProperties(props); // Write the database connection information in the configuration file to the DataSource property in the DataSourceFactory
return factory;
}
throw new BuilderException("Environment declaration requires a DataSourceFactory.");
}
The PooledDataSourceFactory class is as follows. A PooledDataSource object will be created in the constructor:
public class PooledDataSourceFactory extends UnpooledDataSourceFactory {
public PooledDataSourceFactory() {
this.dataSource = new PooledDataSource();
}
}
PooledDataSourceFactory inheritance system:
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-vDPrZgl2-1629363020508)(./images/1629342636942.png)]
Open session
When a SqlSession session is opened, the Transaction object Transaction is created from the Transaction factory and the DataSource object is passed to it.
SqlSession sqlSession = sqlSessionFactory.openSession();
org.apache.ibatis.session.defaults.DefaultSqlSessionFactory#openSession
org.apache.ibatis.session.defaults.DefaultSqlSessionFactory#openSessionFromDataSource
private SqlSession openSessionFromDataSource(ExecutorType execType, TransactionIsolationLevel level, boolean autoCommit) {
Transaction tx = null;
try {
final Environment environment = configuration.getEnvironment();
final TransactionFactory transactionFactory = getTransactionFactoryFromEnvironment(environment);
tx = transactionFactory.newTransaction(environment.getDataSource(), level, autoCommit); // Instantiate the Transaction transaction object through the Transaction factory
final Executor executor = configuration.newExecutor(tx, execType); // Instantiate the Executor executor object, execute SQL through it, and support plug-in extension
return new DefaultSqlSession(configuration, executor, autoCommit);
} catch (Exception e) {
closeTransaction(tx); // may have fetched a connection so lets call close()
throw ExceptionFactory.wrapException("Error opening session. Cause: " + e, e);
} finally {
ErrorContext.instance().reset();
}
}
Since < transactionmanager type = "JDBC" > is configured and the JdbcTransactionFactory is obtained here, the JdbcTransaction transaction object is created.
org.apache.ibatis.transaction.jdbc.JdbcTransactionFactory#newTransaction
@Override
public Transaction newTransaction(DataSource ds, TransactionIsolationLevel level, boolean autoCommit) {
return new JdbcTransaction(ds, level, autoCommit);
}
Official description of transaction manager:
There are two types of transaction managers in MyBatis (that is, type="[JDBC|MANAGED]):
- JDBC – this configuration directly uses JDBC's commit and rollback facilities, which rely on connections obtained from data sources to manage transaction scopes.
- MANAGED – this configuration does little. It never commits or rolls back a connection, but lets the container manage the entire life cycle of the transaction (such as the context of the JEE application server).
If you are using Spring + MyBatis, there is no need to configure the transaction manager, because the Spring module will use its own manager to override the previous configuration.
Connection acquisition
When executing an SQL query, the database Connection object Connection is obtained from the PooledDataSource of the database Connection pool.
Student student01 = sqlSession.selectOne("selectByPrimaryKey", 1);
The call chain is as follows:
org.apache.ibatis.session.defaults.DefaultSqlSession#selectOne
org.apache.ibatis.session.defaults.DefaultSqlSession#selectList
org.apache.ibatis.executor.SimpleExecutor#doQuery
org.apache.ibatis.executor.SimpleExecutor#prepareStatement
org.apache.ibatis.executor.BaseExecutor#getConnection
protected Connection getConnection(Log statementLog) throws SQLException {
Connection connection = transaction.getConnection(); // From the transaction object, get the connection object
if (statementLog.isDebugEnabled()) {
return ConnectionLogger.newInstance(connection, statementLog, queryStack);
} else {
return connection;
}
}
The JdbcTransaction object is singleton in a session! Therefore, the same database connection is used in the same session.
org.apache.ibatis.transaction.jdbc.JdbcTransaction#getConnection
@Override
public Connection getConnection() throws SQLException {
if (connection == null) { // When it is empty, the connection is obtained from the connection pool
openConnection();
}
return connection;
}
In fact, the connection is obtained from the data source object PooledDataSource. What you get here is a proxy object.
org.apache.ibatis.transaction.jdbc.JdbcTransaction#openConnection
org.apache.ibatis.datasource.pooled.PooledDataSource#getConnection
@Override
public Connection getConnection() throws SQLException {
return popConnection(dataSource.getUsername(), dataSource.getPassword()).getProxyConnection();
}
Connection return
When you close a database session, return the database connection to the connection pool.
sqlSession.close();
The call chain is as follows:
org.apache.ibatis.session.defaults.DefaultSqlSession#close
org.apache.ibatis.executor.BaseExecutor#close
org.apache.ibatis.transaction.jdbc.JdbcTransaction#close
org.apache.ibatis.datasource.pooled.PooledConnection#invoke
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { // Proxy method
String methodName = method.getName();
if (CLOSE.equals(methodName)) { // Change the behavior of closing the connection to putting it back into the connection pool
dataSource.pushConnection(this);
return null;
}
try {
if (!Object.class.equals(method.getDeclaringClass())) {
// issue #579 toString() should never fail
// throw an SQLException instead of a Runtime
checkConnection(); // Check the connection before using it
}
return method.invoke(realConnection, args);
} catch (Throwable t) {
throw ExceptionUtil.unwrapThrowable(t);
}
}
summary
In MyBatis, PooledDataSource data source is used as the connection pool object, and the dynamic proxy mode is adopted. The original connection objects, such as com. Com, are reused in the connection pool mysql. cj. jdbc. Connectionimpl, and use org apache. ibatis. datasource. pooled. Pooledconnection is a proxy object generated by dynamic proxy. Its life cycle is within the scope of SqlSession. Ensure that after the session is closed, there will be no problem of multiple threads operating the same database connection.
When obtaining and returning threads from the connection pool, you need to obtain the synchronized lock, so as to achieve thread safety.
The usage process of PooledDataSource is as follows:
- Parsing mybatis config The PooledDataSource connection pool object is created when an XML configuration file is created.
- When you open a SqlSession database session, create a JdbcTransaction transaction transaction object and pass PooledDataSource to it.
- Use JdbcTransaction to maintain access to the database connection pool. In a session, only one connection will be obtained from the connection pool.
- When a SqlSession database session is closed, the connection is returned to the database connection pool.
Author: Sumkor
Link: https://blog.csdn.net/weixin_52801742/article/details/119805965