I preface
Mycat exposes services in the form of a Server, and its main configuration class is Server XML, this article mainly focuses on some small details, mainly including:

Server. How to load XML
Configuration and role in Server
Flow of Server in connection request
II Server.xml configuration
2.1 Server.xml configuration details
Server.xml is mainly loaded in XML schemaloader, server The XML file is mainly divided into the following parts

User management

<user name="user">

<property name="password">user</property>
// User accessible schemas can be separated by commas
<property name="schemas">db001</property>
<property name="readOnly">true</property>
<property name="defaultSchema">db001</property>

</user>
Copy code
configuration management

The configuration file is pulled directly from the source code, which is also very clear. If you are interested, you can pull the source code to have a look

<system>

   <property name="nonePasswordLogin">0</property> <!-- 0 For login with password, 1 for login without password ,The default is 0. If it is set to 1, you need to specify the default account-->
   <property name="ignoreUnknownCommand">0</property><!-- 0 An unrealized message is encountered(Unknown command:),An error will be reported. 1 means to ignore the message and return ok Message.

When some mysql clients are logged in, they will continue to send login messages, and mycat will report an error. This setting can bypass this error -- >

   <property name="useHandshakeV10">1</property>
   <property name="removeGraveAccent">1</property>
   <property name="useSqlStat">0</property>  <!-- 1 To enable real-time statistics, 0 to disable -->
   <property name="useGlobleTableCheck">0</property>  <!-- 1 To enable all overtime consistency detection, and 0 to close -->
   <property name="sqlExecuteTimeout">300</property>  <!-- SQL Execution timeout unit:second-->
   <property name="sequenceHandlerType">1</property>
   <!--<property name="sequnceHandlerPattern">(?:(\s*next\s+value\s+for\s*MYCATSEQ_(\w+))(,|)|\s)*)+</property>
   INSERT INTO `travelrecord` (`id`,user_id) VALUES ('next value for MYCATSEQ_GLOBAL',"xxx");
   -->
   <!--Must have MYCATSEQ_perhaps mycatseq_Enter the sequence matching process. Note: MYCATSEQ_With spaces-->
   <property name="sequnceHandlerPattern">(?:(\s*next\s+value\s+for\s*MYCATSEQ_(\w+))(,|)|\s)*)+</property>
   <property name="subqueryRelationshipCheck">false</property> <!-- When there is an associated query in the sub query,Check whether there is a fragment field in the associated field .default false -->
   <property name="sequenceHanlderClass">io.mycat.route.sequence.handler.HttpIncrSequenceHandler</property>
   <!--  <property name="useCompression">1</property>--> <!--1 For on mysql Compression protocol-->
   <!--  <property name="fakeMySQLVersion">5.6.20</property>--> <!--Set simulated MySQL Version number-->
   <!-- <property name="processorBufferChunk">40960</property> -->
   <!--
   <property name="processors">1</property>
   <property name="processorExecutor">32</property>
    -->
   <!--Default to type 0: DirectByteBufferPool | type 1 ByteBufferArena | type 2 NettyBufferPool -->
   <property name="processorBufferPoolType">0</property>
   <!--The default is 65535 64 K be used for sql Maximum text length when parsing -->
   <!--<property name="maxStringLiteralLength">65535</property>-->
   <!--<property name="sequenceHandlerType">0</property>-->
   <!--<property name="backSocketNoDelay">1</property>-->
   <!--<property name="frontSocketNoDelay">1</property>-->
   <!--<property name="processorExecutor">16</property>-->
   <!--
       <property name="serverPort">8066</property> <property name="managerPort">9066</property>
       <property name="idleTimeout">300000</property> <property name="bindIp">0.0.0.0</property>
       <property name="dataNodeIdleCheckPeriod">300000</property> 5 * 60 * 1000L; //Connection idle check
       <property name="frontWriteQueueSize">4096</property> <property name="processors">32</property> -->
   <!--Distributed transaction switch: 0 means not filtering distributed transactions, 1 means filtering distributed transactions (if only global tables are involved in distributed transactions, then not filtering), and 2 means not filtering distributed transactions,However, distributed transaction logs are recorded-->
   <property name="handleDistributedTransactions">0</property>

   <!--
   off heap for merge/order/group/limit      1 On 0 off

-->

   <property name="useOffHeapForMerge">0</property>

   <!--
       Unit is m
   -->
   <property name="memoryPageSize">64k</property>

   <!--
       Unit is k
   -->
   <property name="spillsFileBufferSize">1k</property>

   <property name="useStreamOutput">0</property>

   <!--
       Unit is m
   -->
   <property name="systemReserveMemorySize">384m</property>


   <!--Whether to adopt zookeeper Coordinated switching  -->
   <property name="useZKSwitch">false</property>

   <!-- XA Recovery Log Log path -->
   <!--<property name="XARecoveryLogBaseDir">./</property>-->

   <!-- XA Recovery Log Log name -->
   <!--<property name="XARecoveryLogBaseName">tmlog</property>-->
   <!--If yes true Strictly observe the isolation level if necessary,Not just select Statement to switch connections in a transaction-->
   <property name="strictTxIsolation">false</property>
   <!--If 0,Involving multiple DataNode of catlet Tasks do not execute across threads-->
   <property name="parallExecute">0</property>

</system>
Copy code
Firewall configuration

<firewall>
<whitehost>

  <host host="1*7.0.0.*" user="root"/>

</whitehost>
<blacklist check="false">
</blacklist>
</firewall>
Copy code
2.2 Server.xml detail parsing
Server. There are several tags inside XML, mainly user, system and firewall

//User: used to define the user and permission of the logged in Mycat, which is finally mapped to UserConfig. In addition to the above, it also provides the following special attributes:

• benchmark: load balancing policy. 0 means unlimited connections
• Privileges config: table level addition, deletion, modification and query settings

//System tag: system configuration tag, which is finally mapped to SystemConfig object. In addition to the above, it mainly has the following configurations

• charset: configure the character set to be consistent with the database
• defaultSqlParser: Specifies the default parser
• processors: number of threads available to the system (default runtime. Getruntime() availableProcessors())
• processorBufferChunk: every time a Socket Direct Buffer is allocated, the size of bytes obtained will be affected here
• Processorbufferpool: calculation proportion of bufferpool
• Processorbufferlocalpercent: the proportion of threadlocalpool allocated Pool. The default is 100
• Sequncehandlertype: the type of MYCAT global sequence
• private long processorCheckPeriod: clean up NIOProcessor idle interval
• private long dataNodeIdleCheckPeriod: backend connection cleanup interval
• private long dataNodeHeartbeatPeriod: initiates heartbeat interval for backend reads and writes
• private int useOffHeapForMerge: whether to enable off heap for merge 1 - enabled, 0 - not enabled
• private int usingAIO = 0: enable AIO
• Private int packetheadersize = 4: MySQL protocol message length
• Private int maxpacketsize = 16 1024: the maximum size of data that can be carried
• private String memoryPageSize: page size, corresponding to the size of MemoryBlock, unit: M
• private long idleTimeout: the timeout length of the idle time of the connection
• private int txIsolation: the isolation level of the initial front-end connection transaction, corresponding to 1-4

Copy code
III Server. How to load XML
Server.xml is loaded in the ConfigInitializer. Finally, the bit Map is loaded and passed to MycatServer

//Read server xml
XMLConfigLoader configLoader = new XMLConfigLoader(schemaLoader);

//Corresponding schema xml -> dataHost
private final Map<String, DataHostConfig> dataHosts;
//Corresponding schema xml -> dataNode
private final Map<String, DataNodeConfig> dataNodes;
//Corresponding schema XML rules
private final Map<String, SchemaConfig> schemas;
//Corresponding System global configuration
private final SystemConfig system;
//Corresponding User specific Config
private final Map<String, UserConfig> users;
//Corresponding firewall configuration
private final FirewallConfig firewall;
//Corresponding slice configuration
private final ClusterConfig cluster;
Copy code
IV Configured usage scenarios
4.1 node 1: Server connection
When connecting through the tool, the init DB operation will be performed first, and the DB information will be obtained to obtain the connection

C- FrontendConnection # initDB
public void initDB(byte[] data) {

//S1: get data through MySQLMessage object
//init request parameters: \ u0006 \ u0000 \ u0000 \ u0000 \ u0002db001 - > DB001

//S2: check DB validity
db == null || !privileges.schemaExists(db)

//S3: verify whether the current user exists
privileges.userExists(user, host)

//S4: get Schemas
Set<String> schemas = privileges.getUserSchemas(user);
if (schemas == null || schemas.size() == 0 || schemas.contains(db)) {

  this.schema = db;
  // OkPacket.OK : 07 00 00 02 00 00 00 02 00 00 00
  write(writeToBuffer(OkPacket.OK, allocate()));

} else {

  String s = "Access denied for user '" + user + "' to database '" + db + "'";
  writeErrMessage(ErrorCode.ER_DBACCESS_DENIED_ERROR, s);

}
}

Copy code
Supplement 1: okpocket OK: what does 700010002000 mean? MySQL OK package return structure

An OK packet is sent from the server to the client, indicating the successful completion of the command. In MySQL 5.7.5, OK packets are also used to indicate EOF, and the EOF packet has been discarded.

The packages will contain the following data: packet header, affected rows, last inserted ID, status flag status, etc. there will be an opportunity to find a case to see in detail later

Supplement 2: privileges object

As you can see, FrontendPrivileges frequently appears in the above code. What is the function of this object?

FrontendPrivileges is a permission provider interface. It provides several common methods, such as:

schemaExists: check whether the schema exists
userExists: check whether the user exists and can use host to implement isolation policy
checkFirewallWhiteHostPolicy: check firewall policy
.... wait

It can be understood that FrontendPrivileges is for server The main implementation class of logical processing of XML data is MycatPrivileges

Its internal logic is also relatively simple, mainly dealing with MycatConfig, for example:

//Check whether the schema exists
public boolean schemaExists(String schema) {

MycatConfig conf = MycatServer.getInstance().getConfig();
return conf.getSchemas().containsKey(schema);

}

//Very good practice, directly return the instance object
public static final MycatServer getInstance() {

// private static final MycatServer INSTANCE = new MycatServer();
// The static object has completed the initialization of related config in the constructor
return INSTANCE;

}

Copy code
You can see that the internal configuration basically includes the common Config

! [uploading...] ()

4.2 node 2: configuration during query
The configuration will take effect in each phase. Here are some common scenarios:

Common usage scenarios of timeout

//Regularly check tasks and handle recycled resources:
C- MycatServer # processorCheck :
C- NIOProcessor # backendCheck
private void backendCheck() {

// S1: get timeout

long sqlTimeout = MycatServer.getInstance().getConfig().getSystem().getSqlExecuteTimeout() * 1000L;

//S2: iterate over all connections
Iterator<Entry<Long, BackendConnection>> it = backends.entrySet().iterator();

//S3: first delete the empty connection, and then close the connection with SQL execution timeout
if (c.isBorrowed() && c.getLastTime() < TimeUtil.currentTimeMillis() - sqlTimeout) {

  //.....

}

}

//PS: there are many usage scenarios, so I won't look at them one by one here

Copy code
From this case, it is obvious that the configuration is mainly processed by obtaining the configuration from MycatServer, and MycatServer is one of the core objects of the whole process. Let's take a detailed look in the next article

V Data reception of Server
When Mycat NIO processes the request, you can see a Byte [] continuous flow. What data does this Byte contain?

//In FrontendConnection # initDB, we can see the specific data of data. Take a connection as an example:

• \u0006\u0000\u0000\u0000\u0002: prefix to determine the specific type
• db001: specific Server Schema

//Supplement: judge the C- FrontendCommandHandler # handle by the first four types
public void handle(byte[] data) {

if (source.getLoadDataInfileHandler() != null && source.getLoadDataInfileHandler().isStartLoadData()) {
    MySQLMessage mm = new MySQLMessage(data);
    int packetLength = mm.readUB3();
    if (packetLength + 4 == data.length) {
        source.loadDataInfileData(data);
    }
    return;
}

// Can see
switch (data[4]) {
    // public static final byte COM_INIT_DB = 2;
    case MySQLPacket.COM_INIT_DB:
        commands.doInitDB();
        source.initDB(data);
        break;
    // public static final byte COM_QUERY = 3;    
    case MySQLPacket.COM_QUERY:
        commands.doQuery();
        source.query(data);
        break;
    case MySQLPacket.COM_PING:
        commands.doPing();
        source.ping();
        break;
    case MySQLPacket.COM_QUIT:
        commands.doQuit();
        source.close("quit cmd");
        break;
    //.....  Omit part type
    default:
        commands.doOther();
        MycatConfig config = MycatServer.getInstance().getConfig();
        if (config.getSystem().getIgnoreUnknownCommand() == 1) {
            LOGGER.warn("Unknown command:{}", data[4]);
            source.ping();
        } else {
            LOGGER.error("Unknown command:{}", new String(data));
            source.writeErrMessage(ErrorCode.ER_UNKNOWN_COM_ERROR,
                    "Unknown command");
        }
}

}
Copy code
Generally, a query is divided into two parts:

Initialize connection
Execute the current operation, taking Query as an example
! [uploading...] ()

It can be seen here that Byte[0] - Byte[4] is still a flag bit, and specific SQL will be passed in later Whether initDB or Query, a key step will be taken:

MySQLMessage mm = new MySQLMessage(data);
//Parsing flag bit
mm.position(5);
String db = mm.readString();

//When the label bit parsing is completed, the corresponding Handler will be called to complete the subsequent logic

• queryHandler.query(sql);

Copy code
summary
After a simple study of the entrance, reference to the documents and some supplements, you can slowly go deep into the whole system in the follow-up

last
If you think this article is a little helpful to you, give it a compliment. Or you can join my development exchange group: 1025263163 learn from each other, and we will have professional technical Q & A to solve doubts

If you think this article is useful to you, please click star: http://github.crmeb.net/u/defu esteem it a favor!

PHP learning manual: https://doc.crmeb.com
Technical exchange forum: https://q.crmeb.com