With the increasing business load, containerization and virtualization have become the necessary capabilities for all kinds of online applications. For distributed database, containerization is also a good way to improve rapid deployment and operation and maintenance efficiency.
We have re optimized the way of Docker deployment to help you get started with SequoiaDB cluster faster. This article will introduce the SequoiaDB distributed cluster rapid deployment based on Docker.
1. Cluster configuration
We will deploy a multi node, highly available SequoiaDB cluster in the middle of six containers, as follows:
(this article takes SequoiaDB v3.2.3 as an example)
The cluster includes a coordinator node, a directory node, and three data groups. Each data group has three replica data nodes and one MySQL instance node.
2. environment
OS : Ubuntu 18Docker Version : 18.09.7Docker Compose Version: 1.25.3Database Version : SequoiaDB 3.2.3MySql Client: Sequoiasql-mysqlCluster Deployment : 1 coordinator, 1 catalog, 3 data nodes, and 1 MySQL instance
2.1 Docker installation
2.1.1 Docker
For Docker installation on different platforms, users can refer to the Docker installation guide.
sudo apt-get install -y docker.io
2.1.2 Docker-compose
Docker compose is a tool for defining and deploying multi container docking applications defined in YAML files (i.e: docker compose. YAML).
sudo curl -L "https://github.com/docker/compose/releases/download/1.25.3/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-composesudo chmod +x /usr/local/bin/docker-composesudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
2.1.3 MySQL client
Install MySQL client (if not already installed)
sudo apt install mysql-client-core-5.7
3. SequoiaDB distributed cluster deployment
Define SequoiaDB cluster configuration "SequoiaDB? Containers / docker-compose.yaml". Users have two options to download the required software package.
Option 1: Clone repo address
(click [read original] at the end of the article to get the software package)
Option 2: Download manually (keep the same directory structure)
(click [read original] at the end of the article to get the software package)
3.1 deploy SequoiaDB cluster
The docker compose up command will:
- if it is not found locally, you can go to dockerhub to download the image
- create the required container
- start container
- initialize SequoiaDB cluster and MySQL instance
1) Deployment cluster
cd sequoiadb_containers (or to the docker compose directory. yaml file download) sudo docker compose up - D
2) Check the status of the cluster
$ sudo docker-compose psName Command State Ports--------------------------------------------------------------------------------------------sequoiadb_containers_catalog_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_coord_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data1_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data2_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data3_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_mysql_1 sh -c chmod +x /startup.sh ... Up 0.0.0.0:3310->3310/tcp
3) Wait for the cluster to be deployed and MySQL to connect to the coordinator node. The user can check the cluster status as follows.
#Check the coordinator log $sudo docker compose logs coord... Coord 〝 1 〝 deploy sequoiadb 〝 coord 〝 1 | create Catalog: Catalog: 11800 coord | 1 | create coord: coord: 11810 coord | 1 | create data: data1:11820 coord | create data: data2:11820 coord_1 | Create data: data3:11820 coord_1 | Create data: data1:11830 coord_1 | Create data: data2:11830 coord_1 | Create data: data3:11830 coord_1 | Create data: data1:11840 coord_1 | Create data: data2:11840 coord_1 | Create data: data3:11840 coord_1 | SDB Cluster successfully created.
#Check MySQL log $sudo docker compose logs MySQL... MySQL | > waiting for coord node to come up at (coord: 11810)... MySQL | > waiting for coord node to come up at (coord: 11810)... MySQL | > waiting for coord node to come up at (coord: 11810)... MySQL | >| Successfully reached COORD node at (coord:11810) mysql_1 | COMMAND: /init.sh --port=3310 --coord=coord:11810 mysql_1 | Creating SequoiaSQL instance: MySQLInstance mysql_1 | Modify configuration file and restart the instance: MySQLInstance mysql_1 | Restarting instance: MySQLInstance mysql_1 | Opening remote access to user root mysql_1 | Restarting instance: MySQLInstance mysql_1 | Instance MySQLInstance is created on port 3310, default user is root mysql_1 | Init command returned: 0
3.2 verify MySQL connection
$ mysql -h 127.0.0.1 -P 3310 -u root
Welcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 4Server version: 5.7.25 Source distributionCopyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql>
Verify SequoiaDB configuration:
mysql> show variables like 'sequoiadb%';+---------------------------------------+-------------+| Variable_name | Value |+---------------------------------------+-------------+| sequoiadb_bulk_insert_size | 2000 || sequoiadb_conn_addr | coord:11810 || sequoiadb_debug_log | OFF || sequoiadb_execute_only_in_mysql | OFF || sequoiadb_optimizer_select_count | ON || sequoiadb_password | || sequoiadb_replica_size | 1 || sequoiadb_selector_pushdown_threshold | 30 || sequoiadb_use_autocommit | ON || sequoiadb_use_bulk_insert | ON || sequoiadb_use_partition | ON || sequoiadb_user | |+---------------------------------------+-------------+12 rows in set (0.01 sec)mysql> show storage engines;+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+| Engine | Support | Comment | Transactions | XA | Savepoints |+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+| SequoiaDB | DEFAULT | SequoiaDB storage engine(Community). Plugin: eebcb4f, Driver: 3.2.3, BuildTime: | YES | NO | NO || MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO || PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO || BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO || MyISAM | YES | MyISAM storage engine | NO | NO | NO || CSV | YES | CSV storage engine | NO | NO | NO || ARCHIVE | YES | Archive storage engine | NO | NO | NO || InnoDB | YES | Supports transactions, row-level locking, and foreign keys | YES | YES | YES || FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL || MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+10 rows in set (0.00 sec)
3.3 test
You can use MySQL commands to create databases and tables:
mysql> create database sample;Query OK, 1 row affected (0.01 sec)mysql> use sample;Database changedmysql> create table t1 (c1 int);Query OK, 0 rows affected (0.47 sec)mysql> show table status;+------+-----------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+-------------+-------------+------------+-------------+----------+----------------+---------+| Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment |+------+-----------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+-------------+-------------+------------+-------------+----------+----------------+---------+| t1 | SequoiaDB | 10 | Fixed | 0 | 0 | 0 | 8796093022208 | 131072 | 0 | NULL | NULL | NULL | NULL | utf8mb4_bin | NULL | | |+------+-----------+---------+------------+------+----------------+-------------+-----------------+--------------+-----------+----------------+-------------+-------------+------------+-------------+----------+----------------+---------+1 row in set (0.03 sec)
4. Cluster management
4.1 stop/start cluster
After creating a SequoiaDB cluster using the up command, it can stop and start as follows.
Note:
"stop"The command does not delete the container, so no data is lost.
4.1.1 stop command
$ sudo docker-compose stopStopping sequoiadb_containers_mysql_1 ... doneStopping sequoiadb_containers_coord_1 ... doneStopping sequoiadb_containers_data2_1 ... doneStopping sequoiadb_containers_data1_1 ... doneStopping sequoiadb_containers_catalog_1 ... doneStopping sequoiadb_containers_data3_1 ... done$ sudo docker-compose psName Command State Ports------------------------------------------------------------------------------sequoiadb_containers_catalog_1 sh -c chmod +x /startup.sh ... Exit 137sequoiadb_containers_coord_1 sh -c chmod +x /startup.sh ... Exit 137sequoiadb_containers_data1_1 sh -c chmod +x /startup.sh ... Exit 137sequoiadb_containers_data2_1 sh -c chmod +x /startup.sh ... Exit 137sequoiadb_containers_data3_1 sh -c chmod +x /startup.sh ... Exit 137sequoiadb_containers_mysql_1 sh -c chmod +x /startup.sh ... Exit 137
4.1.2 start command
$ sudo docker-compose startStarting catalog ... doneStarting data1 ... doneStarting data2 ... doneStarting data3 ... doneStarting coord ... doneStarting mysql ... done$ sudo docker-compose psName Command State Ports--------------------------------------------------------------------------------------------sequoiadb_containers_catalog_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_coord_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data1_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data2_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data3_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_mysql_1 sh -c chmod +x /startup.sh ... Up 0.0.0.0:3310->3310/tcp# Verity mysql client is up$ sudo docker-compose logs mysqlAttaching to sequoiadb_containers_mysql_1mysql_1 | Starting service sequoiasql-mysql ...mysql_1 | ok. (PID: 38)...mysql_1 | MySql instance is started
4.2 down/up Cluster
4.2.1 down command
The down command stops and deletes the container for the SequoiaDB cluster deployment.
Note:
down The command should be executed with caution because it cannot be rolled back when the container is destroyed/Restore.
$ sudo docker-compose downStopping sequoiadb_containers_mysql_1 ... doneStopping sequoiadb_containers_coord_1 ... doneStopping sequoiadb_containers_data2_1 ... doneStopping sequoiadb_containers_data1_1 ... doneStopping sequoiadb_containers_catalog_1 ... doneStopping sequoiadb_containers_data3_1 ... doneRemoving sequoiadb_containers_mysql_1 ... doneRemoving sequoiadb_containers_coord_1 ... doneRemoving sequoiadb_containers_data2_1 ... doneRemoving sequoiadb_containers_data1_1 ... doneRemoving sequoiadb_containers_catalog_1 ... doneRemoving sequoiadb_containers_data3_1 ... doneRemoving network sequoiadb_containers_sequoiadb_net$ sudo docker-compose psName Command State Ports------------------------------
4.2.2 up command
The up command creates and starts the service (container, network) defined for sequoiadb cluster.
$ sudo docker-compose up -dCreating network "sequoiadb_containers_sequoiadb_net" with the default driverCreating sequoiadb_containers_data2_1 ... doneCreating sequoiadb_containers_data3_1 ... doneCreating sequoiadb_containers_catalog_1 ... doneCreating sequoiadb_containers_data1_1 ... doneCreating sequoiadb_containers_coord_1 ... doneCreating sequoiadb_containers_mysql_1 ... done$ sudo docker-compose psName Command State Ports--------------------------------------------------------------------------------------------sequoiadb_containers_catalog_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_coord_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data1_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data2_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_data3_1 sh -c chmod +x /startup.sh ... Upsequoiadb_containers_mysql_1 sh -c chmod +x /startup.sh ... Up 0.0.0.0:3310->3310/tcp# Verify MySQL connection$ mysql -h 127.0.0.1 -P 3310 -u rootWelcome to the MySQL monitor. Commands end with ; or \g.Your MySQL connection id is 2Server version: 5.7.25 Source distributionCopyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates. Other names may be trademarks of their respectiveowners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> show storage engines;+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+| Engine | Support | Comment | Transactions | XA | Savepoints |+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+| SequoiaDB | DEFAULT | SequoiaDB storage engine(Community). Plugin: eebcb4f, Driver: 3.2.3, BuildTime: | YES | NO | NO || MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO || PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO || BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO || MyISAM | YES | MyISAM storage engine | NO | NO | NO || CSV | YES | CSV storage engine | NO | NO | NO || ARCHIVE | YES | Archive storage engine | NO | NO | NO || InnoDB | YES | Supports transactions, row-level locking, and foreign keys | YES | YES | YES || FEDERATED | NO | Federated MySQL storage engine | NULL | NULL | NULL || MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |+--------------------+---------+----------------------------------------------------------------------------------+--------------+------+------------+
I don't know if you have already done it?
SequoiaDB will continue to launch a series of technical tutorials to help you get started with the distributed database faster and understand the principle of the distributed database better. Please look forward!
