Preparation before use:
1) Configure the public redis.conf file unrelated to the port, and place it in the same directory as the tool
2) Configure the PORT related template redis-PORT.conf file, and put it in the same directory as the tool (PORT will be replaced with specific PORT number during deployment)
3) Configure the node file redis? Cluster.nodes, which is also in the same directory as the tool
The file format of Redis cluster.nodes is one node of Redis cluster for each line. It supports comment lines starting with "ා". Example format:
127.0.0.1 6381
127.0.0.1 6382
127.0.0.1 6383
127.0.0.1 6384
127.0.0.1 6385
127.0.0.1 6386
4) Create the directory where redis is installed (the build batch tool Mooon? SSH is completed, and deploy? Redis? Cluster.sh mainly uses this batch tool)
5) Other more detailed can directly see the source code, there are detailed instructions.
Set up to download https://github.com/eyjian/redis-tools/tree/master/deploy to a directory. When you run the deploy ﹣ redis ﹣ cluster.sh tool, it will prompt various preconditions, such as whether redis cli is available.
Source code (available from https://github.com/eyjian/redis-tools Download):
#!/bin/bash # Source code: https://github.com/eyjian/redis-tools # a tool to deploy a redis cluster # # Automatically deploy redis cluster tools, # Remote operation is enough without logging in to any machine in the Redis cluster. # # To batch create user redis as root: # export H=192.168.0.5,192.168.0.6,192.168.0.7,192.168.0.8,192.168.0.9 # export U=root # export P='root^1234' # mooon_ssh -c='groupadd redis; useradd -g redis -m redis; echo "redis:redis#1234"|chpasswd' # # Batch create the redis installation directory / data/redis-4.0.11, and set the owner as user redis, and the user group as redis example: # mooon_ssh -c='mkdir /data/redis-4.0.11;ln -s /data/redis-4.0.11 /data/redis;chown redis:redis /data/redis*' # # Process monitor.sh can be used to monitor the redis server process restart: # https://github.com/eyjian/libmooon/blob/master/shell/process_monitor.sh # Use example: # * * * * * /usr/local/bin/process_monitor.sh "/usr/local/redis/bin/redis-server 6379" "/usr/local/redis/bin/redis-server /usr/local/redis/conf/redis-6379.conf" # * * * * * /usr/local/bin/process_monitor.sh "/usr/local/redis/bin/redis-server 6380" "/usr/local/redis/bin/redis-server /usr/local/redis/conf/redis-6380.conf" # You can find the operation log of process_monitor.sh in the / tmp directory. When the process of the corresponding port is not available, it will restart the process of the corresponding port within 5 seconds. # # Operation parameters: # Parameter 1 SSH port # Parameter 2 installation user # Parameter 3 install user password # Parameter 4 installation directory # # Preconditions (you can use the batch tools Mooon? SSH and Mooon? Upload to complete): # 1) The installation user has created # 2) Setup user password has been set # 3) The installation directory has been created, and the owner of the directory is the installation user # 4) ruby is installed on the machine where the tool is executed, and the version number is no less than 2.0.0 # 5) redis-X.Y.Z.gem is installed on the machine executing this tool, and the version number is no less than redis-3.0.0.gem # # 6) The following executables exist in the same directory: # 6.1)redis-server # 6.2)redis-cli # 6.3)redis-check-rdb # 6.4)redis-check-aof # 6.5)redis-trib.rb # # 7) There are two configuration files in the same directory: # 7.1)redis.conf # 7.2)redis-PORT.conf # redis.conf is the public configuration file, # redis-PORT.conf is the profile template of the specified port, # At the same time, you need to replace the directory and port in the redis-PORT.conf file with INSTALLDIR and REDISPORT respectively. For example: # include INSTALLDIR/conf/redis.conf # pidfile INSTALLDIR/bin/redis-REDISPORT.pid # logfile INSTALLDIR/log/redis-REDISPORT.log # port REDISPORT # dbfilename dump-REDISPORT.rdb # dir INSTALLDIR/data/REDISPORT # # Where INSTALLDIR is replaced with the value of parameter 4, # And REDISPORT will use the port number in redis? Cluster.nodes instead # # The configuration file redis? Cluster.nodes defines the nodes where redis is installed # File format (in“#”Note at the beginning): # Each line consists of IP and port numbers separated by spaces, commas, semicolons, or TAB characters # # Dependence: # 1) Mooon SSH batch command tool for remote operation of multiple machines # 2) Mooon? Upload remote operation multiple machines batch upload tool # 3)https://raw.githubusercontent.com/eyjian/libmooon # 4) libmooon also relies on libssh2 (http://www.libssh2.org/) BASEDIR=$(dirname $(readlink -f $0)) REDIS_CLUSTER_NODES=$BASEDIR/redis_cluster.nodes # Batch command tool MOOON_SSH=mooon_ssh # Bulk upload tool MOOON_UPLOAD=mooon_upload # Create redis cluster tool REDIS_TRIB=$BASEDIR/redis-trib.rb # redis-server REDIS_SERVER=$BASEDIR/redis-server # redis-cli REDIS_CLI=$BASEDIR/redis-cli # redis-check-aof REDIS_CHECK_AOF=$BASEDIR/redis-check-aof # redis-check-rdb REDIS_CHECK_RDB=$BASEDIR/redis-check-rdb # redis.conf REDIS_CONF=$BASEDIR/redis.conf # redis-PORT.conf REDIS_PORT_CONF=$BASEDIR/redis-PORT.conf # global variable # Total nodes of the redis cluster num_nodes=0 # All IP arrays that make up the redis cluster redis_node_ip_array=() # Array of all nodes that make up the redis cluster (IP+port constructs a redis node) redis_node_array=() # usage function usage() { echo -e "\033[1;33mUsage\033[m: `basename $0` \033[0;32;32mssh-port\033[m install-user \033[0;32;32minstall-user-password\033[m install-dir" echo -e "\033[1;33mExample\033[m: `basename $0` \033[0;32;32m22\033[m redis \033[0;32;32mredis^1234\033[m /usr/local/redis-4.0.11" } # Five parameters need to be specified if test $# -ne 4; then usage echo "" exit 1 fi ssh_port="$1" install_user="$2" install_user_password="$3" install_dir="$4" echo -e "[ssh port] \033[1;33m$ssh_port\033[m" echo -e "[install user] \033[1;33m$install_user\033[m" echo -e "[install directory] \033[1;33m$install_dir\033[m" echo "" # Check if ruby is available which ruby > /dev/null 2>&1 if test $? -eq 0; then echo -e "Checking ruby OK" else echo -e "ruby \033[0;32;31mnot exists or not executable\033[m" echo "https://www.ruby-lang.org" echo -e "Exit now\n" exit 1 fi # Check if gem is available which gem > /dev/null 2>&1 if test $? -eq 0; then echo -e "Checking gem OK" else echo -e "gem \033[0;32;31mnot exists or not executable\033[m" echo "https://rubygems.org/pages/download" echo -e "Exit now\n" exit 1 fi # Check whether Mooon? SSH is available which "$MOOON_SSH" > /dev/null 2>&1 if test $? -eq 0; then echo -e "Checking $MOOON_SSH OK" else echo -e "$MOOON_SSH \033[0;32;31mnot exists or not executable\033[m" echo "There are two versions: C++ and GO:" echo "https://github.com/eyjian/libmooon/releases" echo "https://raw.githubusercontent.com/eyjian/libmooon/master/tools/mooon_ssh.cpp" echo "https://raw.githubusercontent.com/eyjian/libmooon/master/tools/mooon_ssh.go" echo -e "Exit now\n" exit 1 fi # Check whether Mooon? Upload is available which "$MOOON_UPLOAD" > /dev/null 2>&1 if test $? -eq 0; then echo -e "Checking $MOOON_UPLOAD OK" else echo -e "$MOOON_UPLOAD \033[0;32;31mnot exists or not executable\033[m" echo "There are two versions: C++ and GO:" echo "https://github.com/eyjian/libmooon/releases" echo "https://raw.githubusercontent.com/eyjian/libmooon/master/tools/mooon_upload.cpp" echo "https://raw.githubusercontent.com/eyjian/libmooon/master/tools/mooon_upload.go" echo -e "Exit now\n" exit 1 fi # Check whether redis-trib.rb is available which "$REDIS_TRIB" > /dev/null 2>&1 if test $? -eq 0; then echo -e "Checking $REDIS_TRIB OK" else echo -e "$REDIS_TRIB \033[0;32;31mnot exists or not executable\033[m" echo -e "Exit now\n" exit 1 fi # Check whether redis server is available which "$REDIS_SERVER" > /dev/null 2>&1 if test $? -eq 0; then echo -e "Checking $REDIS_SERVER OK" else echo -e "$REDIS_SERVER \033[0;32;31mnot exists or not executable\033[m" echo -e "Exit now\n" exit 1 fi # Check whether redis cli is available which "$REDIS_CLI" > /dev/null 2>&1 if test $? -eq 0; then echo -e "Checking $REDIS_CLI OK" else echo -e "$REDIS_CLI \033[0;32;31mnot exists or not executable\033[m" echo -e "Exit now\n" exit 1 fi # Check whether redis check AOF is available which "$REDIS_CHECK_AOF" > /dev/null 2>&1 if test $? -eq 0; then echo -e "Checking $REDIS_CHECK_AOF OK" else echo -e "$REDIS_CHECK_AOF \033[0;32;31mnot exists or not executable\033[m" echo -e "Exit now\n" exit 1 fi # Check whether redis check RDB is available which "$REDIS_CHECK_RDB" > /dev/null 2>&1 if test $? -eq 0; then echo -e "Checking $REDIS_CHECK_RDB OK" else echo -e "$REDIS_CHECK_RDB \033[0;32;31mnot exists or not executable\033[m" echo -e "Exit now\n" exit 1 fi # Check whether redis.conf is available if test -r "$REDIS_CONF"; then echo -e "Checking $REDIS_CONF OK" else echo -e "$REDIS_CONF \033[0;32;31mnot exists or not readable\033[m" echo -e "Exit now\n" exit 1 fi # Check whether redis-PORT.conf is available if test -r "$REDIS_PORT_CONF"; then echo -e "Checking $REDIS_PORT_CONF OK" else echo -e "$REDIS_PORT_CONF \033[0;32;31mnot exists or not readable\033[m" echo -e "Exit now\n" exit 1 fi # Resolve the redis? Cluster.nodes file, # All the nodes that make up the redis cluster are obtained. function parse_redis_cluster_nodes() { redis_nodes_str= redis_nodes_ip_str= while read line do # Remove leading and trailing spaces line=`echo "$line" | xargs` if test -z "$line" -o "$line" = "#"; then continue fi # Skip annotations begin_char=${line:0:1} if test "$begin_char" = "#"; then continue fi # Get IP and port eval $(echo "$line" | awk -F[\ \:,\;\t]+ '{ printf("ip=%s\nport=%s\n",$1,$2); }') # Both IP and port must have if test ! -z "$ip" -a ! -z "$port"; then if test -z "$redis_nodes_ip_str"; then redis_nodes_ip_str=$ip else redis_nodes_ip_str="$redis_nodes_ip_str,$ip" fi if test -z "$redis_nodes_str"; then redis_nodes_str="$ip:$port" else redis_nodes_str="$redis_nodes_str,$ip:$port" fi fi done < $REDIS_CLUSTER_NODES if test -z "$redis_nodes_ip_str"; then num_nodes=0 else # Get the IP array redis? Node? IP? Array redis_node_ip_array=`echo "$redis_nodes_ip_str" | tr ',' '\n' | sort | uniq` # Get the node array redis_node_array=`echo "$redis_nodes_str" | tr ',' '\n' | sort | uniq` for redis_node in ${redis_node_array[@]}; do num_nodes=$((++num_nodes)) echo "$redis_node" done fi } # check redis_cluster.nodes if test ! -r $REDIS_CLUSTER_NODES; then echo -e "File $REDIS_CLUSTER_NODES \033[0;32;31mnot exits\033[m" echo "" echo -e "\033[0;32;32mFile format\033[m (columns delimited by space, tab, comma, semicolon or colon):" echo "IP1 port1" echo "IP2 port2" echo "" echo -e "\033[0;32;32mExample\033[m:" echo "127.0.0.1 6381" echo "127.0.0.1 6382" echo "127.0.0.1 6383" echo "127.0.0.1 6384" echo "127.0.0.1 6385" echo "127.0.0.1 6386" echo -e "Exit now\n" exit 1 else echo -e "\033[0;32;32m" parse_redis_cluster_nodes echo -e "\033[m" if test $num_nodes -lt 1; then echo -e "Checking $REDIS_CLUSTER_NODES \033[0;32;32mfailed\033[m: no any node" echo -e "Exit now\n" exit 1 else echo -e "Checking $REDIS_CLUSTER_NODES OK, the number of nodes is \033[1;33m${num_nodes}\033[m" fi fi # Continue after confirmation while true do # At least six nodes are needed to form a redis cluster if test $num_nodes -lt 6; then echo -e "\033[0;32;32mAt least 6 nodes are required to create a redis cluster\033[m" fi # Prompt whether to continue echo -en "Are you sure to continue? [\033[1;33myes\033[m/\033[1;33mno\033[m]" read -r -p " " input if test "$input" = "no"; then echo -e "Exit now\n" exit 1 elif test "$input" = "yes"; then echo "Starting to install ..." echo "" break fi done # Do you want to empty the installation directory before installing? clear_install_directory= while true do echo -en "Clear install directory? [\033[1;33myes\033[m/\033[1;33mno\033[m]" read -r -p " " clear_install_directory if test "$clear_install_directory" = "no"; then echo "" break elif test "$clear_install_directory" = "yes"; then echo "" break fi done # Install public, including executable files and public configuration files function install_common() { redis_ip="$1" # Check whether the installation directory exists and has read-write permission echo "$MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c=\"test -d $install_dir && test -r $install_dir && test -w $install_dir && test -x $install_dir\"" $MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c="test -d $install_dir && test -r $install_dir && test -w $install_dir && test -x $install_dir" if test $? -ne 0; then echo "" echo -e "Directory $install_dir \033[1;33mnot exists or no (rwx) permission\033[m" echo -e "Exit now\n" exit 1 fi # Empty installation directory if test "$clear_install_directory" = "yes"; then echo "" echo "$MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c=\"killall -q -w -u $install_user redis-server\"" $MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c="killall -q -w -u $install_user redis-server" echo "$MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c=\"rm -fr $install_dir/*\"" $MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c="rm -fr $install_dir/*" if test $? -ne 0; then echo -e "Exit now\n" exit 1 fi fi # create directory echo "" echo "$MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c=\"cd $install_dir;mkdir -p bin conf log data\"" $MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c="cd $install_dir;mkdir -p bin conf log data" if test $? -ne 0; then echo -e "Exit now\n" exit 1 fi # upload configuration files echo "" echo "$MOOON_UPLOAD -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -s=redis.conf -d=$install_dir/conf" $MOOON_UPLOAD -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -s=redis.conf -d=$install_dir/conf if test $? -ne 0; then echo -e "Exit now\n" exit 1 fi # upload executable files echo "" echo "$MOOON_UPLOAD -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -s=redis-server,redis-cli,redis-check-aof,redis-check-rdb -d=$install_dir/bin" $MOOON_UPLOAD -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -s=redis-server,redis-cli,redis-check-aof,redis-check-rdb,redis-trib.rb -d=$install_dir/bin if test $? -ne 0; then echo -e "Exit now\n" exit 1 fi } # Install node profile function install_node_conf() { redis_ip="$1" redis_port="$2" # Generate node profile cp redis-PORT.conf redis-$redis_port.conf sed -i "s|INSTALLDIR|$install_dir|g;s|REDISPORT|$redis_port|g" redis-$redis_port.conf # create data directory for the given node echo "" echo "$MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c=\"cd $install_dir;mkdir -p data/$redis_port\"" $MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c="cd $install_dir;mkdir -p data/$redis_port" if test $? -ne 0; then rm -f redis-$redis_port.conf echo -e "Exit now\n" exit 1 fi # upload configuration files echo "" echo "$MOOON_UPLOAD -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -s=redis-$redis_port.conf -d=$install_dir/conf" $MOOON_UPLOAD -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -s=redis-$redis_port.conf -d=$install_dir/conf if test $? -ne 0; then rm -f redis-$redis_port.conf echo -e "Exit now\n" exit 1 fi rm -f redis-$redis_port.conf } function start_redis_node() { redis_ip="$1" redis_port="$2" # start redis instance echo "" echo "$MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c=\"$install_dir/bin/redis-server $install_dir/conf/redis-$redis_port.conf\"" $MOOON_SSH -h=$redis_ip -P=$ssh_port -u=$install_user -p=$install_user_password -c="nohup $install_dir/bin/redis-server $install_dir/conf/redis-$redis_port.conf > /dev/null 2>&1 &" if test $? -ne 0; then echo -e "Exit now\n" exit 1 fi } # Install public, including executable files and public configuration files echo "" echo -e "\033[1;33m================================\033[m" for redis_node_ip in $redis_node_ip_array; do echo -e "[\033[1;33m$redis_node_ip\033[m] Installing common ..." install_common $redis_node_ip done # Install node profile echo "" echo -e "\033[1;33m================================\033[m" for redis_node in ${redis_node_array[@]}; do node_ip= node_port= eval $(echo "$redis_node" | awk -F[\ \:,\;\t]+ '{ printf("node_ip=%s\nnode_port=%s\n",$1,$2); }') if test -z "$node_ip" -o -z "$node_port"; then continue fi echo -e "[\033[1;33m$node_ip:$node_port\033[m] Installing node ..." install_node_conf $node_ip $node_port done # Continue after confirmation echo "" echo -e "\033[1;33m================================\033[m" while true do echo -en "Start redis? [\033[1;33myes\033[m/\033[1;33mno\033[m]" read -r -p " " input if test "$input" = "no"; then echo "" exit 1 elif test "$input" = "yes"; then echo "Starting to start redis ..." echo "" break fi done # start redis instance for redis_node in ${redis_node_array[@]}; do eval $(echo "$redis_node" | awk -F[\ \:,\;\t]+ '{ printf("node_ip=%s\nnode_port=%s\n",$1,$2); }') if test -z "$node_ip" -o -z "$node_port"; then continue fi echo -e "[\033[1;33m$node_ip:$node_port\033[m] Starting node ..." start_redis_node $node_ip $node_port done echo "" echo -e "\033[1;33m================================\033[m" echo "Number of nodes: $num_nodes" if test $num_nodes -lt 6; then echo "Number of nodes less than 6, can not create redis cluster" echo -e "Exit now\n" exit 1 else redis_nodes_str=`echo "$redis_nodes_str" | tr ',' ' '` # Continue after confirmation echo "" while true do echo -en "Create redis cluster? [\033[1;33myes\033[m/\033[1;33mno\033[m]" read -r -p " " input if test "$input" = "no"; then echo "" exit 1 elif test "$input" = "yes"; then echo "Starting to create redis cluster with $redis_nodes_str ... ..." echo "" break fi done # Create a redis cluster # redis-trib.rb create --replicas 1 $REDIS_TRIB create --replicas 1 $redis_nodes_str echo -e "Exit now\n" exit 0 fi