Greenplum 6.17 cluster construction
1, Cluster planning
| Host | ip | function |
|---|---|---|
| master | 192.168.2.20 | master |
| segment1 | 192.168.2.21 | segment |
| segment2 | 192.168.2.22 | segment |
The password is 1
2, Environmental description
| name | explain |
|---|---|
| operating system | Centos7 |
| Database installation package | open-source-greenplum-db-6.17.2-rhel7-x86_64.rpm |
3, Preparation before installation
1. Other tools required for virtual machine minimum installation
sudo yum install -y epel-release sudo yum install -y psmisc nc net-tools rsync vim lrzsz ntp libzstd openssl-static tree iotop git
2. Modify system files
Modify the system resource limit, / etc / security / limits Add the following information to the conf file
* soft nofile 65536 * hard nofile 65536 * soft nproc 131072 * hard nproc 131072
Modify the system kernel information, / etc / sysctl Add the following information to the conf file:
# kernel.shmall calculates echo $(expr $(getconf _phys _pages) / 2 with the following command kernel.shmall = 357475 # kernel.shmmax calculates echo $(expr $(getconf _PHYS_PAGES) / 2 \* $(getconf PAGE_SIZE)) with the following command kernel.shmmax = 1464217600 kernel.shmmni = 4096 vm.overcommit_memory = 2 vm.overcommit_ratio = 95 kernel.sem = 500 2048000 200 4096 kernel.sysrq = 1 kernel.core_uses_pid = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 kernel.msgmni = 2048 net.ipv4.tcp_syncookies = 1 net.ipv4.conf.default.accept_source_route = 0 net.ipv4.tcp_max_syn_backlog = 4096 net.ipv4.conf.all.arp_filter = 1 net.core.netdev_max_backlog = 10000 net.core.rmem_max = 2097152 net.core.wmem_max = 2097152 vm.swappiness = 10 vm.zone_reclaim_mode = 0 vm.dirty_expire_centisecs = 500 vm.dirty_writeback_centisecs = 100 vm.dirty_background_ratio = 3 vm.dirty_ratio = 10
After modification, execute the command to take effect immediately
sysctl -p
Modify / etc / security / limits * * nproc. Under D / The conf file is (the file name may be 20-nproc.conf or 90-nproc.conf):
* soft nproc 131072
3. Turn off the firewall
Turn off the firewall
systemctl stop firewalld
Turn off firewall self startup
systemctl disable firewalld
View firewall status
systemctl status firewalld
systemctl is-enabled firewalld
Close selinux and modify / etc/sysconfig/selinux:
SELINUX=disabled
Configuration effective
setenforce 0
4. Configure host
Modify / etc/hostname
5. Add user group
Add gpadmin user group and grant corresponding permissions
# add group groupadd -g 530 gpadmin # Add user useradd -g 530 -u 530 -m -d /home/gpadmin -s /bin/bash gpadmin # Change owner chown -R gpadmin:gpadmin /home/gpadmin # Change Password passwd gpadmin
Assign sudo execution permission to users
visudo In“ root ALL=(ALL) ALL"Add later gpadmin ALL=(ALL) ALL
==============================================
Clone the other two virtual machines and modify the ip and host
modify host Corresponding file: /etc/hostname modify ip Corresponding file: /etc/sysconfig/network-scripts(Generally ifcfg-ens33)
===============================================
4, Pre installation configuration
1. Add host mapping
Add cluster host and ip in / etc/hosts
192.168.2.20 master 192.168.2.21 segment1 192.168.2.22 segment2
2. ssh password free login (gpadmin user)
Generate secret key (three nodes)
ssh-keygen -t rsa
The segment node sends the public key to the master node
# segment1 public key replication cp ~/.ssh/id_rsa.pub ~/.ssh/segment1_id_rsa.pub # segment2 public key replication cp ~/.ssh/id_rsa.pub ~/.ssh/segment1_id_rsa.pub # segment1 public key sending scp ~/.ssh/segment1_id_rsa.pub master:~/.ssh # segment2 public key sending scp ~/.ssh/segment2_id_rsa.pub master:~/.ssh
master node public key merge
cat id_rsa.pub segment1_id_rsa.pub segment2_id_rsa.pub >> authorized_keys
master node merge public key sending
scp ~/.ssh/authorized_keys segment1:~/.ssh scp ~/.ssh/authorized_keys segment2:~/.ssh
Modify permissions
chmod 700 ~/.ssh chmod 600 ~/.ssh/authorized_keys
3. Create master node profile
Switch to gpadmin user
su - gpadmin
Create configuration folder
mkdir -p /home/gpadmin/conf
Create the hostlist and edit the file
vim /home/gpadmin/conf/hostlist master segment1 segment2
Create seg_hosts file and edit
vim /home/gpadmin/conf/seg_hosts segment1 segment2
5, Installation (gpadmin user)
The three machines are installed in the specified directory respectively
rpm -ivh --prefix=/usr/local/services/greenplum/ open-source-greenplum-db-6.17.2-rhel7-x86_64.rpm
Follow the prompts to install the required dependencies
eg(Missing dependent packages for an installation): sudo yum install -y bzip2 libyaml zip libevent
On the master node, switch to the root user and configure the secret free connection for greenplug
source /usr/local/services/greenplum/greenplum-db/greenplum_path.sh gpssh-exkeys -f /home/gpadmin/conf/hostlist
Create data directory in batch and authorize
# Secret free connection, batch operation of three machines gpssh -f /home/gpadmin/conf/hostlist mkdir -p /opt/greenplum/data/master mkdir -p /opt/greenplum/data/primary mkdir -p /opt/greenplum/data/mirror mkdir -p /opt/greenplum/data2/primary mkdir -p /opt/greenplum/data2/mirror # Authorize users chown -R gpadmin:gpadmin /usr/local chown -R gpadmin:gpadmin /opt
Configure environment variables for gpadmin users
# Open file vim /home/gpadmin/.bash_profile # Added content source /usr/local/services/greenplum/greenplum-db/greenplum_path.sh export MASTER_DATA_DIRECTORY=/opt/greenplum/data/master/gpseg-1 export GPPORT=5432 export PGDATABASE=gp_sydb # Apply changes with immediate effect source /home/gpadmin/.bash_profile
6, Initialize database
Switch to gpadmin user and create a new initialization configuration file initgp_config
cd /usr/local/services/greenplum/greenplum-db/docs/cli_help/gpconfigs cp gpinitsystem_config initgp_config
Modify configuration file initgp_config
declare -a DATA_DIRECTORY=(/opt/greenplum/data/primary /opt/greenplum/data/primary /opt/greenplum/data2/primary /opt/greenplum/data2/primary) declare -a MIRROR_DATA_DIRECTORY=(/opt/greenplum/data/mirror /opt/greenplum/data/mirror /opt/greenplum/data2/mirror /opt/greenplum/data2/mirror) ARRAY_NAME="gp_sydb" #Initialize database name MASTER_HOSTNAME=centos01 #Master node name MASTER_DIRECTORY=/opt/greenplum/data/master #The resource directory is a resource directory created earlier MASTER_DATA_DIRECTORY=/opt/greenplum/data/master/gpseg-1 DATABASE_NAME=gp_sydb #Configured initialization database name MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts
Switch the gpadmin user and perform initialization (the user operation created before must be used, and cannot be operated under the root user)
source /usr/local/services/greenplum/greenplum-db/greenplum_path.sh gpinitsystem -c initgp_config -D
If there is an error in this step, you need to initialize the generated gpseg Delete all files and restart initialization(/opt/greenplum/data/ /opt/greenplum/data2 within primary,master Created within gpseg-1 And so on), the specific error information can be /home/gpadmin/gpAdminLogs Log view in
7, Greenplum connects to external clients
Modify / opt / greenplus / data / Master / gpseg-1 / PG under the master node_ hba. Conf configuration file
# Add a line to indicate that any user is allowed to connect host all all 0.0.0.0/0 trust
8, Database operation
| command | significance |
|---|---|
| gpstart | Start database |
| gpstop -r | restart |
| gpstop -u | Reload only configuration file changes |
| gpstop | Stop database |
| psql -d gp_sydb | Log in to gp_sydb database |