Greenplum 6.17 cluster construction

Greenplum 6.17 cluster construction

1, Cluster planning

Hostipfunction
master192.168.2.20master
segment1192.168.2.21segment
segment2192.168.2.22segment

The password is 1

2, Environmental description

nameexplain
operating systemCentos7
Database installation packageopen-source-greenplum-db-6.17.2-rhel7-x86_64.rpm

3, Preparation before installation

1. Other tools required for virtual machine minimum installation

sudo yum install -y epel-release
sudo yum install -y psmisc nc net-tools rsync vim lrzsz ntp libzstd openssl-static tree iotop git

2. Modify system files

Modify the system resource limit, / etc / security / limits Add the following information to the conf file

* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

Modify the system kernel information, / etc / sysctl Add the following information to the conf file:

# kernel.shmall calculates echo $(expr $(getconf _phys _pages) / 2 with the following command
kernel.shmall = 357475    
# kernel.shmmax calculates echo $(expr $(getconf _PHYS_PAGES) / 2 \* $(getconf PAGE_SIZE)) with the following command
kernel.shmmax = 1464217600         
kernel.shmmni = 4096
vm.overcommit_memory = 2
vm.overcommit_ratio = 95

kernel.sem = 500 2048000 200 4096
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.swappiness = 10
vm.zone_reclaim_mode = 0
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.dirty_background_ratio = 3
vm.dirty_ratio = 10

After modification, execute the command to take effect immediately

sysctl -p

Modify / etc / security / limits * * nproc. Under D / The conf file is (the file name may be 20-nproc.conf or 90-nproc.conf):

*          soft    nproc     131072

3. Turn off the firewall

Turn off the firewall

systemctl stop firewalld

Turn off firewall self startup

systemctl disable firewalld

View firewall status

systemctl status firewalld
systemctl is-enabled firewalld

Close selinux and modify / etc/sysconfig/selinux:

SELINUX=disabled

Configuration effective

setenforce 0

4. Configure host

Modify / etc/hostname

5. Add user group

Add gpadmin user group and grant corresponding permissions

# add group 
groupadd -g 530 gpadmin
# Add user
useradd -g 530 -u 530 -m -d /home/gpadmin -s /bin/bash gpadmin
# Change owner
chown -R gpadmin:gpadmin /home/gpadmin
# Change Password
passwd gpadmin

Assign sudo execution permission to users

visudo

In“ root    ALL=(ALL)       ALL"Add later
gpadmin    ALL=(ALL)       ALL

==============================================

Clone the other two virtual machines and modify the ip and host

modify host Corresponding file: /etc/hostname
 modify ip Corresponding file: /etc/sysconfig/network-scripts(Generally ifcfg-ens33)   

===============================================

4, Pre installation configuration

1. Add host mapping

Add cluster host and ip in / etc/hosts

192.168.2.20 master
192.168.2.21 segment1
192.168.2.22 segment2

2. ssh password free login (gpadmin user)

Generate secret key (three nodes)

ssh-keygen -t rsa

The segment node sends the public key to the master node

# segment1 public key replication
cp ~/.ssh/id_rsa.pub ~/.ssh/segment1_id_rsa.pub
# segment2 public key replication
cp ~/.ssh/id_rsa.pub ~/.ssh/segment1_id_rsa.pub
# segment1 public key sending
scp ~/.ssh/segment1_id_rsa.pub master:~/.ssh
# segment2 public key sending
scp ~/.ssh/segment2_id_rsa.pub master:~/.ssh

master node public key merge

cat id_rsa.pub segment1_id_rsa.pub segment2_id_rsa.pub >> authorized_keys

master node merge public key sending

scp ~/.ssh/authorized_keys segment1:~/.ssh
scp ~/.ssh/authorized_keys segment2:~/.ssh

Modify permissions

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

3. Create master node profile

Switch to gpadmin user

su - gpadmin

Create configuration folder

mkdir -p /home/gpadmin/conf

Create the hostlist and edit the file

vim /home/gpadmin/conf/hostlist

master
segment1
segment2

Create seg_hosts file and edit

vim /home/gpadmin/conf/seg_hosts
 
segment1
segment2

5, Installation (gpadmin user)

The three machines are installed in the specified directory respectively

rpm -ivh --prefix=/usr/local/services/greenplum/  open-source-greenplum-db-6.17.2-rhel7-x86_64.rpm

Follow the prompts to install the required dependencies

eg(Missing dependent packages for an installation):
sudo yum install -y bzip2 libyaml zip libevent

On the master node, switch to the root user and configure the secret free connection for greenplug

source /usr/local/services/greenplum/greenplum-db/greenplum_path.sh 
gpssh-exkeys -f /home/gpadmin/conf/hostlist

Create data directory in batch and authorize

# Secret free connection, batch operation of three machines
gpssh -f /home/gpadmin/conf/hostlist

mkdir -p /opt/greenplum/data/master
mkdir -p /opt/greenplum/data/primary
mkdir -p /opt/greenplum/data/mirror
mkdir -p /opt/greenplum/data2/primary
mkdir -p /opt/greenplum/data2/mirror

# Authorize users
chown -R gpadmin:gpadmin /usr/local
chown -R gpadmin:gpadmin /opt

Configure environment variables for gpadmin users

# Open file
vim /home/gpadmin/.bash_profile

# Added content
source /usr/local/services/greenplum/greenplum-db/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/opt/greenplum/data/master/gpseg-1
export GPPORT=5432
export PGDATABASE=gp_sydb

# Apply changes with immediate effect
source /home/gpadmin/.bash_profile

6, Initialize database

Switch to gpadmin user and create a new initialization configuration file initgp_config

cd /usr/local/services/greenplum/greenplum-db/docs/cli_help/gpconfigs
cp gpinitsystem_config initgp_config

Modify configuration file initgp_config

declare -a DATA_DIRECTORY=(/opt/greenplum/data/primary /opt/greenplum/data/primary  /opt/greenplum/data2/primary /opt/greenplum/data2/primary)
declare -a MIRROR_DATA_DIRECTORY=(/opt/greenplum/data/mirror /opt/greenplum/data/mirror  /opt/greenplum/data2/mirror /opt/greenplum/data2/mirror)
 
ARRAY_NAME="gp_sydb"                                           #Initialize database name
MASTER_HOSTNAME=centos01                                       #Master node name
MASTER_DIRECTORY=/opt/greenplum/data/master                    #The resource directory is a resource directory created earlier
MASTER_DATA_DIRECTORY=/opt/greenplum/data/master/gpseg-1       
DATABASE_NAME=gp_sydb                                          #Configured initialization database name
MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts     

Switch the gpadmin user and perform initialization (the user operation created before must be used, and cannot be operated under the root user)

source /usr/local/services/greenplum/greenplum-db/greenplum_path.sh
gpinitsystem -c initgp_config -D
If there is an error in this step, you need to initialize the generated gpseg Delete all files and restart initialization(/opt/greenplum/data/ /opt/greenplum/data2 within primary,master Created within gpseg-1 And so on), the specific error information can be /home/gpadmin/gpAdminLogs Log view in

7, Greenplum connects to external clients

Modify / opt / greenplus / data / Master / gpseg-1 / PG under the master node_ hba. Conf configuration file

# Add a line to indicate that any user is allowed to connect
host   all   all   0.0.0.0/0    trust

8, Database operation

commandsignificance
gpstartStart database
gpstop -rrestart
gpstop -uReload only configuration file changes
gpstopStop database
psql -d gp_sydbLog in to gp_sydb database

Keywords: Linux Database

Added by ilangovanbe2005 on Thu, 24 Feb 2022 11:44:41 +0200