If you use the Oushu Lava public cloud or private cloud 2.0 +, you can automatically deploy OushuDB through the Lava UI. For details, see:   http://oushu.io/docs/ch/lava-....

If you do not use Oushu Lava and only want to deploy OushuDB separately, please follow the steps in this section.

First, in oushum1, modify / usr/local/hawq/etc/slaves and write the hostname of all OushuDB segment nodes into the slaves. In this installation, oushus1 and oushus2 should be written into the slaves. The contents of the slaves are:
oushus1oushus2

Install hawq on other nodes:

hawq ssh -h oushum2 -e "yum install -y hawq"hawq ssh -f slaves -e "yum install -y hawq"

On the oushum1 node, add the following to the configuration file / etc/sysctl.conf:

kernel.shmmax = 1000000000kernel.shmmni = 4096kernel.shmall = 4000000000kernel.sem = 250 512000 100 2048kernel.sysrq = 1kernel.core_uses_pid = 1kernel.msgmnb = 65536kernel.msgmax = 65536kernel.msgmni = 2048net.ipv4.tcp_syncookies = 0net.ipv4.conf.default.accept_source_route = 0net.ipv4.tcp_tw_recycle = 1net.ipv4.tcp_max_syn_backlog = 200000net.ipv4.conf.all.arp_filter = 1net.ipv4.ip_local_port_range = 10000 65535net.core.netdev_max_backlog = 200000net.netfilter.nf_conntrack_max = 524288fs.nr_open = 3000000kernel.threads-max = 798720kernel.pid_max = 798720# increase networknet.core.rmem_max=2097152net.core.wmem_max=2097152net.core.somaxconn=4096

Copy the configuration file in / etc/sysctl.conf on oushum1 to all nodes:

hawq scp -r -f hostfile /etc/sysctl.conf =:/etc/
stay oushum1，Use“ hawq ssh"Execute the following command to make all nodes/etc/sysctl.conf The system configuration in takes effect>: 
hawq ssh -f hostfile -e "sysctl -p"

In oushum1, create the file / etc/security/limits.d/gpadmin.conf:

* soft nofile 1048576* hard nofile 1048576* soft nproc 131072* hard nproc 131072

Copy the configuration file in / etc/security/limits.d/gpadmin.conf on oushum1 to all nodes:

hawq scp -r -f hostfile /etc/security/limits.d/gpadmin.conf =:/etc/security/limits.d

In oushum1, create / hawq/default_filespace on Hadoop and grant gpadmin permission:

sudo -u hdfs hdfs dfs -mkdir -p /hawq/default_filespacesudo -u hdfs hdfs dfs -chown -R gpadmin /hawq

In oushum1, create mhostfile and record the hostname of all hawq masters and standby master s, similar to hostfile:
touch mhostfile

mhostfile record content:
oushum1oushum2

In oushum1, create a shostfile and record the hostname of all segment s of hawq, similar to hostfile:
touch shostfile

shostfile record content:
oushus1oushus2

In oushum1, use "hawq ssh" to create the master metadata directory and temporary file directory on the master and standby nodes, and grant gpadmin permission:

establish master Metadata directory hawq ssh -f mhostfile -e 'mkdir -p /data1/hawq/masterdd'#Create a temporary file directory hawq SSH - F mhostfile - E 'MKDIR - P / data1 / hawq / tmp' hawq SSH - F mhostfile - E 'MKDIR - P / data2 / hawq / tmp' hawq SSH - F mhostfile - E 'chown - R gpadmin: gpadmin / data1 / hawq'hawq SSH - F mhostfile - e' chown - R gpadmin: gpadmin / data2 / hawq '

In oushum1, use "hawq ssh" to create segment metadata directory and temporary file directory on all segments, and grant gpadmin permission:

establish segment Metadata directory hawq ssh -f shostfile -e 'mkdir -p /data1/hawq/segmentdd'#Create temporary file directory hawq ssh -f shostfile -e 'mkdir -p /data1/hawq/tmp'hawq ssh -f shostfile -e 'mkdir -p /data2/hawq/tmp'hawq ssh -f shostfile -e 'chown -R gpadmin:gpadmin /data1/hawq'hawq ssh -f shostfile -e 'chown -R gpadmin:gpadmin /data2/hawq'

In oushum1, switch to the hawq user. The hawq related configuration files need to use this user permission:

su - gpadmin
 modify/usr/local/hawq/etc/hdfs-client.xml: (And hdfs Similar, remove it first HA Comments for:
<configuration><property><name>dfs.nameservices</name><value>oushu</value></property><property><name>dfs.ha.namenodes.oushu</name><value>nn1,nn2</value></property><property><name>dfs.namenode.rpc-address.oushu.nn1</name><value>oushum2:9000</value></property><property><name>dfs.namenode.rpc-address.oushu.nn2</name><value>oushum1:9000</value></property><property><name>dfs.namenode.http-address.oushu.nn1</name><value>oushum2:50070</value></property><property><name>dfs.namenode.http-address.oushu.nn2</name><value>oushum1:50070</value></property>...<property><name>dfs.domain.socket.path</name><value>/var/lib/hadoop-hdfs/dn_socket</value><description>Optional.  This is a path to a UNIX domain socket that will be used for communication between the DataNode and local HDFS clients.If the string "_PORT" is present in this path, it will be replaced by the TCP port of the DataNode.</description></property>...</configuration>

In oushum1, modify / usr/local/hawq/etc/hawq-site.xml. Note: oushu in hawq_dfs_url is the value of dfs.nameservices, which is configured in hdfs-site.xml; the value in sigma_nodes_url should preferably take the first two lines in / usr/local/hawq/etc/slaves file:

<configuration><property><name>hawq_master_address_host</name><value>oushum1</value></property>...<property><name>hawq_standby_address_host</name><value>oushum2</value><description>The host name of hawq standby master.</description></property>...<property><name>hawq_dfs_url</name><value>oushu/hawq/default_filespace</value><description>URL for accessing HDFS.</description></property><property><name>magma_nodes_url</name><value>oushus1:6666,oushus2:6666</value><description>urls for accessing magma.</description></property><property><name>hawq_master_directory</name><value>/data1/hawq/masterdd</value><description>The directory of hawq master.</description></property><property><name>hawq_segment_directory</name><value>/data1/hawq/segmentdd</value><description>The directory of hawq segment.</description></property><property><name>hawq_master_temp_directory</name><value>/data1/hawq/tmp,/data2/hawq/tmp</value><description>The temporary directory reserved for hawq master. NOTE: please DONOT add " " between directories. </description></property><property><name>hawq_segment_temp_directory</name><value>/data1/hawq/tmp,/data2/hawq/tmp</value><description>The temporary directory reserved for hawq segment. NOTE: please DONOT add " " between directories. </description></property><property><name>default_storage</name><value>hdfs</value><description>Sets the default storage when creating table</description></property><property><name>hawq_init_with_hdfs</name><value>true</value><description>Choose whether init cluster with hdfs</description></property>...<property><name>hawq_rm_yarn_address</name><value>oushum1:8032</value><description>The address of YARN resource manager server.</description></property><property><name>hawq_rm_yarn_scheduler_address</name><value>oushum1:8030</value><description>The address of YARN scheduler server.</description></property>...<property><name>hawq_rm_yarn_app_name</name><value>hawq</value><description>The application name to register hawq resource manager in YARN.</description></property>...<property><name>hawq_re_cgroup_hierarchy_name</name><value>hawq</value><description>The name of the hierarchy to accomodate CGroup directories/files for resource enforcement.For example, /sys/fs/cgroup/cpu/hawq for CPU sub-system.</description></property>...</configuration>
OushuDB4.0 New version Magma Separate configuration and start stop function, using magam Service time, stay oushum1，Use“ hawq ssh"At all slave Node creation node Data directory and grant gpadmin jurisdiction
hawq ssh -f shostfile -e 'mkdir -p /data1/hawq/magma_segmentdd'hawq ssh -f shostfile -e 'mkdir -p /data2/hawq/magma_segmentdd'hawq ssh -f shostfile -e 'chown -R gpadmin:gpadmin /data1/hawq'hawq ssh -f shostfile -e 'chown -R gpadmin:gpadmin /data2/hawq'

Then edit the configuration / usr / local / hawq / etc / sigma-site.xml:

<property><name>nodes_file</name><value>slaves</value><description>The magma nodes file name at GPHOME/etc</description></property><property><name>node_data_directory</name><value>file:///data1/hawq/magma_segmentdd,file:///data2/hawq/magma_segmentdd</value><description>The data directory for magma node</description></property><property><name>node_log_directory</name><value>~/hawq-data-directory/segmentdd/pg_log</value><description>The log directory for magma node</description></property><property><name>node_address_port</name><value>6666</value><description>The port magma node listening</description></property><property><name>magma_range_number</name><value>2</value></property><property><name>magma_replica_number</name><value>3</value></property><property><name>magma_datadir_capacity</name><value>3</value></property>

In oushum1, switch to root:
su - root

Copy the configuration file in / usr/local/hawq/etc on oushum1 to all nodes:

source /usr/local/hawq/greenplum_path.shhawq scp -r -f hostfile /usr/local/hawq/etc =:/usr/local/hawq

In oushum1, switch to the gpadmin user and create hhostfile:

su - gpadminsource /usr/local/hawq/greenplum_path.sh  #Set the hawq environment variable touch hhostfile

The hhostfile file records the host names of all OushuDB nodes, as follows:
oushum1oushum2oushus1oushus2

Log in to each machine with the root user and modify the gpadmin user password:

sudo echo 'password' | sudo passwd  --stdin gpadmin

Exchange key s for gpadmin users, and enter the gpadmin user password of the corresponding node as prompted:

su - gpadminsource /usr/local/hawq/greenplum_path.sh  #Set the hawq environment variable hawq SSH exkeys - F hhostfile
 stay oushum1，use gpadmin User rights, initialization OushuDB Cluster, when prompted“ Continue with HAWQ init"When, enter Y: 
hawq init cluster   //Oushudb 4.0 does not start the sigma service by default
hawq init cluster --with_magma   //New in oushudb 4.0, this usage is not supported in version 3.X

// The -- with_sigma option is added in oushudb 4.0, but only the hawq init|start|stop cluster command can have the -- with_sigma option.

be careful:
When initializing the OushuDB cluster, you need to ensure that masterdd and segmentdd > under the created / data*/hawq / directory are empty, and / hawq/default_filespace created on hadoop is empty # in addition, if the hawq init cluster fails, you can first execute the following command to stop the hawq cluster, clear the directory, find out the cause of the > problem, and then re initialize. Hawq stop clusters Server# at the OushuDB master node, according to the configuration of this installation, s use the following command to clear all the hawq directories, and then rebuild the hawq subdirectories:

hawq ssh -f hhostfile -e 'rm -fr /data1/hawq/masterdd/*'hawq ssh -f hhostfile -e 'rm -fr /data1/hawq/segmentdd/*'hawq ssh -f hhostfile -e 'rm -fr /data1/hawq/magma_masterdd/*'hawq ssh -f hhostfile -e 'rm -fr /data1/hawq/magma_segmentdd/*'hawq ssh -f hhostfile -e 'rm -fr /data2/hawq/magma_segmentdd/*'#On the HDFS namenode, use the following command to clear the / hawq/default_filespace. If there is user data in the / hawq/default_filespace, pay attention to backing up the data to avoid losses: hdfs dfs -rm -f -r /hawq/default_filespace/*

You also need to check whether the parameter configuration of HDFS is correct, preferably with the gpadmin user. If the parameter configuration is incorrect, > although HDFS can start normally sometimes, HDFS will make errors under high load.

su - gpadminsource /usr/local/hawq/greenplum_path.shhawq check -f hostfile --hadoop /usr/hdp/current/hadoop-client/ --hdfs-ha
Check whether OushuDB is running normally:

su - gpadmin
source /usr/local/hawq/greenplum_path.sh
psql -d postgres
select * from gp_segment_configuration;  #Make sure all nodes are up

create table t(i int);
insert into t select generate_series(1,1000);
select count(*) from t;