Resource address:
Get the CM package (tar.gz)
http://archive-primary.cloudera.com/cm5/cm/5/
Get CDH packages
http://archive.cloudera.com/cdh5/parcels/5.6/
I. Environmental preparation
1. Install JDK (each node)
1) decompression (/opt)
tar zxvf jdk-7u55-linux-x64.tar.gz
(2) Configuration/etc/profile
vi /etc/profile
export JAVA_HOME=/usr/jdk1.7.0_55 export PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/jre/bin CLASSPATH=.:/usr/jdk1.7.0_55/lib:/usr/jdk1.7.0_55/jre/lib:$CLASSPATH export CLASSPATH
3) Refresh environment variables: source/etc/profile
4) Check the Java environment: java-version
2. Close the firewall
service iptables stop
chkconfig iptables off
3. Close selinux
vi /etc/selinux/config
SELINUX=disabled
4. Network Configuration
Configuration/etc/hosts
192.168.252.137 cdh1 192.168.252.138 cdh2 192.168.252.139 cdh3
Configuration/etc/sysconfig/network
Modify the machine name to cdh1cdh2cdh3
3) Configuration/etc/sysconfig/network-scripts/ifcfg-eth0
Check the network card HWaddr according to ifconfig and modify it to ifcfg-eth0
vi /etc/sysconfig/network-scripts/ifcfg-eth0
4) Restart the network
service network restart
5. Configuring ssh mutual trust
1) Generate public key:
ssh-keygen -t rsa
2)cdh1 generation file authorized_keys:
cp id_rsa.pub authorized_keys
3) Copy id_rsa.pub of cdh2cdh 3 to authorized_keys of cdh1
4) Distributing authorized_keys of cdh1 to cdh2cdh3
scp authorized_keys cdh2:~/.ssh/ scp authorized_keys cdh2:~/.ssh/
5) Testing: ssh cdh2
6. Time synchronization
1) Install NTP (each node)
yum install ntp chkconfig ntpd on
2) Configuring NTP
vi /etc/ntp.conf
Master configuration: server ntp.sjtu.edu.cn prefer
Slaves configuration: server cdh1
Installation of mysql
1. Installation services
yum install -y mysql-server yum install -y mysql-devel yum install -y mysql-libs yum install -y mysql chkconfig mysqld on
2. Start up the service
service mysqld start
3. Setting Password
mysqladmin -u root password 'mysql'
Password is not allowed. Password setting may cause subsequent connection problems.
4. Log in and build the required Library
mysql -uroot -pmysql
--Cluster Monitoring Database create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci; --hive data base create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci; --hue data base create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci; --oozie data base create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
5. Authorization
--grant root User access rights grant all privileges on *.* to 'root'@'%' identified by 'mysql' with grant option; --Refresh permissions flush privileges;
III. Installation of CM
1. Decompression (/opt)
tar zxvf cloudera-manager-el6-cm5.9.2_x86_64.tar.gz
2. Create users (nodes)
useradd --system --home=/opt/cm-5.9.2/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
3.cdh1 modifies sudoers (to avoid configurations that cannot be downloaded to the agent)
vim /etc/sudoers
--stay root ALL=(ALL) ALL Later added cloudera-scm ALL=(ALL) NOPASSWD: ALL
4. Deploy mysql jar package
Deploy jar packages for cm
cp /opt/mysql-connector-java-5.1.39-bin.jar /opt/cm-5.9.2/share/cmf/lib/
2) Deploy jar packages for hive and oozie
cp /opt/mysql-connector-java-5.1.39-bin.jar /opt/cloudera/parcels/CDH/lib/hive/lib/ cp /opt/mysql-connector-java-5.1.39-bin.jar /opt/cloudera/parcels/CDH/lib/oozie/lib/ cp /opt/mysql-connector-java-5.1.39-bin.jar /var/lib/oozie/
5. Modify agent configuration
vi /opt/cm-5.9.2/etc/cloudera-scm-agent/config.ini
Change the server_host configuration item from localhost to cdh1
6. Distribute cm-5.9 to other nodes
scp –r cm-5.9.2 cdh2:/opt/ scp –r cm-5.9.2 cdh3:/opt/
7. Initialize the mysql Library of cm
/opt/cm-5.9.2/share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -u root -pmysql --scm-host localhost scm scm scm
8. Deploying local sources of cdh
The following packages are placed in / opt / cloudera / package-repo
CDH-5.6.1-1.cdh5.6.1.p0.3-el6.parcel
CDH-5.6.1-1.cdh5.6.1.p0.3-el6.parcel.sha1
manifest.json
Rename (to avoid unrecognized re-downloads)
mv CDH-5.6.1-1.cdh5.6.1.p0.3-el6.parcel.sha1 CDH-5.6.1-1.cdh5.6.1.p0.3-el6.parcel.sha
IV. Start CM
1. Host Node Start
/opt/cm-5.9.2/etc/init.d/cloudera-scm-server start
Check logs
tail -f /opt/cm-5.9.2/log/cloudera-scm-server/cloudera-scm-server.log
2. Subnode Startup
/opt/cm-5.9.2/etc/init.d/cloudera-scm-agent start
3. View ports
netstat -apn|grep 7180
CM service needs to wait a little while to detect port 7180 open
Installation of CDH
web page login cdh:7180, initial username password admin, follow the prompt steps to install CDH