1. Installation and configuration of Jenkins tools:
In this blog, I will mainly explain the following knowledge points and practical experience for your reference:
Installation and configuration of Jenkins:
_2. Jenkins Slave node configuration:
_3. Create a maven project:
Configuration code checks sonar:
_5. Jenkins Pipeline configuration:
6. Backup and restore of Jenkins:
1. Installation and configuration of Jenkins:
1.1 Introduction to Jenkins:
_Jenkins is an open source software project. It is a continuous integration tool based on Java. It is used to monitor the continuous repetitive work. It aims to provide an open and easy-to-use software platform to make the continuous integration of software possible. Jenkins is a CI/CD tool that integrates continuous release tools. The main process is: developing and uploading local code to git server - - > merging branch to master - > Jenkins server git clone master to Jenkins local server - -> Jenkins compiles war packages or jar packages through local maven command - > war packages or jar packages are stored under the target directory of jenkins workspace Directory - - - Jenkins replicates war packages or jar packages through ssh plug-in scp or ansible To the deployment machine - > Jenkins calls the script of the target machine through ssh plug-in to restart the service;
1.2 Jenkins Installation:
Firstly install jdk:
cd /software wget ftp://bqjrftp:Pass123$%^@10.83.20.27:9020/software/jdk/jdk-8u111-linux-x64.tar.gz tar xzvf jdk-8u111-linux-x64.tar.gz -C /usr/local/ vim /etc/profile.d/java.sh JAVA_HOME=/usr/local/jdk1.8.0_111 PATH=$PATH:$JAVA_HOME/bin export PATH JAVA_HOME [root@devops jdk1.8.0_111]# . /etc/profile #Configure an environment variable [root@devops jdk1.8.0_111]# java -version java version "1.8.0_111" Java(TM) SE Runtime Environment (build 1.8.0_111-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)
_Then install maven, a tool for compiling builds, and of course some projects may also use ant to build:
cd /software/ wget ftp://bqjrftp:Pass123$%^@10.83.20.27:9020/software/maven/apache-maven-3.3.9-bin.tar.gz tar xzvf apache-maven-3.3.9-bin.tar.gz -C /usr/local/ cd /usr/local/ ln -sv apache-maven-3.3.9/ maven vim /etc/profile.d/maven.sh MAVEN_HOME=/usr/local/maven PATH=$PATH:$MAVEN_HOME/bin export PATH MAVEN_HOME source /etc/profile
_Then install git,linux operating system default git command, but the default git version is relatively low, need to install a higher version of their own:
cd /software yum remove git-1.7.1-9.el6_9.x86_64 #Unload the old git rpm package first yum install openssl-devel curl-devel expat-devel perl-ExtUtils-MakeMaker gettext gettext-libs gettext-devel asciidoc xmlto docbook2X Installation dependency ln -s /usr/bin/db2x_docbook2texi /usr/bin/docbook2x-texi wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.14.tar.gz#installation dependency tar xzvf libiconv-1.14.tar.gz ./configure --prefix=/usr/local/libiconv Make && make install ln -sv /usr/local/libiconv/lib/libiconv.so /usr/lib64/ ln -sv /usr/local/libiconv/lib/libiconv.so.2 /usr/lib64/ cd /software wget https://www.kernel.org/pub/software/scm/git/git-2.7.3.tar.gz tar xzvf git-2.7.3.tar.gz cd git-2.7.3 ./configure prefix=/usr/local/git all --with-iconv=/usr/local/libiconv/ Make Make install echo "export PATH=$PATH:/usr/local/git/bin" >> /etc/bashrc source /etc/bashrc Git --version //View version, already is [root@devops git-2.7.3]# git --version git version 2.7.3
_Next comes the installation of jenkins. Jenkins can be installed in two ways. One is to use the yum source of Jenkins directly, install jinkins through yum, and the other is to download the jenkins.war package from the official website of Jenkins and run in tomcat. This document uses Tomcat container to run Jenkins
# Install tomcat first cd /software/ wget ftp://bqjrftp:Pass123$%^@10.83.20.27:9020/software/tomcat/apache-tomcat-8.5.8.tar.gz tar xzvf apache-tomcat-8.5.8.tar.gz -C /usr/local cd /usr/local ln -sv apache-tomcat-8.5.8/ tomcat cd /usr/local/tomcat/webapps rm -rf * # Delete useless projects # Download the war package at the official Jenkins website: https://jenkins.io/download/, then place it under the / usr/local/tomcat/webapps directory and start Tomcat /usr/local/tomcat/bin/catalina.sh
_After the successful launch of jenkins, the first login needs to enter the Jenkins administrator password, according to the prompt to enter the administrator password, and then install common plug-ins. There is a problem that needs to be solved. By default, the Jenkins server installed by us can not connect to the update site of Jenkins to install the plug-in. The following configuration is needed to install the plug-in.
vim /root/.jenkins/updates/default.json
# Modified from www.google.com to www.baidu.com
vim /root/.jenkins/hudson.model.UpdateCenter.xml
# Change the update site here to a domestic site
<?xml version='1.1' encoding='UTF-8'?>
<sites>
<site>
<id>default</id>
<url>https://mirrors.tuna.tsinghua.edu.cn/jenkins/updates/update-center.json</url>
</site>
</sites>
#Then restart jenkins
/usr/local/tomcat/bin/catalina.sh stop
/usr/local/tomcat/bin/catalina.sh start
_is followed by some initialization configurations, mainly including the following aspects:
- Tool Configuration: For example, jdk, maven, git, ant, npm, etc. System Management - "Global Tool Configuration"
- Configuration of mail alarm: Configuration of user information and mail server information for sending mail, etc.
- Global Security Configuration: Defines whether the user who logs on to jenkins is a local user or an LDAP user, and whether to open the project authorization;
2. Jenkins Slave node configuration:
_jenkins supports master-slave mode. Master mainly does task management, slave is the machine that actually runs job. So the slave machine also needs to install maven, jdk and other tools; the configuration steps are as follows:
tar xzvf jdk-8u111-linux-x64.tar.gz -C /usr/local/ mv apache-maven-3.3.9-bin.tar.gz /home/jenkins/ useradd jenkins passwd jenkins chown -R jenkins.jenkins /home/jenkins/ vim /etc/ssh/sshd_config AllowUser jenkins service sshd restart su - jenkins vim /etc/profile.d/java.sh source /etc/profile.d/java.sh # Create jenkins users on slave nodes, install maven and jdk tools, configure environment variables, and configure to allow jenkins users to log in
System Management - Node Management - New Node:
3. Create a maven project:
4. Configuration code checks sonar:
Sonar is an open source platform for code quality management to manage the quality of Java source code. Through the plug-in mechanism, Sonar can integrate different testing tools, code analysis tools, and continuous integration tools, such as pmd-cpd, checkstyle, findbugs, Jenkins. These results are reprocessed by different plug-ins, and the changes of code quality are measured quantitatively, so that code quality management can be conveniently carried out for projects of different sizes and types.
Sonar Qube is an open source platform for managing the quality of source code. Sonar is not only a quality data reporting tool, but also a code quality management platform. The supported languages include Java, PHP, C#, C, Cobol, PL/SQL, Flex, etc.
Install jdk:
lftp bqjrftp:Pass123$%^@10.83.20.27:9020 cd /software/jdk/ mget jdk-8u111-linux-x64.tar.gz Tar xzvf jdk-8u111-linux-x64.tar.gz -C /usr/local vim /etc/profile.d/java.sh JAVA_HOME=/usr/local/jdk1.8.0_111 JRE_HOME=/usr/local/jdk1.8.0_111/jre CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin export JAVA_HOME JRE_HOME CLASS_PATH PATH source /etc/profile Echo $JAVA_HOME /usr/local/jdk1.8.0_111 //If the result is the output above, the jdk installation is successful
Install mysql:
_Be sure to install version 5.6 or more of mysql. If the version installed is 5.5 or MariaDB 10.2.15, it will not work. I just started installing a lower version of mysql, and later started sonarqubo when the error MySQL version is too low.
lftp bqjrftp:Pass123$%^@10.83.20.27:9020 cd /software/mysql/ mget mariadb-5.5.54-linux-x86_64.tar.gz tar xzvf mariadb-5.5.54-linux-x86_64.tar.gz -C /usr/local/ mkdir -p /application/mysql/data #Create a data catalog Cd /usr/local/mysql/scripts ./mysql_install_db --basedir=/usr/local/mysql/ --datadir=/application/mysql/data/ --user=mysql # Ensure that the system has created mysql accounts and initialized the database vim /etc/my.cnf [mysqld] sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES [client] port = 3306 socket = /var/lib/mysql/mysql.sock # Specify the socket file path [mysqld] user = mysql port = 3306 socket = /var/lib/mysql/mysql.sock basedir = /usr/local/mysql datadir = /application/mysql/data/ # Specify a data catalog log-error = /var/log/mysqldb/mysql_error.log # Specify log directories pid-file = /application/mysql/data/mysql.pid # Specify PID file path mkdir -p /var/lib/mysql && chown -R mysql.mysql /var/lib/mysql mkdir -p /var/log/mysqldb && chown -R mysql.mysql /var/log/mysqldb cp -r /usr/local/mysql/support-files/mysql.server /etc/init.d/mysqld #Copy the default service startup file [root@SZ3FUAT0NAS0100 scripts]# cd /etc/init.d/ [root@SZ3FUAT0NAS0100 init.d]# chmod +x mysqld [root@SZ3FUAT0NAS0100 init.d]# chkconfig mysqld on [root@SZ3FUAT0NAS0100 init.d]# vim mysqld [root@SZ3FUAT0NAS0100 init.d]# service mysqld status ERROR! MySQL is not running [root@SZ3FUAT0NAS0100 init.d]# service mysqld start Starting MySQL.180523 10:01:09 mysqld_safe Logging to '/var/log/mysqldb/mysql_error.log'. 180523 10:01:09 mysqld_safe Starting mysqld daemon with databases from /application/mysql/data/ . SUCCESS! [root@SZ3FUAT0NAS0100 init.d]# service mysqld status SUCCESS! MySQL running (59947) [root@SZ3FUAT0NAS0100 init.d]# //Configure mysql environment variables [root@SZ3FUAT0NAS0100 profile.d]# cat /etc/profile.d/mysql.sh MYSQL_HOME=/usr/local/mysql/bin PATH=$MYSQL_HOME:$PATH export MYSQL_HOME PATH mysqladmin -u root password 'abc1234' #Modify the password of mysql root account //Install mycli tool, mysql client with extended function: cd /usr/local/python2.7.12/bin/ python -m ensurepip --default-pip #Install the corresponding version of the pip tool /usr/local/python2.7.12/bin/pip install mycli #Download and install mycli module #Create a database, database users, and grant permissions mariadb root@localhost:(none)> create database sonar character set utf8 collate utf8_general_ci; Query OK, 1 row affected Time: 0.001s mariadb root@localhost:(none)> create user 'sonar' identified by 'sonar'; Query OK, 0 rows affected Time: 0.001s mariadb root@localhost:(none)> grant all on sonar.* TO 'sonar'@'%' identified by 'sonar'; Query OK, 0 rows affected Time: 0.001s mariadb root@localhost:(none)> grant all on sonar.* TO 'sonar'@'localhost' identified by 'sonar'; Query OK, 0 rows affected Time: 0.001s mariadb root@localhost:(none)> flush privileges; Query OK, 0 rows affected Time: 0.001s mariadb root@localhost:(none)> //Log in to mysql through the mycli tool, and the input will be prompted. The command purpose of the previous paragraph is to create sonar database, create sonar users, and give sonar database permissions.
_Now start installing sonar server and sonar-runner SonarQube as servers, which have two main functions:
- Analysis of source code;
- Because it has built-in Apache modules, it provides interface access on the Web side.
SonarQube Runner is a command-line tool that uses SonarQube server to analyze code. It can be simply understood as a client. Therefore, in order to facilitate installation and debugging, it is recommended that both SonarQube and SonarQube Runner be downloaded.
cd /software/sonarqube mget sonarqube-6.7.3.zip mget sonar-runner-dist-2.4.zip cd /usr/local unzip sonarqube-6.7.3.zip unzip sonar-runner-dest-2.4.zip # Install sonar and sonar-runner vim /etc/profile.d/sonar.sh SONAR_HOME=/usr/local/sonarqube-6.7.3 SONAR_RUNNER_HOME=/usr/local/sonar-runner-2.4 PATH=$SONAR_HOME/bin:$SONAR_RUNNER_HOME/bin:$PATH export SONAR_HOME SONAR_RUNNER_HOME PATH # Configure sonar server, sonar command line tool environment variables #Modify sonarqubo server configuration file [root@SZ3FUAT0NAS0100 conf]# egrep -v "^#" sonar.properties |sed '/^[[:space:]]*$/d' sonar.jdbc.username=sonar sonar.jdbc.password=sonar sonar.jdbc.url=jdbc:mysql://127.0.0.1:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance&useSSL=false sonar.web.host=10.83.64.130 sonar.web.port=9000 [root@SZ3FUAT0NAS0100 conf]# # Modify the configuration file of sonar-runner client [root@SZ3FUAT0NAS0100 conf]# egrep -v "^#" /usr/local/sonar-runner-2.4/conf/sonar-runner.properties |sed '/^[[:space:]]*$/d' sonar.host.url=http://10.83.64.130:9000 sonar.jdbc.url=jdbc:mysql://127.0.0.1:3306/sonar?useUnicode=true&characterEncoding=utf8 sonar.jdbc.username=sonar sonar.jdbc.password=sonar sonar.sourceEncoding=UTF-8 sonar.login=admin sonar.password=admin [root@SZ3FUAT0NAS0100 conf]# # Configure sonar to check the project's configuration file [root@SZ3FUAT0NAS0100 sonar-runner-2.4]# egrep -v "^#" /usr/local/sonar-runner-2.4/sonar-project.properties |sed '/^[[:space:]]*$/d' sonar.projectKey=my:clf sonar.projectName=clf sonar.projectVersion=1.0 sonar.sources=/root/java sonar.language=java sonar.sourceEncoding=UTF-8 sonar.my.property=value [root@SZ3FUAT0NAS0100 sonar-runner-2.4]# # Sonarqube comes with elastic search, whose configuration file is [root@SZ3FUAT0NAS0100 local]# egrep -v "^$|#" /usr/local/sonarqube-6.7.3/elasticsearch/config/elasticsearch.yml bootstrap.memory_lock: false bootstrap.system_call_filter: false [root@SZ3FUAT0NAS0100 local]# # Modify to false # Start sonarqube instead of using root users, you need to start it with elastic search users. # Prior to startup, privileges need to be granted chown -R elasticsearch.elasticsearch sonarqube-6.7.3/ # Then switch to elastic search cd /usr/local/sonarqube-6.7.3/bin/linux-x86-64/ sh sonar.sh start [elasticsearch@SZ3FUAT0NAS0100 linux-x86-64]$ sh sonar.sh status SonarQube is running (71514). [elasticsearch@SZ3FUAT0NAS0100 linux-x86-64]$ //Represents that sonarqube started successfully [root@SZ3FUAT0NAS0100 ~]# cat /usr/local/sonarqube-6.7.3/conf/sonar.properties |grep Xms sonar.web.javaOpts=-Xmx1024m -Xms512m -XX:+HeapDumpOnOutOfMemoryError # Modify java memory startup [root@SZ3FUAT0NAS0100 ~]# cat /usr/local/sonarqube-6.7.3/bin/linux-x86-64/sonar.sh |grep USER RUN_AS_USER=elasticsearch #Configure sonarqube to start elastic search
_5. Jenkins Pipeline configuration:
_First of all, I will introduce what Jenkins 2.0 is. The essence of Jenkins 2.0 is Pipeline as Code, which is an important role to help Jenkins realize the transition from CI to CD. What is Pipeline? Simply speaking, it is a set of workflow framework running on Jenkins, which connects tasks that originally run independently on a single or multiple nodes, and realizes complex publishing process that a single task is difficult to accomplish. The implementation of Pipeline is a Groovy DSL. Any publishing process can be described as a Groovy script. Jenkins supports reading scripts directly from the code base, thus realizing the concept of Pipeline as Code.
Several basic concepts of_Pipeline:
- Stage: A Pipeline can be divided into several Stages, each representing a set of operations. Note that Stage is a logical grouping concept that can span multiple Node s.
- Node: Node, a Node is a Jenkins node, or Master, or Agent, which is the specific runtime environment for implementing Step.
- Step: Step is the basic unit of operation, from creating a directory to building a Docker image, provided by various Jenkins Plugin s
_Jenkinsfile can be written in two grammars - Declarative and cripted
pipeline {
agent any
// Define variables. Distribution is a character class variable, i.e. manually input branch names; define radio variables to represent optional modules; and define character type variables to represent ip to be published.
parameters {
string(name: 'brach',description: 'branch')
choice(name: 'module',choices: ['aip-manager-server','auth-server','config-server','eureka-server','info-test-server','info-update-server','mq-server','ordertest-service-server','posttest-loan-server','pretest-loan-server','zuul-server'], description: 'module_name')
string(name: 'ip',description: 'ip')
}
tools {
maven 'M3'
jdk 'jdk1.8.0_92'
}
// Define git's code path and access key
stages{
stage('get code'){
steps{
git branch: '$brach', credentialsId: 'c943389e-ac1e-438e-9aaf-07342173a37a', url: 'http://test02@git.test.club/YWZC/AIP.git'
}
}
// Definition construction
stage('build'){
steps{
sh "/usr/local/maven/bin/mvn -Dmaven.test.skip=true clean install -f openapi/pom.xml"
}
}
// Define deployment. If you choose this module and this server ip when deploying, you will execute ansible to connect to the server, synchronize the compiled jar package and execute the script of the target server to complete the service restart.
stage('deploy'){
parallel {
stage('test-loan-server-192.168.20.186') {
when {
allOf{
environment name: 'module', value: 'test-loan-server'
environment name: 'ip', value: '192.168.20.186'
}
}
steps {
sh "ansible mmt-naip-testloan-1 -m synchronize -a 'mode=push src=$WORKSPACE/openapi/test-loan-server/target/test-loan-server-0.0.1-SNAPSHOT.jar dest=/home/weblogic/app/test-loan-server/test-loan-server-0.0.1-SNAPSHOT.jar'"
sh "ansible mmt-naip-testloan-1 -m shell -a 'sh /home/weblogic/script/deploy_naip.sh testloan '"
}
}
stage('test-loan-server-192.168.20.187') {
when {
allOf{
environment name: 'module', value: 'test-loan-server'
environment name: 'ip', value: '192.168.20.187'
}
}
steps {
sh "ansible mmt-naip-testloan-2 -m synchronize -a 'mode=push src=$WORKSPACE/openapi/test-loan-server/target/test-loan-server-0.0.1-SNAPSHOT.jar dest=/home/weblogic/app/test-loan-server/test-loan-server-0.0.1-SNAPSHOT.jar'"
sh "ansible mmt-naip-testloan-2 -m shell -a 'sh /home/weblogic/script/deploy_naip.sh testloan '"
}
}
}
}
}
// The following section configures mail notifications, including the use of the jenkins default variable.
post {
success {
emailext (
subject: "'${env.JOB_NAME} [${env.BUILD_NUMBER}]' Update normal",
body: """
//Details:
SUCCESSFUL: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'
//Status: ${env.JOB_NAME} jenkins update is working properly
URL : ${env.BUILD_URL}
//Project name: ${env.JOB_NAME}
//Project update schedule: ${env.BUILD_NUMBER}
//Code branch: ${env.brach}
//Business server IP: ${env.ip}
//Publishing module: ${env.module}
""",
to: "test@maimaiti.cn"
recipientProviders: [[$class: 'DevelopersRecipientProvider']]
)
}
failure {
emailext (
subject: "'${env.JOB_NAME} [${env.BUILD_NUMBER}]' Update failed",
body: """
//Details:
FAILED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'
//Status: ${env.JOB_NAME} jenkins failed to run
URL : ${env.BUILD_URL}
//Project name: ${env.JOB_NAME}
//Project update schedule: ${env.BUILD_NUMBER}
//Code branch: ${env.brach}
//Business server IP: ${env.ip}
//Publishing module: ${env.module}
""",
to: "gaoyang@test.cn",
recipientProviders: [[$class: 'DevelopersRecipientProvider']]
)
}
}
}
6. Backup and restore of Jenkins:
For more details, please pay attention to my personal Wechat public number "IT Operations and Maintenance in the Cloud Age". This public number aims to share new technologies and trends of Internet Operations and Maintenance, including IT Operations and Maintenance Industry consultation, Operations and Maintenance Technology Document Sharing. Focus on devops, jenkins, zabbix monitoring, kubernetes, ELK, the use of various middleware, such as redis, MQ, shell and python and other operational and maintenance programming languages; I have been engaged in IT operations and maintenance related work for more than 10 years. Since 2008, I have been engaged in Linux/Unix system operation and maintenance, and have a certain understanding of operation and maintenance related technologies. All the blog posts of this public number are the summary of my practical work experience, basically original blog posts. I would like to share my accumulated experience, experience and technology with you. I hope to grow and progress with you on the IT operation and maintenance career path.
