Before we begin, let's review the traditional code deployment methods in the past.
Usually, the operation and maintenance personnel have to make a lot of preparations before receiving the task of launching the code (new project), including physical host, virtual machine, code running environment, database installation and configuration, creation of various accounts, system monitoring in the later stage of operation, application log collection, performance optimization and so on.
Think about it. This process is not very complicated, but it is cumbersome and inefficient. If debugging is needed, it also needs to provide developers with online system permissions, etc. if you don't pay attention to the details, it will also cause various problems such as the difficulty of solving the problem.
OK, after finishing the above problems, there will be corresponding solutions.
General structure of the scheme:
Jenkins+Gitlab+Harbor+Rancher+k8s
Functional description of each component
Jenkins
(1) Download the project code in gitlab
(2) Load execution image construction, upload and download
(3) Deploy to k8s cluster
Gitlab
(1) Project code and configuration
(2) Dockerfile file
Harbor
This is the open source docker image warehouse management system of vmware company, which is more convenient to manage and maintain images
(1) Responsible for the storage of the mirror after construction
Rancher
Container orchestration management tool
(1) Update stack/service
(2) Realize the expansion and contraction of services
k8s
(1) Simplify application deployment (2) improve hardware resource utilization (3) health check and self repair (4) automatic capacity expansion and reduction (5) service discovery and load balancing
Architecture diagram
Architecture diagram description
The project development language is java, which uses the popular spring boot framework. The manven update source is Alibaba cloud, which compiles and generates jar files
① Developers submit code to gitlab
② Manually execute jenkins build (or gitlab hook triggers jenkins to execute build), download the latest version of code, which contains Dockerfile
③ jenkins executes shell script: mvn compiles and generates jar files. Package into image through docker build instruction
④ Upload the built image and push it to the harbor image warehouse
⑤ jenkins remotely accesses the k8s master node and updates the service image address to upgrade the container (that is, update the code version).
The above process completely realizes CI/CD. Here, jenkins is one of the key positions.
Environmental description
| system | ip | host name | to configure | edition |
|---|---|---|---|---|
| CentOS 7.6 | 10.212.20.94 | k8s-master | 2-core 4g | Kubernetes1.18.1 |
| CentOS 7.6 | 10.212.20.240 | k8s-node01 | 2-core 4g | Kubernetes1.18.1 |
| CentOS 7.6 | 10.212.82.89 | jenkins | 2-core 4g | 2.222.4 |
| CentOS 7.6 | 10.212.82.90 | gitlab | 2-core 4g | 10.5.1 |
| CentOS 7.6 | 10.212.82.86 | harbor | 2-core 4g | v2.0.0 |
| CentOS 7.6 | 10.212.82.87 | rancher | 2-core 4g | v2.4.3 |
For k8s 1.18.1 installation, please refer to the link:
https://www.cnblogs.com/xiao987334176/p/12696740.html
For jenkins installation, please refer to the link:
https://www.cnblogs.com/xiao987334176/p/13032339.html
For Gitlab and Harbor installation, please Baidu by yourself
For the installation of rancher and the import of existing k8s clusters, please refer to the link:
https://www.cnblogs.com/xiao987334176/p/12965945.html
Project description
General background management system based on Spring Boot/Spring Security/thymeleaf
Project address:
https://github.com/jonsychen/admin
This project depends on mysql, so you need to deploy mysql in Rancher in advance.
mysql deployment
Log in to k8s-node01 host and create data directory
mkdir \-p /data/mysql/data
Visit the Harbor background and click deploy service
Port mapping
Setting environment variables
TZ\=Asia/Shanghai MYSQL\_ROOT\_PASSWORD\=abcd@1234
Data volume mapping
Click Start
Code configuration
Download code: https://github.com/jonsychen/admin
After decompression, enter the directory admin master \ SRC \ main \ resources
Modify application default Yaml, modify the red part.
server:
port: 8088
compression:
enabled: true
connection\-timeout: 3000
debug: false
##Log in and remember my token encryption key
remember:
key: yintong
##Actor config. The actor runs in a separate webappcontext. see AnnotationConfigEmbeddedWebApplicationContext
management:
context\-path: /management
security:
enabled: false
spring:
application:
name: admin
datasource:
url: jdbc:mysql://db\-mysql.default.svc.cluster.local:3306/admin?characterEncoding\=utf\-8
username: root
password: abcd@1234
explain:
Port: the port number on which the 8088 project runs
db-mysql.default.svc.cluster.local indicates the SVC address of the DB MySQL service.
Format Description: service name Command space default.svc.cluster.local, where the service name and namespace are based on the actual situation, and the latter part is fixed. This long string of domain names will be resolved to SVC addresses.
password: abcd@1234 mysql root user password
The configuration modification of application-prod.yaml is the same as above.
Create a dockerfile at the root of this project
FROM mayan31370/openjdk\-alpine\-with\-chinese\-timezone:8\-jdk ADD admin\-0.1.0.jar / EXPOSE 8088 ENTRYPOINT \[ "java","-jar","/admin-0.1.0.jar"\]
The top-level directory structure is as follows:
\# tree \-L 1 . ├── dockerfile ├── etc ├── pom.xml ├── README.md └── src
Submit the project code to gitlab.
sql import
Log in to the k8s master node and view the svc mapping port. Because the nodeport port is random when deploying mysql with Rancher.
\# kubectl get svc NAME TYPE CLUSTER\-IP EXTERNAL\-IP PORT(S) AGE db\-mysql ClusterIP 10.1.116.0 <none\> 3306/TCP 61s db\-mysql\-nodeport NodePort 10.1.86.36 <none\> 3306:31959/TCP 61s kubernetes ClusterIP 10.1.0.1 <none\> 443/TCP 20d
You can see that the randomly mapped port is 31959
Connect mysql with navicat software
New database book
create database admin default character set utf8mb4 collate utf8mb4\_unicode\_ci;
Enter book and execute the SQL file in the project. The path is: etc / DDL sql
After successful execution, the table is as follows:
Harbor configuration
Create a new java project. The access level is public. Note: after setting public, downloading images does not require authentication.
Push image
Enter the jenkins host and download the gitlab code.
git clone ssh://git@10.212.20.94:/home/git/git\_storage/admin\-master
Modify docker configuration and add Harbor library address.
vim /etc/docker/daemon.json
Add secure registers
{"insecure-registries": \["192.168.10.122"\]}
Restart service
systemctl restart docker
Log in to Harbor, otherwise the image cannot be pushed
docker login 10.212.82.86:1180
Enter the project directory and compile the code using mvn
mvn \-f pom.xml clean package
To push an image, execute the following command:
cp dockerfile target/cd target docker build \-t 10.212.82.86:1180/java/admin\-master:1 . docker push 10.212.82.86:1180/java/admin\-master:1docker rmi 10.212.82.86:1180/java/admin\-master:1
Admin master deployment
Login k8s-node01 node
Modify docker configuration and add Harbor library address.
vim /etc/docker/daemon.json
Add secure registers
{"insecure-registries": \["192.168.10.122"\]}
Restart service
systemctl restart docker
Visit the Rancher background and click deploy service
Port mapping
Click Start
Wait a few minutes and Running indicates normal operation.
Log in to the k8s master node and view the svc mapping port. Because when using Rancher deployment, the nodeport port is random.
\# kubectl get svc NAME TYPE CLUSTER\-IP EXTERNAL\-IP PORT(S) AGE admin\-master ClusterIP 10.1.10.23 <none\> 8088/TCP 10m admin\-master\-nodeport NodePort 10.1.238.46 <none\> 8088:31581/TCP 10m db\-mysql ClusterIP 10.1.116.0 <none\> 3306/TCP 61s db\-mysql\-nodeport NodePort 10.1.86.36 <none\> 3306:31959/TCP 61s kubernetes ClusterIP 10.1.0.1 <none\> 443/TCP 20d
You can see that the random port mapped by nodeport is 31581
Access admin master
http://10.212.20.94:31581/
Enter the user name and password, both of which are root
The effect of home page is as follows:
Previously, mysql and admin master have been deployed through Rancher. Next, we will demonstrate how to realize Pipeline based publishing and rollback through jenkins.
Install the plug-in Git Parameter
Create a new job named test_admin, select pipeline.
General settings
Parametric construction
Pipeline script
The complete code is as follows:
env.CREDENTIALSID \= '7a294fc5-2b2b-4d2d-92ff-54324e1b032a'
env.BRANCHES \= 'master'
env.GIT\_URL \= 'ssh://git@10.212.20.94:/home/git/git\_storage/admin-master'
env.HARBOR\_PROJECT \= '10.212.82.86:1180/java/admin-master'
env.PROJECT \= 'admin-master'
env.K8S\_MASTER \= '10.212.20.94'
env.NAMESPACE \= 'default'
node {
if (env.Status \== 'Deploy'){
stage('code pull') {
checkout(\[$class: 'GitSCM', branches: \[\[name: env.BRANCHES\]\],
doGenerateSubmoduleConfigurations: false,
userRemoteConfigs: \[\[credentialsId: env.CREDENTIALSID, url: env.GIT\_URL\]\]\])
}
stage('code Build') {
sh 'mvn -f pom.xml clean package'
}
stage('docker push') {
sh 'cd ${WORKSPACE} && cp dockerfile target'
sh 'cd ${WORKSPACE}/target && docker build -t ${HARBOR\_PROJECT}:${BUILD\_NUMBER} .'
sh 'docker push ${HARBOR\_PROJECT}:${BUILD\_NUMBER}'
sh 'docker rmi ${HARBOR\_PROJECT}:${BUILD\_NUMBER}'
}
stage('k8s deploy') {
sh 'ssh ${K8S\_MASTER} "kubectl -n ${NAMESPACE} set image deploy ${PROJECT} \*=${HARBOR\_PROJECT}:${BUILD\_NUMBER}"'
}
}else{
stage('k8s rollback') {
sh 'ssh ${K8S\_MASTER} "kubectl -n ${NAMESPACE} set image deploy ${PROJECT}\*=${HARBOR\_PROJECT}:${BUILD\_ID}"'
}
}
}
According to the actual situation, modify the red text parameter, that is, the global variable.
explain:
7a294fc5-2b2b-4d2d-92ff-54324e1b032a this is the jenkins global credential, which is set for gitlab.
Click build
Direct click build
After successful construction, the effects are as follows:
If you need to roll back to the previous version, enter BUILD_ID.
Due to the last successful BUILD_ID is 10, so enter 9
The effect is as follows:
Reference link of this article: