Image production specification
First create the most common layer, and each subsequent image depends on the previous parent image.
Build images hierarchically and deploy tomcat high availability
tomcat1, nginx1, haproxy1 and keepalived1 are deployed on the 10.0.0.51 node
tomcat2, nginx2, haproxy2 and keepalived2 are deployed on the 10.0.0.52 node
Image construction is mainly performed at node 51.
1. Create a basic CentOS base image
1.1 create image classification directory structure
$ mkdir /opt/dockerfile/{web/{nginx,tomcat,jdk,apache},system/{centos,ubuntu,redhat}} -pv
1.2 write CentOS base image dockerfile.
$ cd /opt/dockerfile/system/centos
$ mkdir 7.9 && cd 7.9
$ vim Dockerfile
FROM centos:7.9.2009
LABEL author=LiangDong
RUN rpm -ivh http://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm \
&& yum update -y \
&& yum install -y vim wget tree lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop \
&& groupadd -g 2021 maple \
&& useradd -u 2021 -g 2021 maple
1.3 write the build command to sh to facilitate subsequent mirroring.
$ cat build-command.sh #!/bin/bash docker build -t harbor.maple.com/oss/centos-base:v7.9.2009 .
1.4 build image and test
$ bash build-command.sh $ docker run -it --rm harbor.maple.com/oss/centos-base:v7.9.2009 bash # Check whether the user is created and whether there are commands such as vim
2. Build jdk8 image from CentOS base image
2.1 upload jdk8u291 package
$ cd /opt/dockerfile/web/jdk $ mkdir jdk8u291 && cd jdk8u291 # Upload jdk8u291 package $ ls jdk-8u291-linux-x64.tar.gz
2.2 writing jdk Dockerfile
$ vim Dockerfile
FROM harbor.maple.com/oss/centos-base:v7.9.2009
MAINTAINER "LiangDong 395539184@qq.com"
ADD jdk-8u291-linux-x64.tar.gz /usr/local/src/
RUN ln -s /usr/local/src/jdk1.8.0_291 /usr/local/jdk \
&& echo "export JAVA_HOME=/usr/local/jdk" > /etc/profile.d/jdk.sh \
&& echo "export JRE_HOME=\$JAVA_HOME/jre" >> /etc/profile.d/jdk.sh \
&& echo "export CLASSPATH=\$JAVA_HOME/lib/:\$JRE_HOME/lib/" >> /etc/profile.d/jdk.sh \
&& echo "export PATH=\$PATH:\$JAVA_HOME/bin" >> /etc/profile.d/jdk.sh \
&& source /etc/profile.d/jdk.sh \
&& rm -rf /etc/localtime \
&& ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
2.3 build and test run
$ ls build-command.sh Dockerfile jdk-8u291-linux-x64.tar.gz $ bash build-command.sh $ docker run -it --rm harbor.maple.com/oss/centos-base-jdk:v1.8.291 bash # Test whether the date is consistent with the local time zone # Test whether the root user has java environment variables and can execute Java commands # Test whether the maple user has java environment variables and can execute Java commands
3. Build tomcat8 image based on jdk8 image
3.1 uploading tomcat package
$ cd /opt/dockerfile/web/tomcat/8.5.64 $ ls apache-tomcat-8.5.64.tar.gz build-command.sh Dockerfile
3.2 create Dockerfile of tomcat
$ cat Dockerfile FROM harbor.maple.com/oss/centos-base-jdk:v1.8.291 LABEL author="LiangDong 395539184@qq.com" ADD apache-tomcat-8.5.64.tar.gz /apps/ RUN ln -s /apps/apache-tomcat-8.5.64 /apps/tomcat
3.3 build and test
$ bash build-command.sh $ docker run -it --rm -p 8080:8080 harbor.maple.com/oss/tomcat:v8.5.64 bash # /apps/tomcat/bin/catalina.sh start # Page access 8080 test
4. Build business image based on tomcat8 image 1
4.1 preparation of configuration files and test page files
$ cd /opt/dockerfile/web/tomcat/myapp1 $ mkdir myapp1 # Prepare page test file $ echo "<h1>tomcat myapp1</h1>" > myapp1/index.jsp $ tar cvf myapp1.tar.gz myapp1/ $ rm -rf myapp1 # Prepare the tomcat configuration file and modify the tomcat project directory $ docker run -it --rm -p 8080:8080 harbor.maple.com/oss/tomcat:v8.5.64 bash # Another terminal performs replication docker cp 9070a1b76dce:/apps/tomcat/conf/server.xml . # Modify the tomcat project deployment directory to / data/tomcat/webapps # Modification does not automatically decompress and deploy, which is easy to attack. vim server.xml <Host name="localhost" appBase="/data/tomcat/webapps" unpackWARs="false" autoDeploy="false">
4.2 prepare the tomcat running script so that the tomcat service is not a process with container PID 1.
$ vim run_tomcat.sh #!/bin/bash echo "nameserver 223.5.5.5" > /etc/resolv.conf # Avoid running tomcat as root su - maple -c "/apps/tomcat/bin/catalina.sh start" su - maple -c "tail -f /etc/hosts"
$ chmod +x run_tomcat.sh # needs execution permission, otherwise the script cannot be executed in the container
4.3 write Dockerfile file of myapp1
$ vim Dockerfile
FROM harbor.maple.com/oss/tomcat:v8.5.64
ENV TOMCAT_HOME /apps/tomcat
COPY server.xml $TOMCAT_HOME/conf/server.xml
COPY run_tomcat.sh $TOMCAT_HOME/bin/
ADD myapp1.tar.gz /data/tomcat/webapps/
RUN chown -R maple.maple $TOMCAT_HOME/ \
&& chown -R maple.maple /data/tomcat
CMD ["/bin/bash","-c","$TOMCAT_HOME/bin/run_tomcat.sh"]
be careful:
# The following chown command failed to be modified, but it is strange to modify / data/tomcat chown -R maple.maple $TOMCAT_HOME #Successfully added a / modifiable permission chown -R maple.maple $TOMCAT_HOME/
4.4 build and test
$ bash build-command.sh $ docker run -it --rm -p 8080:8080 harbor.maple.com/oss/tomcat-myapp1:v1
5. Create business image 2
$ cd /opt/dockerfile/web/tomcat $ cp myapp1 -r myapp2 $ cd myapp2 # Modify build command $ vim build-command.sh docker build -t harbor.maple.com/oss/tomcat-myapp2:v1 . # Modify test page content $ tar xf myapp1.tar.gz $ vim myapp1/index.jsp <h1>tomcat myapp2</h1> $ tar cf myapp1.tar.gz myapp1 $ rm -rf myapp1 # structure $ bash build-command.sh # Run test $ docker run -it --rm -p 8080:8080 harbor.maple.com/oss/tomcat-myapp2:v1
Import to another docker host
$ docker save harbor.maple.com/oss/tomcat-myapp2 > myapp2.tar $ scp myapp2.tar 10.0.0.52:/home/maple/
Another host is loaded and running
$ docker load -i myapp2.tar $ docker run -it --rm -p 8080:8080 harbor.maple.com/oss/tomcat-myapp2:v1
6. Create nginx image
6.1 first build a test nginx to obtain nginx Conf configuration file
$ cd /opt/dockerfile/web/nginx
$ ls
Dockerfile index.html nginx-1.20.1.tar.gz
$ vim Dockerfile
FROM harbor.maple.com/oss/centos-base:v7.9.2009
MAINTAINER liangsircc@gmail.com
ADD nginx-1.20.1.tar.gz /usr/local/src
RUN yum install -y make vim wget tree lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop \
&& cd /usr/local/src/nginx-1.20.1 \
&& ./configure --prefix=/usr/local/nginx --with-http_sub_module \
&& make \
&& make install
COPY index.html /usr/local/nginx/html/index.html
RUN useradd -s /sbin/nologin nginx \
&& ln -s /usr/local/nginx/sbin/nginx /usr/sbin/nginx
CMD ["nginx","-g","daemon off;"]
$ docker cp 6b8e2d2a5f19:/usr/local/nginx/conf/nginx.conf . $ ls Dockerfile index.html nginx-1.20.1.tar.gz nginx.conf
6.2 modifying configuration files
$ vim nginx.conf
user maple maple;
http {
up_stream myapp1 {
server 10.0.0.51:8080;
server 10.0.0.52:8080;
}
server {
location / {
root html;
index index.html index.htm;
}
location /myapp1 {
proxy_pass http://myapp1;
}
}
}
6.3 modify Dockerfile and add nginx configuration file
$ vim Dockerfile
FROM harbor.maple.com/oss/centos-base:v7.9.2009
MAINTAINER liangsircc@gmail.com
ADD nginx-1.20.1.tar.gz /usr/local/src
RUN yum install -y make vim wget tree lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop \
&& cd /usr/local/src/nginx-1.20.1 \
&& ./configure --prefix=/usr/local/nginx --with-http_sub_module \
&& make \
&& make install
COPY index.html /usr/local/nginx/html/index.html
ADD nginx.conf /usr/local/nginx/conf/
RUN ln -s /usr/local/nginx/sbin/nginx /usr/sbin/nginx
CMD ["nginx","-g","daemon off;"]
6.4 build run test
$ bash build-command.sh $ docker run -p 8888:80 -d harbor.maple.com/oss/nginx:v1
You can see that the work process is started by ordinary users
7. Copy the image to another host to run nginx2
$ docker save harbor.maple.com/oss/nginx:v1 > nginx.tar $ scp nginx.tar 10.0.0.52:/home/maple/ #Another node is loaded and running $ docker load -i nginx.tar $ docker run -p 8888:80 -d harbor.maple.com/oss/nginx:v1
test
8. Configure haproxy image 1
8.1 prepare the configuration file first
$ cd /opt/dockerfile/web/haproxy $ ls build-command.sh Dockerfile haproxy-2.4.1.tar.gz haproxy.cfg run_haproxy.sh
8.2 preparing haproxy configuration files
$ cat haproxy.cfg
global
chroot /usr/local/haproxy
stats socket /var/lib/haproxy/haproxy.sock mode 600 level admin
uid 99
gid 99
daemon
nbproc 1
pidfile /usr/local/haproxy/run/haproxy.pid
log 127.0.0.1 local3 info
defaults
option http-keep-alive
option forwardfor
mode http
timeout connect 300000ms
timeout client 300000ms
timeout server 300000ms
listen stats
mode http
bind 0.0.0.0:9999
stats enable
log global
stats uri /haproxy-status
stats auth haadmin:123456
listen web_port
bind 0.0.0.0:80
mode http
log global
balance roundrobin
server web1 10.0.0.51:8888 check inter 3000 fall 2 rise 5
server web2 10.0.0.52:8888 check inter 3000 fall 2 rise 5
8.3 preparation run_haproxy.sh script
$ cat run_haproxy.sh #!/bin/bash /usr/local/haproxy/sbin/haproxy -f /etc/haproxy/haproxy.cfg tail -f /etc/hosts $ chmod +x run_haproxy.sh
8.4 dockerfile file
$ cat Dockerfile
FROM harbor.maple.com/oss/centos-base:v7.9.2009
ADD haproxy-2.4.1.tar.gz /usr/local/src/
ADD haproxy.cfg /etc/haproxy/
ADD run_haproxy.sh /usr/bin/
RUN yum install -y yum install gcc gcc-c++ glibc glibc-devel pcre pcre-devel openssl openssl-devel systemd-devel net-tools vim iotop bc zip unzip zlib-devel lrzsz tree screen lsof tcpdump wget ntpdate \
&& cd /usr/local/src/haproxy-2.4.1 \
&& make ARCH=x86_64 TARGET=linux-glibc USE_PCRE=1 USE_OPENSSL=1 USE_ZLIB=1 \
USE_SYSTEMD=1 USE_CPU_AFFINITY=1 PREFIX=/usr/local/haproxy \
&& make install PREFIX=/usr/local/haproxy \
&& cp haproxy /usr/sbin/ \
&& mkdir /usr/local/haproxy/run \
&& mkdir /var/lib/haproxy
EXPOSE 80 9999
CMD ["/usr/bin/run_haproxy.sh"]
8.5 Build run test
$ docker run -d -p 80:80 -p 9999:9999 harbor.maple.com/oss/haproxy:v1
9. Configure haproxy image 2
$ docker save harbor.maple.com/oss/haproxy:v1 > haproxy.tar $ scp haproxy.tar 10.0.0.52:/home/maple/ #Another host loads and starts the container $ docker load -i haproxy.tar $ docker run -d -p 80:80 -p 9999:9999 harbor.maple.com/oss/haproxy:v1
10. Configure keepalived
Two ways
- The host runs keepalived
- When the container runs keepalived, you need to configure – network=host
10.1 host machine running keepalived
10.1.1 installation
$ sudo apt install -y keepalived
#create profile
$ cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
liangsircc@gmail.com
}
notification_email_from 395539184@qq.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ka1.maple.com
vrrp_skip_check_adv_addr
# vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_script check_haproxy {
script "/etc/keepalived/check_haproxy.sh"
interval 1
weight -40
fall 3
rise 2
timeout 2
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 66
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass admin
}
virtual_ipaddress {
10.0.0.50 dev eth0 label eth0:1
}
unicast_src_ip 10.0.0.51
unicast_peer {
10.0.0.52
}
track_script {
check_haproxy
}
}
#Create detection script
$ sudo vim /etc/keepalived/check_haproxy.sh
#!/bin/bash
if ss -tln | grep -E "\<80\>" &> /dev/null ;then
exit 0;
else
exit 1;
fi
#start-up
sudo systemctl start keepalived.service
10.1.2 another host
$ apt install -y keepalived
$ cat /etc/keepalived/keepalived.conf
global_defs {
notification_email {
liangsircc@gmail.com
}
notification_email_from 395539184@qq.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ka1.maple.com
vrrp_skip_check_adv_addr
# vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 66
priority 70
advert_int 1
authentication {
auth_type PASS
auth_pass admin
}
virtual_ipaddress {
10.0.0.50 dev eth0 label eth0:1
}
unicast_src_ip 10.0.0.52
unicast_peer {
10.0.0.51
}
}
#start-up
$ sudo systemctl start keepalived.service
You don't need to modify haproxy,
Because the haproxy configuration listens to 0.0.0.0, which contains the vip address
Test stop haproxy container
docker stop 07e3d40467c0
Observe whether vip is switched to another host 10.0.0.52
Whether the website is accessed normally
10.2 container operation kept alive
10.2.1 prepare package and configuration file first host
$ sudo systemctl stop keepalived.service #Stop the keepalived of the host first
$ cd /opt/dockerfile/web/keepalived
$ sudo cp /etc/keepalived/* .
# Build command
$ cat build-command.sh
#!/bin/bash
docker build -t harbor.maple.com/oss/keepalived:v1 .
#Detection script
$ cat check_haproxy.sh
#!/bin/bash
if ss -tln | grep -E "\<80\>" &> /dev/null ;then
exit 0;
else
exit 1;
fi
$ ls
build-command.sh check_haproxy.sh Dockerfile keepalived.conf
10.2.2 building dockerfile
$ cat Dockerfile FROM harbor.maple.com/oss/centos-base:v7.9.2009 RUN yum install -y keepalived ADD check_haproxy.sh /etc/keepalived/ ADD keepalived.conf /etc/keepalived/ CMD keepalived -f /etc/keepalived/keepalived.conf -l -n -D --vrrp
10.2.3 build and start the container
$ bash build-command.sh # -- privileged needs to be added to enable the container to have permission to operate the host # You need -- network=host to share the same network card between the container and the host $ docker run -d --privileged --network=host harbor.maple.com/oss/keepalived:v1 # VIP 50 has been configured. $ hostname -I 10.0.0.51 10.0.0.50 172.22.16.1 172.18.0.1 172.17.0.1
10.2.4 create ka2 new directory, modify configuration and build image
10.2.4.1 preparation of configuration file
$ cd /opt/dockerfile/web/keepalived-backup $ ls build-command.sh Dockerfile keepalived.conf
10.2.4.2 Dockerfile
$ cat Dockerfile FROM harbor.maple.com/oss/centos-base:v7.9.2009 RUN yum install -y keepalived ADD keepalived.conf /etc/keepalived/ CMD keepalived -f /etc/keepalived/keepalived.conf -l -n -D --vrrp
10.2.4.3 modify keepalived Conf configuration file
$ cat keepalived.conf
global_defs {
notification_email {
liangsircc@gmail.com
}
notification_email_from 395539184@qq.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id ka1.maple.com
vrrp_skip_check_adv_addr
# vrrp_strict
vrrp_garp_interval 0
vrrp_gna_interval 0
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 66
priority 70
advert_int 1
authentication {
auth_type PASS
auth_pass admin
}
virtual_ipaddress {
10.0.0.50 dev eth0 label eth0:1
}
unicast_src_ip 10.0.0.52
unicast_peer {
10.0.0.51
}
}
10.2.4.4 export to another host
$ docker save harbor.maple.com/oss/keepalived-backup:v1 > keepalived-backup.tar $ scp keepalived-backup.tar 10.0.0.52:/home/maple/
10.2.4.5 load and start
$ docker load -i keepalived-backup.tar $ docker run -d --privileged --network=host harbor.maple.com/oss/keepalived-backup:v1
Test 10.0.0.51 the master node stops the haproxy container, vip automatically switches to 10.0.0.52 node, and the website is accessed normally.