Environmental Science:
- 202.106.0.17: as Client
- 202.106.0.27: as router
- 192.168.205.37: as lvs1
- 192.168.205.47: as lvs2
- 192.168.205.57: as websrv1
- 192.168.205.67: as websrv2
- 192.168.205.77: as websrv3
- 192.168.205.87: as websrv4
Note: Firealld is disabled by default for all operating systems, IPtable is empty, selinux is closed
Edition:
- OS: centos 7 1810 with mini install
- keepalived
- httpd
Purpose:
Using keepalived to monitor four web servers, two servers are in a group, each using a virtual IP address, lvs1 as the master of vip2, lvs2 as the master of vip2, and vip1 as the master of each other, and using LVS as load balancing, the service down loader can be found automatically and the fault host can be removed automatically, so as to achieve fault-free scheduling.
Configure router
- Turn on routing forwarding
[root@router data]#ech 1 > /proc/sys/net/ipv4/ip_forward
- Adding a 10.1.1.1 to eth1 of router, because I have to have routing when I want to go to the LVS 1 and lvs2 servers, because the virtual IP of two hosts is likely to drift, so when one of them down s, it is impossible to determine which host is on. In order to be safe and not routing, I directly connect with IP.
[root@router data]#ip a a 10.1.1.1/24 dev eth0:1
Configure four real server s first
-
Run the following script on two servers of app1 192.168.205.57/67
[root@websrv1 data]#cat lvs_dr_rs.sh #!/bin/bash vip=10.1.1.100 gateway=192.168.205.27 mask='24' dev=lo:1 rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null service httpd start &> /dev/null && echo "The httpd Server is Ready!" echo "this is `hostname`" > /var/www/html/index.html case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ip a a $vip/$mask dev $dev #broadcast $vip up ip route add default via $gateway dev eth0 #route add -host $vip dev $dev echo "The RS Server is Ready!" ;; stop) systemctl restart network echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce echo "The RS Server is Canceled!" ;; *) echo "Usage: $(basename $0) start|stop" exit 1 ;; esac -
Run the following script on two app2 servers 192.168.205.77/87
[root@websrv3 data]#cat lvs_dr_rs.sh #!/bin/bash vip=10.1.1.200 gateway=192.168.205.27 mask='24' dev=lo:1 rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null service httpd start &> /dev/null && echo "The httpd Server is Ready!" echo "this is `hostname`" > /var/www/html/index.html case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ip address add $vip/$mask dev $dev #broadcast $vip up ip route add default via $gateway dev eth0 #route add -host $vip dev $dev echo "The RS Server is Ready!" ;; stop) systemctl restart network echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce echo "The RS Server is Canceled!" ;; *) echo "Usage: $(basename $0) start|stop" exit 1 ;; esacDo the same action on two lvs (except for special instructions)
- Install httpd service and place sorry server on two local hosts
yum install httpd echo server is under maitenance > /var/www/html/index.html
- Be sure to add the default route to router. If it is correct, it will work, but sorry server will not work.
ip route add default via 192.168.205.27 dev eth0
- In order to access conveniently, it is better to do ssh key verification between two lvs servers. The following method does not need to do it again on 47, and realizes mutual authentication authentically.
ssh-keygen ssh-copy-id 127.0.0.1 scp -r /root/.ssh 192.168.205.47:/root
- It's better to add the host file to the name resolution of two hosts.
vi /etc/hosts 192.168.205.37 websrv1 192.168.205.47 websrv2 scp /etc/hosts 192.168.205.47:/etc
- Install keepalive. In order to see how to add the lvs policy, we installed iplvadm as well.
yum install keepalived ipvsadm
- Modify the configuration file on lvs1
[root@lsv1 ~]#vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalive@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS1 vrrp_mcast_group4 224.0.0.100 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 37 priority 100 advert_int 1 authentication { auth_type PASS auth_pass centos } virtual_ipaddress { 10.1.1.100/24 dev eth0 label eth0:0 } } vrrp_instance VI_2 { state BACKUP interface eth0 virtual_router_id 47 priority 80 advert_int 1 authentication { auth_type PASS auth_pass centos } virtual_ipaddress { 10.1.1.200/24 dev eth0 label eth0:1 } } virtual_server 10.1.1.100 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.205.57 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.205.67 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } virtual_server 10.1.1.200 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.205.77 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.205.87 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } 11. For convenience lsv1 Of keepalive.conf Copy to lvs2 Up and revise [root@lvs2 ~]#vi /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from keepalive@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS2 vrrp_mcast_group4 224.0.0.100 } vrrp_instance VI_1 { state BACKUP interface eth0 virtual_router_id 37 priority 80 advert_int 1 authentication { auth_type PASS auth_pass centos } virtual_ipaddress { 10.1.1.100/24 dev eth0 label eth0:0 } } vrrp_instance VI_2 { state MASTER interface eth0 virtual_router_id 47 priority 100 advert_int 1 authentication { auth_type PASS auth_pass centos } virtual_ipaddress { 10.1.1.200/24 dev eth0 label eth0:1 } } virtual_server 10.1.1.100 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.205.57 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.205.67 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } virtual_server 10.1.1.200 80 { delay_loop 6 lb_algo rr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.205.77 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.205.87 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } - Start the keepalived service
systemctl start keepalived
- See RR Scheduling in lvs1
[root@lsv1 ~]#ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.100:80 rr -> 192.168.205.57:80 Route 1 0 0 -> 192.168.205.67:80 Route 1 0 0 TCP 10.1.1.200:80 rr -> 192.168.205.77:80 Route 1 0 0 -> 192.168.205.87:80 Route 1 0 0
- Only 10.1.1.100 IP can be seen in LVS 1, and only 10.1.1.200 IP can be seen in Lvs2.
[root@lsv1 ~]#ip a 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:56:e1:ea brd ff:ff:ff:ff:ff:ff inet 192.168.205.37/24 brd 192.168.205.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet 10.1.1.100/24 scope global eth0:0 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe56:e1ea/64 scope link noprefixroute valid_lft forever preferred_lft forever [root@lvs2 ~]#ip a 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:37:f9:93 brd ff:ff:ff:ff:ff:ff inet 192.168.205.47/24 brd 192.168.205.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet 10.1.1.200/24 scope global eth0:1 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe37:f993/64 scope link noprefixroute valid_lft forever preferred_lft forevertest
- Running a loop on client for testing
[root@client ~]#while : ;do curl 10.1.1.100; sleep 0.5; done this is websrv1 this is websrv2 this is websrv1 this is websrv2 this is websrv1 this is websrv2 [root@client ~]#while : ;do curl 10.1.1.200; sleep 0.5; done this is websrv4 this is websrv3 this is websrv4 this is websrv3 this is websrv4
- Stop web1, test again, and find that only web2 will be scheduled.
[root@websrv1 data]#systemctl stop httpd [root@client ~]#while : ;do curl 10.1.1.100; sleep 0.5; done this is websrv2 this is websrv2 this is websrv2 this is websrv2 this is websrv2
- Stop web2, test again, find that sorry server takes over, and you can see 127.0.0.1 load on lvs1
[root@websrv2 ~]#systemctl stop httpd [root@client ~]#while : ;do curl 10.1.1.100; sleep 0.5; done server under maintenance server under maintenance server under maintenance server under maitnenance [root@lsv1 ~]#ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.1.1.100:80 rr -> 127.0.0.1:80 Route 1 0 4 TCP 10.1.1.200:80 rr -> 192.168.205.77:80 Route 1 0 0 -> 192.168.205.87:80 Route 1 0 0
- Restore two websrv1 and websrv2, and stop lvs2, found no impact, but you can see that both VIPs are back on lvs1
[root@websrv1 ~]#systemctl start httpd [root@websrv2 ~]#systemctl start httpd [root@lvs2 ~]#systemctl stop keepalived [root@client ~]#while : ;do curl 10.1.1.100; sleep 0.5; done this is websrv2 this is websrv1 this is websrv2 this is websrv1 [root@client ~]#while : ;do curl 10.1.1.200; sleep 0.5; done this is websrv3 this is websrv4 this is websrv3 this is websrv4 [root@lsv1 ~]#ip a 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:56:e1:ea brd ff:ff:ff:ff:ff:ff inet 192.168.205.37/24 brd 192.168.205.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet 10.1.1.100/24 scope global eth0:0 valid_lft forever preferred_lft forever inet 10.1.1.200/24 scope global secondary eth0:1 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe56:e1ea/64 scope link noprefixroute valid_lft forever preferred_lft forever - Restore lvs2 to its original state because preemption returns to primary and backup
[root@client ~]#while : ;do curl 10.1.1.100; sleep 0.5; done this is websrv2 this is websrv1 this is websrv2 [root@client ~]#while : ;do curl 10.1.1.200; sleep 0.5; done this is websrv4 this is websrv3 this is websrv4 [root@lsv1 ~]#ip a 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:56:e1:ea brd ff:ff:ff:ff:ff:ff inet 192.168.205.37/24 brd 192.168.205.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet 10.1.1.100/24 scope global eth0:0 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe56:e1ea/64 scope link noprefixroute valid_lft forever preferred_lft forever [root@lvs2 ~]#ip a 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:37:f9:93 brd ff:ff:ff:ff:ff:ff inet 192.168.205.47/24 brd 192.168.205.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever inet 10.1.1.200/24 scope global eth0:1 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fe37:f993/64 scope link noprefixroute valid_lft forever preferred_lft forever