18.11 LVS DR mode construction
preparation
Three machines have public IP.
- Scheduler (director)
IP: 192.168.230.135 - real server1(real1)
IP: 192.168.230.130 - real server2(real2)
IP: 192.168.230.145 - VIP: 192.168.230.200
Start building
Configure director
[root@cham002 ~]# vim /usr/local/sbin/lvs_dr.sh #! /bin/bash echo 1 > /proc/sys/net/ipv4/ip_forward ipv=/usr/sbin/ipvsadm vip=192.168.230.200 rs1=192.168.230.130 rs2=192.168.230.145 #Pay attention to the name of the network card here ifdown ens33 ifup ens33 ifconfig ens33:2 $vip broadcast $vip netmask 255.255.255.255 up route add -host $vip dev ens33:2 $ipv -C $ipv -A -t $vip:80 -s rr $ipv -a -t $vip:80 -r $rs1:80 -g -w 1 $ipv -a -t $vip:80 -r $rs2:80 -g -w 1 //Execute script: [root@cham002 ~]# sh /usr/local/sbin/lvs_dr.sh //Device 'ens33' disconnected successfully. //Successfully activated connection (D-Bus activation path / org/freedesktop/NetworkManager/ActiveConnection/6) [root@cham002 ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:b6:9f:e3 brd ff:ff:ff:ff:ff:ff inet 192.168.230.135/24 brd 192.168.230.255 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.230.200/32 brd 192.168.230.200 scope global ens33:2 valid_lft forever preferred_lft forever inet 192.168.230.150/24 brd 192.168.230.255 scope global secondary ens33:0 valid_lft forever preferred_lft forever inet6 fe80::6f15:52d3:ebeb:e193/64 scope link valid_lft forever preferred_lft forever 3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:b6:9f:ed brd ff:ff:ff:ff:ff:ff inet 192.168.118.147/24 brd 192.168.118.255 scope global ens37 valid_lft forever preferred_lft forever inet 192.168.100.1/24 brd 192.168.100.255 scope global ens37 valid_lft forever preferred_lft forever inet6 fe80::1801:cbbb:ebcc:89a3/64 scope link valid_lft forever preferred_lft forever
Note: VIP is bound to the ens33 network card.
Script interpretation
#! /bin/bash echo 1 > /proc/sys/net/ipv4/ip_forward #Turn on port forwarding ipv=/usr/sbin/ipvsadm vip=192.168.230.200 rs1=192.168.230.130 rs2=192.168.230.145 #Pay attention to the name of the network card here ifdown ens33 ifup ens33 #The purpose of restarting the network card here is to avoid repeatedly setting the IP provided by the command line ifconfig ens33:2 $vip broadcast $vip netmask 255.255.255.255 up #Virtual network card ens33:2 bound VIP to dir route add -host $vip dev ens33:2 #Add gateway $ipv -C $ipv -A -t $vip:80 -s wrr $ipv -a -t $vip:80 -r $rs1:80 -g -w 1 $ipv -a -t $vip:80 -r $rs2:80 -g -w 1 #Set ipvsadm rule, - g=gateway: use default gateway (DR mode)
Configure real server
Configure the following scripts in real1 and real2 respectively:
##real1(130) [root@cham1 ~]# vi /usr/local/sbin/lvs_rs.sh #/bin/bash vip=192.168.230.200 #The purpose of binding vip to lo is to realize rs to directly return the result to the client ifdown lo ifup lo ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up route add -host $vip lo:0 #The following operation is to change the arp kernel parameters so that rs can send the mac address to the client smoothly #Reference document: www.cnblogs.com/lgfeng/archive/2012/10/16/2726308.html echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce //Execute script: [root@cham1 ~]# vi /usr/local/sbin/lvs_rs.sh [root@cham1 ~]# sh !$ sh /usr/local/sbin/lvs_rs.sh [root@cham1 ~]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.230.2 0.0.0.0 UG 100 0 0 ens33 192.168.230.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33 192.168.230.200 0.0.0.0 255.255.255.255 UH 0 0 0 lo [root@cham1 ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet 192.168.230.200/32 brd 192.168.230.200 scope global lo:0 valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:90:35:d9 brd ff:ff:ff:ff:ff:ff inet 192.168.230.130/24 brd 192.168.230.255 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::7fe3:4489:d9af:a1ed/64 scope link valid_lft forever preferred_lft forever inet6 fe80::6f15:52d3:ebeb:e193/64 scope link tentative dadfailed valid_lft forever preferred_lft forever ##rs2(145) [root@test ~]# vim /usr/local/sbin/lvs_rs.sh [root@test ~]# sh !$ sh /usr/local/sbin/lvs_rs.sh [root@test ~]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.230.2 0.0.0.0 UG 100 0 0 ens33 192.168.230.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33 192.168.230.200 0.0.0.0 255.255.255.255 UH 0 0 0 lo [root@test ~]# ip add 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet 192.168.230.200/32 brd 192.168.230.200 scope global lo:0 valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:50:f9:44 brd ff:ff:ff:ff:ff:ff inet 192.168.230.145/24 brd 192.168.230.255 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::9b07:b28d:f5e9:d107/64 scope link valid_lft forever preferred_lft forever
135
[root@cham002 ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.230.200:80 rr -> 192.168.230.130:80 Route 1 0 1 -> 192.168.230.145:80 Route 1 0 1 [root@cham002 ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.230.200:80 rr -> 192.168.230.130:80 Route 1 2 1 -> 192.168.230.145:80 Route 1 1 1 [root@cham002 ~]#
test
In the browser, visit VIP: 192.168.230.200, refresh the web page, and the visit results are alternately replied by real1 and real2.
Open another virtual machine test in the same network segment!
Type `help' to learn how to use Xshell prompt. [d:\~]$ ssh 192.168.230.140 Connecting to 192.168.230.140:22... Connection established. To escape to local shell, press 'Ctrl+Alt+]'. Last login: Fri Nov 17 15:14:29 2017 from 192.168.230.1 [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 130 [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 130 [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 130 [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 130
18.12 Keepalived LVS
The complete architecture requires two servers (the role is dir) to install the Keepalived tool respectively, in order to achieve high availability, but Keepalived also has load balancing function, so only one Keepalived can be installed in this use. Keeping alive has the function of ipvsadm built in, so you don't need to install ipvsadm package, and you don't need to write and execute LVS? Dr script.
preparation
Three machines:
- Scheduler director:
IP: 192.168.230.135; installed - real server(real1):
IP: 192.168.230.130 - real server(real2):
IP: 192.168.230.145 - VIP: 192.168.230.200
• the / usr/local/sbin/lvs_rs.sh script still needs to be executed on both RS
• keepalived has a better function. When an rs goes down, it can no longer forward the request
• testing
Configure director (because we have installed it before when we made keepalived highly available)
[root@cham002 ~]# vim /etc/keepalived/keepalived.conf vrrp_instance VI_1 { #BACKUP on standby server state MASTER #The network card bound to vip is ens33. Your network card may be different from that of Amin. You need to change it here interface ens33 virtual_router_id 51 #90 on standby server priority 100 advert_int 1 authentication { auth_type PASS auth_pass aminglinux } virtual_ipaddress { 192.168.230.200 } } virtual_server 192.168.230.200 80 { #(query realserver status every 10 seconds) delay_loop 10 #(lvs algorithm) lb_algo wlc #(DR mode) lb_kind DR #(the connection of the same IP is allocated to the same RealServer within 60 seconds) persistence_timeout 0 #(check the state of realserver with TCP protocol) protocol TCP real_server 192.168.230.130 80 { #(weight) weight 100 TCP_CHECK { #(10 seconds no response timeout) connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 192.168.230.145 80 { weight 100 TCP_CHECK { connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } } //Execute ipvsadm -C to clear the previous ipvsadm rules [root@cham002 ~]# ipvsadm -C //Restart the network to turn off the previous vip [root@cham002 ~]# systemctl restart network //To start the preserved service: [root@cham002 ~]# systemctl start keepalived [root@cham002 ~]# ps aux |grep keep root 11296 0.0 0.1 120720 1404 ? Ss 1 April 24 0:00 /usr/sbin/keepalived -D root 11297 0.0 0.3 127460 3272 ? S 1 April 24 0:00 /usr/sbin/keepalived -D root 11298 0.0 0.3 131656 3032 ? S 1 April 24 0:05 /usr/sbin/keepalived -D root 47689 0.0 0.0 112680 976 pts/1 S+ 00:31 0:00 grep --color=auto keep //To view network card information: [root@cham002 ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:b6:9f:e3 brd ff:ff:ff:ff:ff:ff inet 192.168.230.135/24 brd 192.168.230.255 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.230.200/32 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.230.150/24 brd 192.168.230.255 scope global secondary ens33:0 valid_lft forever preferred_lft forever inet6 fe80::6f15:52d3:ebeb:e193/64 scope link valid_lft forever preferred_lft forever 3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:b6:9f:ed brd ff:ff:ff:ff:ff:ff inet 192.168.118.147/24 brd 192.168.118.255 scope global ens37 valid_lft forever preferred_lft forever inet 192.168.100.1/24 brd 192.168.100.255 scope global ens37 valid_lft forever preferred_lft forever inet6 fe80::1801:cbbb:ebcc:89a3/64 scope link [root@cham002 ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.230.200:80 wlc -> 192.168.230.130:80 Route 100 0 0 -> 192.168.230.145:80 Route 100 0 0
After adding Keepalived, the down real server will be automatically cleared out of rs list.
If there is one party ngixn It's down. It can be detected automatically. It can be loaded back automatically after it's turned on [root@cham002 ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.230.200:80 wlc -> 192.168.230.145:80 Route 100 0 0 [root@cham002 ~]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.230.200:80 wlc -> 192.168.230.130:80 Route 100 0 0 -> 192.168.230.145:80 Route 100 0 0
test
To escape to local shell, press 'Ctrl+Alt+]'. Last login: Tue Jan 30 23:15:25 2018 from 192.168.230.1 130 Downtime [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. [root@cham3 ~]# curl 192.168.230.200 cham 145 backup backup. 145 Downtime [root@cham3 ~]# curl 192.168.230.200 cham 130 [root@cham3 ~]# curl 192.168.230.200 cham 130 [root@cham3 ~]# curl 192.168.230.200 cham 130 [root@cham3 ~]# curl 192.168.230.200 cham 130
Preserved + LVS effect
- Keepalived builds high availability to ensure that the server will not be paralyzed after the director in LVS goes down
- If only LVS is used, when a real server in LVS architecture goes down, the director will continue to send requests to it,