LVS-DR mode under linux

DIP: the IP address of the scheduler

RIP: the IP address of the real server in the background

VIP: virtual IP used for service provision

Compared with NAT mode, DR mode of LVS is more complex and difficult to implement, because it is closer to hardware based load balancing. Although the performance is still lower than that of hardware based load balancing scheduling, it has been greatly improved compared with NAT and TUN modes, It requires that the scheduler and realserver are in the same network segment (switch can be used to connect, of course, IP based network connection). Here, only one network card can be used on the scheduler, whether DIP is equal to VIP or not. The biggest advantage of DT mode is the significant improvement of service quality. At the same time, relevant settings should be adopted to limit ARP on realserver.

The specific implementation process can be described as follows:

1. The client sends a request message to the scheduler. After receiving the subcontracting, the scheduler does not conduct in-depth processing. It only modifies the target MAC address to the MAC address of a real server in the realserver pool according to the algorithm, and caches a connection record (ensure that all messages of this connection are transmitted to the same realserver)

2. Realserver receives the message and makes relevant processing according to the requirements. After the processing, it rewrites the source and destination addresses of the message according to the VIP address bound by its own network card, and directly replies the message to the client without going through the scheduler, which objectively reduces the burden of the scheduler.

The experimental platform is three linux server hosts under VM (CentOS 7.5, the kernel is version 3.6, which supports LVS by default, so there is no need to recompile the kernel and can be used directly), one of which is used as scheduler and the other two are used as realserver.

First: settings on the scheduler:

[root@localhost ~]# yum -y install ipvsadm / / install LVS management tools
[root@localhost ~]# ifconfig / / view the initial network card information (only one network card is used here)
eth0      Link encap:Ethernet  HWaddr 00:0C:29:09:22:5D  
          inet addr:192.168.1.196  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe09:225d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:17657 errors:0 dropped:0 overruns:0 frame:0
          TX packets:17945 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:7404853 (7.0 MiB)  TX bytes:10817894 (10.3 MiB)
          Interrupt:169 Base address:0x2000 
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:88 errors:0 dropped:0 overruns:0 frame:0
          TX packets:88 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:6797 (6.6 KiB)  TX bytes:6797 (6.6 KiB)
[root@localhost ~]# vi /etc/init.d/lvsdr / / write cluster management scripts. Of course, you can add them one by one
#!/bin/bash
#description:start LVS of Directorserver DR
VIP=192.168.1.200
RIP1=192.168.1.193
RIP2=192.168.1.195


#./etc/rc.d/init.d/functions
case "$1" in
        start)
        echo "start LVS of DirectorServer DR"
        /sbin/ifconfig eth0:0 $VIP broadcast $VIP netmask 255.255.255.255 up
        /sbin/route add -host $VIP dev eth0:0
        echo "1">/proc/sys/net/ipv4/ip_forward
        /sbin/ipvadm -C
        /sbin/ipvsadm -A -t $VIP:80 -s rr           //It is still the rotation algorithm
        /sbin/ipvsadm -a -t $VIP:80 -r $RIP1 –g     //-g means direct routing mode is adopted
        /sbin/ipvsadm -a -t $VIP:80 -r $RIP2 -g
        /sbin/ipvsadm
        ;;
        stop)
        echo "stop LVS of DirectorServer DR"
        echo "0" >/proc/sys/net/ipv4/ip_forward
        /sbin/ipvsadm -C
        /sbin/ifconfig eth0:0 down
        ;;
        *)
        echo "Usage:$0{start|stop}"
        exit 1
 esac
[root@localhost ~]# chmod a+x /etc/init.d/lvsndr / / add executable permission
[root@localhost ~]# /etc/init.d/lvsdr start
start LVS of DirectorServer DR
/etc/init.d/lvsdr: line 13: /sbin/ipvadm: No such file or directory
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.1.200:http rr
  -> 192.168.1.195:http           Route   1      0          0         
  -> 192.168.1.193:http           Route   1      0          0    
[root@localhost ~]# ifconfig / / view the change information
eth0      Link encap:Ethernet  HWaddr 00:0C:29:09:22:5D  
          inet addr:192.168.1.196  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:fe09:225d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:18342 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18549 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:7480550 (7.1 MiB)  TX bytes:10886441 (10.3 MiB)
          Interrupt:169 Base address:0x2000 
eth0:0    Link encap:Ethernet  HWaddr 00:0C:29:09:22:5D  
          inet addr:192.168.1.200  Bcast:192.168.1.200  Mask:255.255.255.255
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:169 Base address:0x2000 
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:88 errors:0 dropped:0 overruns:0 frame:0
          TX packets:88 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:6797 (6.6 KiB)  TX bytes:6797 (6.6 KiB)

Second: settings on nodes (since the settings on the two nodes are basically the same, only one of the nodes is set here)

On 192.168.1.193

[root@localhost ~]# ifconfig / / view the network card information
eth0      Link encap:Ethernet  HWaddr 00:0C:29:B7:97:5B  
          inet addr:192.168.1.193  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:feb7:975b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:48655 errors:0 dropped:0 overruns:0 frame:0
          TX packets:39793 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:16204310 (15.4 MiB)  TX bytes:6641366 (6.3 MiB)
          Interrupt:169 Base address:0x2000 
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:12600 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12600 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:8047060 (7.6 MiB)  TX bytes:8047060 (7.6 MiB)
[root@localhost ~]# vi /etc/init.d/lvscdr / / write ARP and VIP binding scripts (the two nodes are identical)
#!/bin/bash
#descrpption :start realserver DR
VIP=192.168.1.200
#./etc/rc.d/init.d/functions
case "$1" in
        start)
        echo "start LVS of RealServer DR"
        /sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up
        /sbin/route add -host $VIP dev lo:0
        echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignore
        echo "2">/proc/sys/net/ipv4/conf/all/arp_announce
        echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore
        echo "2">/proc/sys/net/ipv4/conf/all/arp_announce
        ;;
        stop)
        /sbin/ifconfig lo:0 down
        echo "0">/proc/sys/net/ipv4/conf/lo/arp_ignore
        echo "0">/proc/sys/net/ipv4/conf/all/arp_announce
        echo "0">/proc/sys/net/ipv4/conf/all/arp_ignore
        echo "0">/proc/sys/net/ipv4/conf/all/arp_announce
        ;;
        *)
        echo "Usage:$0 {start|stop}"
        exit 1
esac
[root@localhost ~]# chmod a+x /etc/init.d/lvscdr 
[root@localhost ~]# /etc/init.d/lvscdr start
[root@localhost ~]# ifconfig / / view the change information
eth0      Link encap:Ethernet  HWaddr 00:0C:29:B7:97:5B  
          inet addr:192.168.1.193  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::20c:29ff:feb7:975b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:48853 errors:0 dropped:0 overruns:0 frame:0
          TX packets:39954 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:16225588 (15.4 MiB)  TX bytes:6659428 (6.3 MiB)
          Interrupt:169 Base address:0x2000 
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:12600 errors:0 dropped:0 overruns:0"frale:0
   $(     TX packets:12600 erpors*0 dropped:8 oferruns:0 carrier:0
          collishons:0 tXqueuelen:0 
     0 `  RX bytes:8047060 (7.6 MiB)  VX bytes:8047060 (7.6 MiB)
ìo:0"     Link encap:Local LoopbacK  
      !   ijet addr:192.16:.1.200  Mcsk:2%5.255.055.255
       `  UP LOOPBACK RUN 
ING  MTU*1643¶  Metric:1

The third part of the test:

In order to test the effect is obvious, so in the rea, the server specially sets the content of the web page to be different. Of course, this is only for testing. In the real environment, it is necessary to maintain the integrity of the web content

Test effect

Enter the name of VIP in the address bar and create new pages

M watch on the reducer

[root@localhost ~] (ipvsaɤTo
Iࡐ Virtual œerѶer versio࡮ 1®2.9 (ѳize=4096) Prot plow LocalAddress:Port0Schedul%r цlags
  -To make a livingࠠRemoteAddr%ss:ѐort   Ƞ       Forward Weight ActiveConn InActConn
TCP  192.168.1.200:http rr
  -> 192.168.1.195:http           Route   1      0          5         
  -> 192.168.1.193:http           Route   1      0          4         
[root@localhost ~]# ipvsadm -lnc
IPVS connection entries
pro expire state       source             virtual            destination
TCP 01:51  FIN_WAIT    192.168.1.188:53751 192.168.1.200:80   192.168.1.193:80
TCP 01:53  FIN_WAIT    192.168.1.188:53760 192.168.1.200:80   192.168.1.195:80
TCP 01:52  FIN_WAIT    192.168.1.188:53755 192.168.1.200:80   192.168.1.193:80
TCP 01:53  FIN_WAIT    192.168.1.188:53756 192.168.1.200:80   192.168.1.195:80
TCP 00:46  FIN_WAIT    192.168.1.188:53746 192.168.1.200:80   192.168.1.195:80
TCP 01:53  FIN_WAIT    192.168.1.188:53759 192.168.1.200:80   192.168.1.193:80
TCP 01:53  FIN_WAIT    192.168.1.188:53757 192.168.1.200:80   192.168.1.193:80
TCP 01:52  FIN_WAIT    192.168.1.188:53754 192.168.1.200:80   192.168.1.195:80
TCP 01:53  FIN_WAIT    192.168.1.188:53758 192.168.1.200:80   192.168.1.195:80

Keywords: Linux network server

Added by ddc on Tue, 01 Feb 2022 10:03:09 +0200