history

tc was written by Alexey N. Kuznetsov and incorporated into the Linux kernel from Linux version 2.2.

Linux supports QoS from kernel 2.1.105, but the kernel needs to be recompiled. When you run make config, it will be exceptional_ Set options to y, set class based queuing (CBQ), token bucket flow and traffic shapers to y, and run make dep; make clean; make bzilo； Generate a new kernel.

In the Linux operating system, the traffic controller (TC) mainly establishes a queue at the output port for traffic control. The control method is based on routing, that is, the traffic control based on the destination IP address or the network number of the destination subnet. The basic functional modules of flow controller TC are queue, classification and filter. The queues supported in the Linux kernel include Class Based Queue, Token Bucket Flow, CSZ, first in, first out, Priority, TEQL, SFQ, ATM and RED. The queue and classification discussed here are based on CBQ(Class Based Queue), while the filter is based on route.

The configuration and use of traffic controller TC are mainly divided into the following aspects: establishing queue, establishing classification, establishing filter and establishing route. In addition, it is also necessary to monitor the existing queue, classification, filter and route.

The basic steps are as follows:
(1) Bind a CBQ queue for network physical devices (such as Ethernet card eth0);
(2) Establishing a classification on the queue;
(3) Establish a routing based filter for each classification;
(4) Finally, cooperate with the filter to establish a specific routing table.

Suppose a simple environment: the IP address of the Ethernet card (eth0) on the traffic controller is 192.168.1.66, and a CBQ queue is established on it. Assuming that the average size of the packet is 1000 bytes, the size of the inter packet transmission unit is 8 bytes, and the maximum number of conflicting transmission packets that can be received is 20 bytes.

If there are three types of flow that need to be controlled:
(1) It is sent to host 1, and its IP address is 192.168.1.24. Its traffic bandwidth is controlled at 8Mbit and its priority is 2;
(2) It is sent to host 2, and its IP address is 192.168.1.26. The flow bandwidth is controlled at 1Mbit and the priority is 1;
(3) It is sent to subnet 1. The subnet number is 192.168.1.0 and the subnet mask is 255.255.255.0. The traffic bandwidth is controlled at 1Mbit and the priority is 6.

The process is as follows:

1. Establish queue

   Generally, only one queue needs to be established for a network card.

   Bind a cbq queue to the network physical device eth0 with the number of 1:0; The actual bandwidth of network physical device eth0 is 10 Mbit, and the average packet size is 1000 bytes; The size of the inter packet transmission unit is 8 bytes, and the minimum transmission packet size is bytes.

   tc qdisc add dev eth0 root handle 1: cbq bandwidth 10Mbit avpkt 1000 cell 8 mpu
   

2. Establish classification

   Classification is based on queues. In general, a root classification needs to be established for a queue, and then a sub classification needs to be established on it. For the classification, it works according to the numbering sequence of the classification, and the small number takes precedence; Once a classification matching rule is met and a packet is sent through the classification, the subsequent classification will no longer work.

   1) Create root classification 1:1; The allocated bandwidth is 10Mbit and the priority is 8.

   tc class add dev eth0 parent 1:0 classid 1:1 cbq bandwidth 10Mbit rate 10Mbit maxburst 20 allot 1514 prio 8 avpkt 1000 cell 8 weight 1Mbit
   

   The maximum available bandwidth of the queue is 10Mbit, the actual allocated bandwidth is 10Mbit, and the maximum number of packets that can receive conflicts is 20 bytes; The size of the maximum transmission unit plus MAC header is 1514 bytes, the priority level is 8, the average size of the packet is 1000 bytes, the size of the inter packet transmission unit is 8 bytes, and the weighting rate corresponding to the actual bandwidth is 1Mbit.

   2) Create category 1:2, its parent category is 1:1, the allocated bandwidth is 8Mbit, and the priority is 2.

   tc class add dev eth0 parent 1:1 classid 1:2 cbq bandwidth 10Mbit rate 8Mbit maxburst 20 allot 1514 prio 2 avpkt 1000 cell 8 weight 800Kbit split 1:0 bounded
   

   The maximum available bandwidth of the queue is 10Mbit, the actually allocated bandwidth is 8Mbit, and the maximum number of packets that can receive conflicts is 20 bytes; The size of the maximum transmission unit plus MAC header is 1514 bytes, the priority level is 1, the average size of the packet is 1000 bytes, the size of the inter packet transmission unit is 8 bytes, the weighted rate corresponding to the actual bandwidth is 800Kbit, the separation point of the classification is 1:0, and the unused bandwidth cannot be borrowed.

   3) Create category 1:3, its parent category is 1:1, the allocated bandwidth is 1Mbit, and the priority is 1.

   tc class add dev eth0 parent 1:1 classid 1:3 cbq bandwidth 10Mbit rate 1Mbit maxburst 20 allot 1514 prio 1 avpkt 1000 cell 8 weight 100Kbit split 1:0
   

   The maximum available bandwidth of the queue is 10Mbit, the actually allocated bandwidth is 1Mbit, and the maximum number of packets that can receive conflicts is 20 bytes; The size of the maximum transmission unit plus MAC header is 1514 bytes, the priority level is 2, the average size of the packet is 1000 bytes, the size of the inter packet transmission unit is 8 bytes, the weighting rate corresponding to the actual bandwidth is 100Kbit, and the separation point of the classification is 1:0.

   4) Create category 1:4, its parent category is 1:1, the allocated bandwidth is 1Mbit, and the priority is 6.

   tc class add dev eth0 parent 1:1 classid 1:4 cbq bandwidth 10Mbit rate 1Mbit maxburst 20 allot 1514 prio 6 avpkt 1000 cell 8 weight 100Kbit split 1:0
   

   The maximum available bandwidth of the queue is 10Mbit, the actually allocated bandwidth is Kbit, and the maximum number of packets that can receive conflicts is 20 bytes; The size of the maximum transmission unit plus MAC header is 1514 bytes, the priority level is 1, the average size of the packet is 1000 bytes, the size of the inter packet transmission unit is 8 bytes, the weighted rate corresponding to the actual bandwidth is 100Kbit, and the separation point of the classification is 1:0.

3. Create filter

   Filters mainly serve classification. Generally, you only need to provide a filter for the root classification, and then provide routing mapping for each sub classification.

   1) Apply the route classifier to the root of the cbq queue, and the parent classification number is 1:0; The filtering protocol is ip, the priority is 100, and the filter is based on the routing table.

   tc filter add dev eth0 parent 1:0 protocol ip prio 100 route
   

   2) Establish routing mapping classification 1:2, 1:3, 1:4

   tc filter add dev eth0 parent 1:0 protocol ip prio 100 route to 2 flowid 1:2
   
   tc filter add dev eth0 parent 1:0 protocol ip prio 100 route to 3 flowid 1:3
   
   tc filter add dev eth0 parent 1:0 protocol ip prio 100 route to 4 flowid 1:4
   

4. Establish route

   The route is one-to-one corresponding to the previously established route map.

   1) Packets sent to the host 192.168.1.24 are forwarded through category 2 (the rate of category 2 is 8Mbit)

   ip route add 192.168.1.24 dev eth0 via 192.168.1.66 realm 2
   

   2) Packets sent to the host 192.168.1.30 are forwarded through category 3 (the rate of Category 3 is 1Mbit)

   ip route add 192.168.1.30 dev eth0 via 192.168.1.66 realm 3
   

   3) Packets sent to subnet 192.168.1.0/24 are forwarded through category 4 (the rate of category 4 is 1Mbit)

   ip route add 192.168.1.0/24 dev eth0 via 192.168.1.66 realm 4
   

   Note: generally, for the network segment directly connected to the flow controller, it is recommended to use the IP host address flow control limit instead of the subnet flow control limit. If it is necessary to use subnet traffic control restrictions on the directly connected subnet, the original route established by the system must be deleted before the route mapping of the subnet is established.

5. monitor

   It mainly includes monitoring the status of existing queues, classifications, filters and routes.

   1) Displays the status of the queue

   Simply display the queue status of the specified device (eth0 in this case)

   tc qdisc ls dev eth0
   

   qdisc cbq 1: rate 10Mbit (bounded,isolated) prio no-transmit
   

   The queue status of the specified device (eth0 in this case) is displayed in detail

   tc -s qdisc ls dev eth0
   

   qdisc cbq 1: rate 10Mbit (bounded,isolated) prio no-transmit
   Sent 76731 bytes 13232 pkts (dropped 0, overlimits 0)
   borrowed 0 overactions 0 avgidle 31 undertime 0
   

   It mainly shows that 13232 packets are sent through the queue, the data flow is 76731 bytes, the number of packets discarded is 0, and the number of packets exceeding the rate limit is 0.

   2) Displays the status of the classification

   Simply display the classification status of the specified device (eth0 here)

   tc class ls dev eth0
   

   class cbq 1: root rate 10Mbit (bounded,isolated) prio no-transmit
   class cbq 1:1 parent 1: rate 10Mbit prio no-transmit 	#No transmit indicates that the priority is 8
   class cbq 1:2 parent 1:1 rate 8Mbit prio 2
   class cbq 1:3 parent 1:1 rate 1Mbit prio 1
   class cbq 1:4 parent 1:1 rate 1Mbit prio 6
   

   The classification status of the specified device (eth0 here) is displayed in detail

   tc -s class ls dev eth0
   

   class cbq 1: root rate 10Mbit (bounded,isolated) prio no-transmit
   Sent 17725304 bytes 32088 pkts (dropped 0, overlimits 0)
   borrowed 0 overactions 0 avgidle 31 undertime 0
   class cbq 1:1 parent 1: rate 10Mbit prio no-transmit
   Sent 16627774 bytes 28884 pkts (dropped 0, overlimits 0)
   borrowed 16163 overactions 0 avgidle 587 undertime 0
   class cbq 1:2 parent 1:1 rate 8Mbit prio 2
   Sent 628829 bytes 3130 pkts (dropped 0, overlimits 0)
   borrowed 0 overactions 0 avgidle 4137 undertime 0
   class cbq 1:3 parent 1:1 rate 1Mbit prio 1
   Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
   borrowed 0 overactions 0 avgidle 159654 undertime 0
   class cbq 1:4 parent 1:1 rate 1Mbit prio 6
   Sent 5552879 bytes 8076 pkts (dropped 0, overlimits 0)
   borrowed 3797 overactions 0 avgidle 159557 undertime 0
   

   It mainly shows the data packets sent through different classifications, data traffic, the number of discarded packets, the number of packets exceeding the rate limit, and so on. The status of the Root Category (class cbq 1:0) should be similar to that of the queue.

   For example, class cbq 1:4 sends 8076 packets, the data flow is 5552879 bytes, the number of discarded packets is 0, and the number of packets exceeding the rate limit is 0.

   3) Displays the status of the filter

   tc -s filter ls dev eth0
   

   filter parent 1: protocol ip pref 100 route
   filter parent 1: protocol ip pref 100 route fh 0xffff0002 flowid 1:2 to 2
   filter parent 1: protocol ip pref 100 route fh 0xffff0003 flowid 1:3 to 3
   filter parent 1: protocol ip pref 100 route fh 0xffff0004 flowid 1:4 to 4
   

   Here, flowid 1:2 represents classification class cbq 1:2, and to 2 represents sending through route 2.

   4) Displays the status of existing routes

   ip route
   

   192.168.1.66 dev eth0 scope link
   192.168.1.24 via 192.168.1.66 dev eth0 realm 2
   202.102.24.216 dev ppp0 proto kernel scope link src 202.102.76.5
   192.168.1.30 via 192.168.1.66 dev eth0 realm 3
   192.168.1.0/24 via 192.168.1.66 dev eth0 realm 4
   192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.66
   172.16.1.0/24 via 192.168.1.66 dev eth0 scope link
   127.0.0.0/8 dev lo scope link
   default via 202.102.24.216 dev ppp0
   default via 192.168.1.254 dev eth0
   

   As shown above, the display line with realm at the end is the active routing filter.

6. maintain

   It mainly includes the addition, modification and deletion of queues, classifications, filters and routes.

   Adding is generally performed in the order of "queue - > classification - > filter - > routing"; There are no requirements for modification action; Deletion is performed in the order of "route - > filter - > classification - > queue".

   1) Queue maintenance

   Generally, for a flow controller, a queue has been configured for each Ethernet card before leaving the factory. Generally, there is no need to add, modify and delete the queue.

   2) Classification maintenance

   A. Add

   The addition action is implemented by the tc class add command, as shown above.

   B. Modification

   The modification action is implemented by the tc class change command, as shown below:

   tc class change dev eth0 parent 1:1 classid 1:2 cbq bandwidth 10Mbit rate 7Mbit maxburst 20 allot 1514 prio 2 avpkt 1000 cell 8 weight 700Kbit split 1:0 bounded
   

   The bounded command should be used with caution. Once added, it should be modified. It can only be added after deletion.

   C. Delete

   The deletion can only be performed before the classification has no work. Once data has been sent through the classification, it cannot be deleted. Therefore, it needs to be modified through shell file and deleted through restart.

   3) Filter maintenance

   A. Add

   The addition action is implemented by the tc filter add command, as shown above.

   B. Modification

   The modification action is realized by the tc filter change command, as shown below:

   tc filter change dev eth0 parent 1:0 protocol ip prio 100 route to 10 flowid 1:8
   

   C. Delete

   The delete action is implemented by the tc filter del command, as shown below:

   tc filter del dev eth0 parent 1:0 protocol ip prio 100 route to 10
   

   4) Maintenance of one-to-one mapping routes with filters

   A. Add

   The add action is implemented by the ip route add command, as shown above.

   B. Modification

   The modification action is realized through the ip route change command, as shown below:

   ip route change 192.168.1.30 dev eth0 via 192.168.1.66 realm 8
   

   C. Delete

   The deletion action is realized by the ip route del command, as shown below:

   ip route del 192.168.1.30 dev eth0 via 192.168.1.66 realm 8
   ip route del 192.168.1.0/24 dev eth0 via 192.168.1.66 realm 4
   

TC rules involve queue, class and filter. The flag bit divided by filter can be realized by u32 or iptables set mark. Generally, it is "controlled to send" but not controlled to receive. There are two network cards under linux, one eth1 is the external network and the other eth0 is the internal network. HTB is done on eth0. (Note: the flag bit of filter division can be marked with u32 marking function or set mark function of iptables. If iptables is used to mark, the downlink rate is controlled at eth0, but the marking should be carried out before entering eth0. Therefore, "- i eth1", example:

The main commands are as follows:

1) Create an HTB root

tc qdisc add dev eth0 root handle 1: htb default 20 

2) Create an HTB class, where the traffic limit is limited, and set the burst

tc class add dev eth0 parent 1: classid 1:1 htb rate 200kbit(rate) ceil 200kbit burst 20k(Burst traffic)

3) Create a filtering rule to filter out the data to limit the traffic and send it to the above class to limit the speed

tc filter add dev eth0 parent 1: prio 1(priority) protocol ip u32 match ip sport 80 0xfff flowid 1:1

Note: keep the delay time of interactive packets low and obtain the idle bandwidth first, such as ssh, telnet, dns, quake3, irc, ftp, smtp commands and packets with SYN marks, all belong to this category. In order to ensure that the uplink data stream will not harm the downlink, the ACK packets must be placed in front of the queue, because the ack of the downlink data must compete with the uplink.