Zeek is an open source network traffic analyzer. Many users use Zeek as a network security monitor (NSM) to support the investigation of suspicious or malicious activities. Zeek also supports various traffic analysis tasks outside the security domain, including performance evaluation and troubleshooting.
PF_RING is a new type of network socket, which can significantly improve the packet capture speed compared with libpcap. If PF is used_ Ring ZC (zero copy) driver can achieve extremely high packet capture / transmission speed, PF_RING ZC can capture 100G packets under sufficient hardware conditions.
reference resources: pf_ring installation and use guide
- Dependency installation
sudo yum install cmake make gcc gcc-c++ flex bison libpcap-devel openssl-devel python3 python3-devel swig zlib-devel
(Note: CentOS needs to install cmake3.0 or above)
sudo apt-get install cmake make gcc g++ flex bison libpcap-dev libssl-dev python3 python3-dev swig zlib1g-dev
- Original code download
git clone --recursive https://github.com/zeek/zeek
cd zeek-X.X.X ./configure --with-pcap=/usr/local/lib make && sudo make install
- Make sure Zeek is properly linked to pf_ring-aware libpcap:
ldd /usr/local/zeek/bin/zeek | grep pcap
- Write configuration file
The configuration file is located in / usr / local / ZEEK / etc / node cfg
Write the following configuration files:
[logger-1] type=logger host=localhost [manager] type=manager host=localhost [proxy-1] type=proxy host=localhost # Configure workers. You can configure multiple workers at the same time [worker-1] type=worker host=localhost interface=ens33 lb_method=pf_ring lb_procs=4 pin_cpus=0,1,2,3 #[worker-2] #type=worker #host=192.168.0.100 #interface=eth0
worker is the node that actually collects traffic
host - IP address of the traffic collection node,
Interface - interface name
lb_method - packet capturing method
lb_procs number of load balancing queues
pin_cpus - the number of bound CPUs is usually the same as the number of load balancing queues
The above is the cluster working mode. Multiple computing nodes can work together. Here I run locally, so woreker and master are configured as localhost (Note: to use the cluster, you only need to write a configuration file on the manager, and of course, ZEEK needs to be installed on the worker node). Workers can be configured as other nodes, but other manager nodes are required to log in to other worker nodes without secret SSH. SSH password free login configuration method.
lb_ Detailed description of procs parameters
lb_ The procs parameter indicates the number of load balancing queues on the monitoring interface. There are usually two cases.
- Using RSS network card multi queue technology
RSS(Receive Side Scaling) is a network card driver technology that can efficiently distribute received messages among multiple CPU s in a multiprocessor system.
Almost all Intel (and other vendors) NIC s have RSS support, which means they can hash packets in hardware to distribute the load across multiple RX queues. In some cases, RSS is not available or flexible enough (for example, custom distribution function is required), and ZC can be used to distribute it through software instead.
If we configure the interface ens33 with four RSS queues, LB will be configured here_ Procs = 4, the corresponding CPU is configured to process the traffic pins of the four queues respectively_ cpus=0,1,2,3.
RSS is a load balancing mechanism designed to deal with large traffic. Using this method can greatly improve the processing capacity of traffic.
- Traffic distribution using pfring ZC
pfring ZC has the function of traffic distribution. It can grab traffic from the network card at high speed and then distribute it to different queues using software. Similar to RSS, but implemented by software
sudo zbalance_ipc -i zc:eth1 -c 99 -n 8 -m 1 -g 8
-c 99 is the cluster ID
-n 8 is the number of queues
-g 8 is zbalance_ cpu binding of IPC
You should use zc: as the interface name, as shown in the following example.
[logger-1] type=logger host=localhost [manager] type=manager host=localhost [proxy-1] type=proxy host=localhost # Configure workers. You can configure multiple workers at the same time [worker-1] type=worker host=localhost interface=zc:99 lb_method=pf_ring lb_procs=8 pin_cpus=0,1,2,3,4,5,6,7 #[worker-2] #type=worker #host=192.168.0.100 #interface=eth0
The terminal enters the zeek console with the following commands:
First, use install to load the configuration, and use the start and stop commands to start and close zeek
[ZeekControl] > install [ZeekControl] > start [ZeekControl] > stop
Acceleration with pfring ZC
PF_RING ™ ZC (zero copy) is a flexible packet processing framework that allows you to achieve 1/10 Gbit wire speed packet processing (RX and TX) at any packet size. Higher speed on enough hardware. Because it uses the method of bypassing the kernel protocol stack to directly grab data packets from the network card, it greatly improves the efficiency. Specific drivers need to be installed to use ZC. Installation method: pf_ring installation and use guide
The method of using ZC in zeek is very simple. Just use the method of ZC: < interface name >, and the configuration example is as follows:
[worker-2] type=worker host=192.168.0.101 lb_method=pf_ring interface=zc:eth0 lb_procs=8 pin_cpus=0,1,2,3,4,5,6,7
Acceleration using pfring FT
Use pfring FT to filter the traffic you don't want to pay attention to, so as to achieve the purpose of acceleration. Pay attention to the difference between pfring FT and pfring ZC acceleration. Pfring ZC is optimized and accelerated when capturing data packets. pfring FT is used to filter useless traffic after capturing data packets, so that we only need to detect the traffic we pay attention to, so as to realize acceleration. And pfring FT is usually used with pfringZC, and can also be used with libpcap. The specific methods are as follows.
- Write pring FT configuration file / etc/pf_ring/ft-rules.conf
The contents of the document are:
[filter] YouTube = discard Netflix = discard
The above is a simple example of filtering rules, which means that the traffic of YouTube and Netflix is discarded due to pf_ring FT uses DPI technology, so it can directly identify and filter the traffic of the application layer.
- zeek profile changes
Just add the following sentence under the corresponding worker configuration in the Zeek configuration file
[worker-2] type=worker host=192.168.0.101 lb_method=pf_ring interface=zc:eth0 lb_procs=8 pin_cpus=0,1,2,3,4,5,6,7 env_vars=PF_RING_FT_CONF=/etc/pf_ring/ft-rules.conf
For further information, please visit our official website and contact us: https://hongwangle.com/
We can also pay attention to our official account.