brief introduction

zeek

Zeek is an open source network traffic analyzer. Many users use Zeek as a network security monitor (NSM) to support the investigation of suspicious or malicious activities. Zeek also supports various traffic analysis tasks outside the security domain, including performance evaluation and troubleshooting.
pfring
PF_RING is a new type of network socket, which can significantly improve the packet capture speed compared with libpcap. If PF is used_ Ring ZC (zero copy) driver can achieve extremely high packet capture / transmission speed, PF_RING ZC can capture 100G packets under sufficient hardware conditions.

Installation tutorial

Install pfring

reference resources: pf_ring installation and use guide

Install zeek

• Dependency installation

For CentOS7

 sudo yum install cmake make gcc gcc-c++ flex bison libpcap-devel openssl-devel python3 python3-devel swig zlib-devel

(Note: CentOS needs to install cmake3.0 or above)

ubuntu/ Debian

sudo apt-get install cmake make gcc g++ flex bison libpcap-dev libssl-dev python3 python3-dev swig zlib1g-dev

• Original code download

git clone --recursive https://github.com/zeek/zeek

• compile

 cd zeek-X.X.X
./configure --with-pcap=/usr/local/lib
make && sudo make install

• Make sure Zeek is properly linked to pf_ring-aware libpcap:

ldd /usr/local/zeek/bin/zeek | grep pcap

• Write configuration file

The configuration file is located in / usr / local / ZEEK / etc / node cfg

Write the following configuration files:

[logger-1]
type=logger
host=localhost
[manager]
type=manager
host=localhost
[proxy-1]
type=proxy
host=localhost

# Configure workers. You can configure multiple workers at the same time
[worker-1]
type=worker
host=localhost    
interface=ens33  
lb_method=pf_ring
lb_procs=4
pin_cpus=0,1,2,3
#[worker-2]
#type=worker
#host=192.168.0.100
#interface=eth0

Note:
worker is the node that actually collects traffic
host - IP address of the traffic collection node,
Interface - interface name
lb_method - packet capturing method
lb_procs number of load balancing queues
pin_cpus - the number of bound CPUs is usually the same as the number of load balancing queues

The above is the cluster working mode. Multiple computing nodes can work together. Here I run locally, so woreker and master are configured as localhost (Note: to use the cluster, you only need to write a configuration file on the manager, and of course, ZEEK needs to be installed on the worker node). Workers can be configured as other nodes, but other manager nodes are required to log in to other worker nodes without secret SSH. SSH password free login configuration method.

lb_ Detailed description of procs parameters

lb_ The procs parameter indicates the number of load balancing queues on the monitoring interface. There are usually two cases.

• Using RSS network card multi queue technology

RSS(Receive Side Scaling) is a network card driver technology that can efficiently distribute received messages among multiple CPU s in a multiprocessor system.
Almost all Intel (and other vendors) NIC s have RSS support, which means they can hash packets in hardware to distribute the load across multiple RX queues. In some cases, RSS is not available or flexible enough (for example, custom distribution function is required), and ZC can be used to distribute it through software instead.

If we configure the interface ens33 with four RSS queues, LB will be configured here_ Procs = 4, the corresponding CPU is configured to process the traffic pins of the four queues respectively_ cpus=0,1,2,3.

RSS is a load balancing mechanism designed to deal with large traffic. Using this method can greatly improve the processing capacity of traffic.

RSS configuration method reference.

• Traffic distribution using pfring ZC

pfring ZC has the function of traffic distribution. It can grab traffic from the network card at high speed and then distribute it to different queues using software. Similar to RSS, but implemented by software

sudo zbalance_ipc -i zc:eth1 -c 99 -n 8 -m 1 -g 8

-c 99 is the cluster ID
-n 8 is the number of queues
-g 8 is zbalance_ cpu binding of IPC
You should use zc: as the interface name, as shown in the following example.

[logger-1]
type=logger
host=localhost
[manager]
type=manager
host=localhost
[proxy-1]
type=proxy
host=localhost

# Configure workers. You can configure multiple workers at the same time
[worker-1]
type=worker
host=localhost    
interface=zc:99
lb_method=pf_ring
lb_procs=8
pin_cpus=0,1,2,3,4,5,6,7
#[worker-2]
#type=worker
#host=192.168.0.100
#interface=eth0

Start zeek

The terminal enters the zeek console with the following commands:

/usr/local/zeek/bin/zeekctl

First, use install to load the configuration, and use the start and stop commands to start and close zeek

[ZeekControl] > install 
[ZeekControl] > start
[ZeekControl] > stop

Acceleration with pfring ZC

PF_RING ™ ZC (zero copy) is a flexible packet processing framework that allows you to achieve 1/10 Gbit wire speed packet processing (RX and TX) at any packet size. Higher speed on enough hardware. Because it uses the method of bypassing the kernel protocol stack to directly grab data packets from the network card, it greatly improves the efficiency. Specific drivers need to be installed to use ZC. Installation method: pf_ring installation and use guide

The method of using ZC in zeek is very simple. Just use the method of ZC: < interface name >, and the configuration example is as follows:

[worker-2]
type=worker
host=192.168.0.101
lb_method=pf_ring
interface=zc:eth0
lb_procs=8
pin_cpus=0,1,2,3,4,5,6,7

Acceleration using pfring FT

Use pfring FT to filter the traffic you don't want to pay attention to, so as to achieve the purpose of acceleration. Pay attention to the difference between pfring FT and pfring ZC acceleration. Pfring ZC is optimized and accelerated when capturing data packets. pfring FT is used to filter useless traffic after capturing data packets, so that we only need to detect the traffic we pay attention to, so as to realize acceleration. And pfring FT is usually used with pfringZC, and can also be used with libpcap. The specific methods are as follows.

• Write pring FT configuration file / etc/pf_ring/ft-rules.conf
  The contents of the document are:

[filter]
YouTube = discard
Netflix = discard

The above is a simple example of filtering rules, which means that the traffic of YouTube and Netflix is discarded due to pf_ring FT uses DPI technology, so it can directly identify and filter the traffic of the application layer.

• zeek profile changes
  Just add the following sentence under the corresponding worker configuration in the Zeek configuration file

env_vars=PF_RING_FT_CONF=/etc/pf_ring/ft-rules.conf

For example:

[worker-2]
type=worker
host=192.168.0.101
lb_method=pf_ring
interface=zc:eth0
lb_procs=8
pin_cpus=0,1,2,3,4,5,6,7
env_vars=PF_RING_FT_CONF=/etc/pf_ring/ft-rules.conf

About us

For further information, please visit our official website and contact us: https://hongwangle.com/

We can also pay attention to our official account.