[DPDK] dpdk-l3fwd test cases are compiled separately

DPDK-20.11. Version 3 uses meson and ninja for compilation and installation. Refer to the previous process DPDK-20.11.3 in CentOS 8 Compile and run on 4 According to this step, after the compilation and installation is completed, it can be found that the test case executable has been generated in the build/app and build/example directories.

At this time, you can start and execute the startup command according to each use case.

But what if you want to add some print information to the test case? The following is a brief introduction to how to compile dpdk test cases separately.

Explain in advance:

1. The DPDK version number is 20.11 3. The operating system is CentOS 8.4;

2. Because meson and ninja tools are not familiar, test cases are compiled through Makefile;

3. There are many test cases for DPDK, which cannot be enumerated one by one, so use l3fwd cases for description;

4. Makefile files that can be compiled directly already exist under the test case, but the compilation depends on the DPDK installation environment. The following needs to be done is to compile without relying on the DPDK installation environment. (that is, put the static library, dynamic library, header file and *. c file required for compilation into one file, so that even if the modified file is moved to other operating systems without DPDK installed, it can be compiled and executed. In the actual application process, that is, just put the static library and header file dependent in the project under the specified path, which can be used by calling relevant API s).

Brief steps:

1. Create the project directory dpdk-l3fwd, and add the required static library, dynamic library, header file, * c) copy the document to the document;

2. Modify the Makefile file, mainly setting the environment variables and link library path required for compilation;

3. Compilation (static compilation and dynamic compilation);

4. Summary.

1. Create project

1.1. Create dpdk-l3fwd directory

Create dpdk-l3fw project directory, subdirectory and related files as follows.

[root@LFTF dpdk-l3fwd]# ll
total 36
drwxr-xr-x. 2 root root   128 Dec 29 13:15 app
drwxr-xr-x. 2 root root     6 Dec 29 10:39 include
drwxr-xr-x. 2 root root     6 Dec 29 10:39 lib
-rw-r--r--. 1 root root     0 Dec 29 13:23 Makefile
drwxr-xr-x. 2 root root     6 Dec 29 13:23 pkgconfig

among

app is a code subdirectory, which is used to store other than main c others c) documents;

include is the header file directory, which is used to store dpdk related header files and app directories c) dependent header file;

lib is the library file directory, which is used to store the static library files compiled by the dpdk source agent;

Makefile, example: the makefile file included in the example needs to be modified;

There are two in the pkgconfig file pc file, which is used to maintain the path linking dpdk static library and dynamic library; (detailed later);

1.2 source and library files

1.2. 1. Copy static / dynamic libraries and related header files

DPDK-20.11.3 after the compilation and installation is completed, the relevant static library and dynamic library files will be generated to the Linux system directory / usr/local/lib64 /, which can be viewed by using the following ll command. The dpdk related header files are in the path of usr/local/include:

View static library files

[root@LFTF ~]# ll /usr/local/lib64/*rte*.a
-rw-r--r--. 1 root root  170924 Dec 25 11:53 /usr/local/lib64/librte_acl.a
-rw-r--r--. 1 root root   76982 Dec 25 11:55 /usr/local/lib64/librte_baseband_acc100.a
...
#Intercepted part

View dynamic library files

[root@localhost ~]# ll /usr/local/lib64/*rte*.so
lrwxrwxrwx. 1 root root 16 Dec 25 11:57 /usr/local/lib64/librte_acl.so -> librte_acl.so.21
lrwxrwxrwx. 1 root root 40 Dec 25 11:57 /usr/local/lib64/librte_baseband_acc100.so -> dpdk/pmds-21.0/librte_baseband_acc100.so
lrwxrwxrwx. 1 root root 47 Dec 25 11:57 /usr/local/lib64/librte_baseband_fpga_5gnr_fec.so -> dpdk/pmds-21.0/librte_baseband_fpga_5gnr_fec.so
lrwxrwxrwx. 1 root root 46 Dec 25 11:57 /usr/local/lib64/librte_baseband_fpga_lte_fec.so -> dpdk/pmds-21.0/librte_baseband_fpga_lte_fec.so
...
#Intercepted part

View DPDK related header files

[root@localhost ~]# ll /usr/local/include/rte*.h
-rw-r--r--. 1 root root   3350 Sep  6 18:28 /usr/local/include/rte_acc100_cfg.h
-rw-r--r--. 1 root root  11081 Sep  6 18:28 /usr/local/include/rte_acl.h
-rw-r--r--. 1 root root    883 Sep  6 18:28 /usr/local/include/rte_acl_osdep.h
-rw-r--r--. 1 root root   2305 Sep  6 18:28 /usr/local/include/rte_alarm.h
-rw-r--r--. 1 root root   1684 Sep  6 18:28 /usr/local/include/rte_approx.h
-rw-r--r--. 1 root root   1914 Sep  6 18:28 /usr/local/include/rte_arp.h
-rw-r--r--. 1 root root   4454 Sep  6 18:28 /usr/local/include/rte_atomic_32.h
-rw-r--r--. 1 root root   4186 Sep  6 18:28 /usr/local/include/rte_atomic_64.h
-rw-r--r--. 1 root root   6598 Sep  6 18:28 /usr/local/include/rte_atomic.h
...
#Intercepted part

Create a dpdk file in the dpdk-l3fwd/lib directory and copy the static library and dynamic library of dpdk under the path / usr/local/lib64 /

[root@LFTF lib]# mkdir dpdk
[root@LFTF lib]# 
[root@LFTF lib]# ll
total 0
drwxr-xr-x. 2 root root 6 Dec 29 13:47 dpdk

Copy static library

[root@LFTF lib]# cp /usr/local/lib64/*rte*.a /home/dpdk-l3fwd/lib/dpdk/
[root@LFTF lib]# 

Copy dynamic library

[root@localhost ~]# cp /mnt/hgfs/code/dpdk/dpdk-l3fwd/lib/dpdk/*.so* /home/dpdk-l3fwd/lib/dpdk
[root@localhost ~]# 

If only static libraries or dynamic libraries are required, operate on demand.

Create a dpdk file in the dpdk-l3fwd/include directory and copy the header file about dpdk in the path of / usr/local/include /

[root@LFTF include]# pwd
/home/dpdk-l3fwd/include
[root@LFTF include]# 
[root@LFTF include]# mkdir dpdk
[root@LFTF include]# ll
total 0
drwxr-xr-x. 2 root root 6 Dec 29 17:24 dpdk

Copy header file

[root@localhost ~]# cp /usr/local/include/rte*.h /home/dpdk-l3fwd/include/dpdk/
[root@localhost ~]# cp -rf /usr/local/include/generic/ /home/dpdk-l3fwd/include/dpdk/
[root@localhost ~]# 

1.2. 2. Copy pkgconfig file

The principle and usage of PKG config can be searched by yourself or referred to Principle and usage of PKG config

Only dpdk related is explained here The storage path of pc files is / usr/local/lib64/pkgconfi

[root@localhost ~]# ll /usr/local/lib64/pkgconfig/
total 8
-rw-r--r--. 1 root root 1055 Dec 25 11:52 libdpdk-libs.pc
-rw-r--r--. 1 root root 4082 Dec 25 11:52 libdpdk.pc

Copy the two files in this directory to dpdk-l3fwd/pkgconfig

[root@localhost ~]# cp -r /usr/local/lib64/pkgconfig/ /home/dpdk-l3fwd/pkgconfig/
[root@localhost ~]# 

When the Makefile is executed later, it depends on the data in this path pc content, so it needs to be changed pc path

libdpdk. Before PC change:

prefix=/usr/local
libdir=${prefix}/lib64
includedir=${prefix}/include

Name: DPDK
Description: The Data Plane Development Kit (DPDK).
Note that CFLAGS might contain an -march flag higher than typical baseline.
This is required for a number of static inline functions in the public headers.
Version: 20.11.3
Requires: libdpdk-libs
...

libdpdk. After PC changes:

prefix=/home/dpdk-l3fwd/pkgconfig
libdir=${prefix}/lib
includedir=${prefix}/include

Name: DPDK
Description: The Data Plane Development Kit (DPDK).
Note that CFLAGS might contain an -march flag higher than typical baseline.
This is required for a number of static inline functions in the public headers.
Version: 20.11.3
Requires: libdpdk-libs
...

libdpdk-libs. Before PC change:

prefix=/usr/local
libdir=${prefix}/lib64
includedir=${prefix}/include

Name: dpdk-libs
Description: Internal-only DPDK pkgconfig file. Not for direct use.
Use libdpdk.pc instead of this file to query DPDK compile/link arguments
Version: 20.11.3
Libs: -Wl,--as-needed -L${libdir} -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd
...

libdpdk-libs. After PC changes:

prefix=/home/dpdk-l3fwd/pkgconfig
libdir=${prefix}/lib
includedir=${prefix}/include

Name: dpdk-libs
Description: Internal-only DPDK pkgconfig file. Not for direct use.
Use libdpdk.pc instead of this file to query DPDK compile/link arguments
Version: 20.11.3
Libs: -Wl,--as-needed -L${libdir} -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd
...

In short, we used to rely on the dynamic library and static library path installed in the system directory of dpdk, but now we have to rely on the static library and dynamic library path under the current project.

1.2. 3. Copy l3fwd source code

Dpdk-20.11 3. Copy the source code under the source code example/l3fwd path to the specified path.

[root@localhost l3fwd]# pwd
/home/dpdk-stable-20.11.3/examples/l3fwd
[root@localhost l3fwd]# ll
total 220
-rw-rw-r--. 1 root root  6601 Sep  6 18:28 l3fwd_altivec.h
-rw-rw-r--. 1 root root  6129 Sep  6 18:28 l3fwd_common.h
-rw-rw-r--. 1 root root 24787 Sep  6 18:28 l3fwd_em.c
-rw-rw-r--. 1 root root  4830 Sep  6 18:28 l3fwd_em.h
-rw-rw-r--. 1 root root  8524 Sep  6 18:28 l3fwd_em_hlm.h
-rw-rw-r--. 1 root root  1275 Sep  6 18:28 l3fwd_em_hlm_neon.h
-rw-rw-r--. 1 root root  1336 Sep  6 18:28 l3fwd_em_hlm_sse.h
-rw-rw-r--. 1 root root  3077 Sep  6 18:28 l3fwd_em_sequential.h
-rw-rw-r--. 1 root root  7305 Sep  6 18:28 l3fwd_event.c
-rw-rw-r--. 1 root root  9983 Sep  6 18:28 l3fwd_event_generic.c
-rw-rw-r--. 1 root root  2246 Sep  6 18:28 l3fwd_event.h
-rw-rw-r--. 1 root root  8851 Sep  6 18:28 l3fwd_event_internal_port.c
-rw-rw-r--. 1 root root  5798 Sep  6 18:28 l3fwd.h
-rw-rw-r--. 1 root root  3823 Sep  6 18:28 l3fwd_lpm_altivec.h
-rw-rw-r--. 1 root root 16472 Sep  6 18:28 l3fwd_lpm.c
-rw-rw-r--. 1 root root  2718 Sep  6 18:28 l3fwd_lpm.h
-rw-rw-r--. 1 root root  4226 Sep  6 18:28 l3fwd_lpm_neon.h
-rw-rw-r--. 1 root root  3410 Sep  6 18:28 l3fwd_lpm_sse.h
-rw-rw-r--. 1 root root  5886 Sep  6 18:28 l3fwd_neon.h
-rw-rw-r--. 1 root root  5726 Sep  6 18:28 l3fwd_sse.h
-rw-rw-r--. 1 root root 34192 Sep  6 18:28 main.c
-rw-rw-r--. 1 root root  1278 Dec 29 10:07 Makefile
-rw-rw-r--. 1 root root   457 Sep  6 18:28 meson.build

among

l3fw**. Copy the C file to the app directory;

main. Copy the C file to the dpdk-l3fwd directory;

l3fwd**.h. copy the file to the include directory;

Copy Makefile file to dpdk-l3fwd directory;

As follows:

[root@LFTF dpdk-l3fwd]# ls
app  include  lib  main.c  Makefile  pkgconfig
[root@LFTF dpdk-l3fwd]# ll *
-rw-r--r--. 1 root root 34192 Dec 29 13:16 main.c
-rw-r--r--. 1 root root     0 Dec 29 13:23 Makefile

app:
total 80
-rw-r--r--. 1 root root 24787 Dec 29 13:15 l3fwd_em.c
-rw-r--r--. 1 root root  7305 Dec 29 13:15 l3fwd_event.c
-rw-r--r--. 1 root root  9983 Dec 29 13:15 l3fwd_event_generic.c
-rw-r--r--. 1 root root  8851 Dec 29 13:15 l3fwd_event_internal_port.c
-rw-r--r--. 1 root root 16472 Dec 29 13:15 l3fwd_lpm.c

include:
total 112
drwxr-xr-x. 3 root root 12288 Dec 29 17:27 dpdk
-rw-r--r--. 1 root root  6601 Dec 29 17:31 l3fwd_altivec.h
-rw-r--r--. 1 root root  6129 Dec 29 17:31 l3fwd_common.h
-rw-r--r--. 1 root root  4830 Dec 29 17:31 l3fwd_em.h
-rw-r--r--. 1 root root  8524 Dec 29 17:31 l3fwd_em_hlm.h
-rw-r--r--. 1 root root  1275 Dec 29 17:31 l3fwd_em_hlm_neon.h
-rw-r--r--. 1 root root  1336 Dec 29 17:31 l3fwd_em_hlm_sse.h
-rw-r--r--. 1 root root  3077 Dec 29 17:31 l3fwd_em_sequential.h
-rw-r--r--. 1 root root  2246 Dec 29 17:31 l3fwd_event.h
-rw-r--r--. 1 root root  5798 Dec 29 17:31 l3fwd.h
-rw-r--r--. 1 root root  3823 Dec 29 17:31 l3fwd_lpm_altivec.h
-rw-r--r--. 1 root root  2718 Dec 29 17:31 l3fwd_lpm.h
-rw-r--r--. 1 root root  4226 Dec 29 17:31 l3fwd_lpm_neon.h
-rw-r--r--. 1 root root  3410 Dec 29 17:31 l3fwd_lpm_sse.h
-rw-r--r--. 1 root root  5886 Dec 29 17:31 l3fwd_neon.h
-rw-r--r--. 1 root root  5726 Dec 29 17:31 l3fwd_sse.h

lib:
total 16
drwxr-xr-x. 2 root root 12288 Dec 29 13:48 dpdk

pkgconfig:
total 0
[root@LFTF dpdk-l3fwd]# 

2,Makefile

2.1 notes before Makefile change

[root@LFTF dpdk-l3fwd]# cat Makefile
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2010-2016 Intel Corporation

# binary name #Compiled program name
APP = l3fwd

# all source are stored in SRCS-y	# Source code, need to compile c what are the documents
SRCS-y := main.c l3fwd_lpm.c l3fwd_em.c l3fwd_event.c
SRCS-y += l3fwd_event_generic.c l3fwd_event_internal_port.c

#PKG config is used to use The pc file specifies the path to the static or dynamic library to link
PKGCONF ?= pkg-config

# Build using pkg-config variables if possible  #See if it can be built using PKG config
ifneq ($(shell $(PKGCONF) --exists libdpdk && echo 0),0)
$(error "no installation of DPDK found")
endif

#Static compilation or dynamic compilation
all: static
.PHONY: shared static
shared: build/$(APP)-shared
	ln -sf $(APP)-shared build/$(APP)
static: build/$(APP)-static
	ln -sf $(APP)-static build/$(APP)

PC_FILE := $(shell $(PKGCONF) --path libdpdk 2>/dev/null)
# Get header file path
CFLAGS += -O3 $(shell $(PKGCONF) --cflags libdpdk)
# Added for 'rte_eth_link_to_str()'
CFLAGS += -DALLOW_EXPERIMENTAL_API
# Get static library file path
LDFLAGS_SHARED = $(shell $(PKGCONF) --libs libdpdk)
# Get dynamic library file path
LDFLAGS_STATIC = $(shell $(PKGCONF) --static --libs libdpdk)

# Dynamic compilation
build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
# Static compilation
build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
	$(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)

build:
	@mkdir -p $@

.PHONY: clean
clean:
	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
	test -d build && rmdir -p build || true

2.2 notes after Makefile change

# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2010-2016 Intel Corporation

# Get environment variables
PKG_DIR = pkgconfig
PKG_ENV = PKG_CONFIG_PATH=$(ROOT_DIR)/$(PKG_DIR)

# The compiled file suffix needs to be loaded
PS = c

# binary name #Program name
APP = l3fwd

# all source are stored in SRCS-y #Source code, traverse the main directory and subdirectories of all c file (wildcard)
SRCS-y := $(wildcard $(ROOT_DIR)/*.$(PS) $(addsuffix /*.$(PS), $(ROOT_DIR)/$(SUB_DIR)))
OBJS-y = $(foreach x,.c,$(patsubst %$(x),%.o,$(filter %$(x),$(SRCS-y))))

# Print the required link o files and environment variables
$(info OBJS1:${OBJS-y})
$(info PKG:$(PKG_ENV))

PKGCONF ?= pkg-config

# Build using pkg-config variables if possible #See if it can be built using PKG config
ifneq ($(shell $(PKG_ENV) $(PKGCONF) --exists libdpdk && echo 0),0)
$(error "no installation of DPDK found")
endif

# Static compilation or dynamic compilation
all: static
.PHONY: shared static 
shared: build/$(APP)-shared
	ln -sf $(APP)-shared build/$(APP)
static: build/$(APP)-static
	ln -sf $(APP)-static build/$(APP)

PC_FILE := $(shell $(PKG_ENV) $(PKGCONF) --path libdpdk 2>/dev/null)
# Ignore warning
CFLAGS+=-W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough
# Header file path
CFLAGS += -O3 $(shell $(PKG_ENV) $(PKGCONF) --cflags libdpdk) -I${ROOT_DIR}/include -I${ROOT_DIR}/include/dpdk -I${ROOT_DIR}/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK
# The dynamic library path is followed by the system dynamic library. If it is not dependent, it can also be ignored
LDFLAGS_SHARED = $(shell $(PKG_ENV) $(PKGCONF) --libs libdpdk) -lssl -lcrypto -lz -ldl -lpthread -lrt -lstdc++ -lnuma -lm -lpcap
# The static library path is followed by the system dynamic library, which can be ignored if it is not dependent
LDFLAGS_STATIC = $(shell $(PKG_ENV) $(PKGCONF) --static --libs libdpdk) -lssl -lcrypto -lz -ldl -lpthread -lrt -lstdc++ -lnuma -lm -lpcap

CFLAGS += -DALLOW_EXPERIMENTAL_API

%.o: %.c  
	$(CC) -c $(CFLAGS) -o $@ $<  

# Dynamic compilation
build/$(APP)-shared: $(OBJS-y) Makefile $(PC_FILE) | build
	$(CC) $(CFLAGS) $(OBJS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED) 

# Static compilation
build/$(APP)-static: $(OBJS-y) Makefile $(PC_FILE) | build
	$(CC) $(CFLAGS) $(OBJS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)

build:
	@mkdir -p $@

.PHONY: clean
clean:
	rm -f $(OBJS-y)
	rm -f build/$(APP) build/$(APP)-static build/$(APP)-shared
	test -d build && rmdir -p build || true

2.3 comparison before and after

There are three modifications:

1. Need to compile c file has project path and wildcard to obtain, which is more flexible and convenient;

2. The environment variable when PKG config is executed is the path where the current project is located, so the link between static library and dynamic library depends on the path under PKG config in the current project pc files;

3. The previous Makefile was not generated o intermediate files have also been added.

3. Compile

By modifying the statements in Makefile, you can control whether to link static libraries or dynamic libraries

3.1 static compilation

all: static
.PHONY: shared static
shared: build/$(APP)-shared
	ln -sf $(APP)-shared build/$(APP)
static: build/$(APP)-static
	ln -sf $(APP)-static build/$(APP)

Compilation process

[root@localhost dpdk-l3fwd]# make
OBJS1:/home/dpdk-l3fwd/main.o /home/dpdk-l3fwd/app/l3fwd_lpm.o /home/dpdk-l3fwd/app/l3fwd_em.o /home/dpdk-l3fwd/app/l3fwd_event_internal_port.o /home/dpdk-l3fwd/app/l3fwd_event.o /home/dpdk-l3fwd/app/l3fwd_event_generic.o
PKG:PKG_CONFIG_PATH=/home/dpdk-l3fwd/pkgconfig
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/main.o /home/dpdk-l3fwd/main.c  
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/app/l3fwd_lpm.o /home/dpdk-l3fwd/app/l3fwd_lpm.c  
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/app/l3fwd_em.o /home/dpdk-l3fwd/app/l3fwd_em.c  
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/app/l3fwd_event_internal_port.o /home/dpdk-l3fwd/app/l3fwd_event_internal_port.c  
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/app/l3fwd_event.o /home/dpdk-l3fwd/app/l3fwd_event.c  
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/app/l3fwd_event_generic.o /home/dpdk-l3fwd/app/l3fwd_event_generic.c  
cc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API /home/dpdk-l3fwd/main.o /home/dpdk-l3fwd/app/l3fwd_lpm.o /home/dpdk-l3fwd/app/l3fwd_em.o /home/dpdk-l3fwd/app/l3fwd_event_internal_port.o /home/dpdk-l3fwd/app/l3fwd_event.o /home/dpdk-l3fwd/app/l3fwd_event_generic.o -o build/l3fwd-static  -Wl,--whole-archive -L/home/dpdk-l3fwd/pkgconfig/lib -l:librte_common_cpt.a -l:librte_common_dpaax.a -l:librte_common_iavf.a -l:librte_common_octeontx.a -l:librte_common_octeontx2.a -l:librte_common_sfc_efx.a -l:librte_bus_dpaa.a -l:librte_bus_fslmc.a -l:librte_bus_ifpga.a -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_common_qat.a -l:librte_mempool_bucket.a -l:librte_mempool_dpaa.a -l:librte_mempool_dpaa2.a -l:librte_mempool_octeontx.a -l:librte_mempool_octeontx2.a -l:librte_mempool_ring.a -l:librte_mempool_stack.a -l:librte_net_af_packet.a -l:librte_net_ark.a -l:librte_net_atlantic.a -l:librte_net_avp.a -l:librte_net_axgbe.a -l:librte_net_bond.a -l:librte_net_bnx2x.a -l:librte_net_bnxt.a -l:librte_net_cxgbe.a -l:librte_net_dpaa.a -l:librte_net_dpaa2.a -l:librte_net_e1000.a -l:librte_net_ena.a -l:librte_net_enetc.a -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_fm10k.a -l:librte_net_i40e.a -l:librte_net_hinic.a -l:librte_net_hns3.a -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_igc.a -l:librte_net_ixgbe.a -l:librte_net_kni.a -l:librte_net_liquidio.a -l:librte_net_memif.a -l:librte_net_mlx4.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a -l:librte_net_nfp.a -l:librte_net_null.a -l:librte_net_octeontx.a -l:librte_net_octeontx2.a -l:librte_net_pcap.a -l:librte_net_pfe.a -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_sfc.a -l:librte_net_softnic.a -l:librte_net_tap.a -l:librte_net_thunderx.a -l:librte_net_txgbe.a -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a -l:librte_net_virtio.a -l:librte_net_vmxnet3.a -l:librte_raw_dpaa2_cmdif.a -l:librte_raw_dpaa2_qdma.a -l:librte_raw_ioat.a -l:librte_raw_ntb.a -l:librte_raw_octeontx2_dma.a -l:librte_raw_octeontx2_ep.a -l:librte_raw_skeleton.a -l:librte_crypto_bcmfs.a -l:librte_crypto_caam_jr.a -l:librte_crypto_ccp.a -l:librte_crypto_dpaa_sec.a -l:librte_crypto_dpaa2_sec.a -l:librte_crypto_nitrox.a -l:librte_crypto_null.a -l:librte_crypto_octeontx.a -l:librte_crypto_octeontx2.a -l:librte_crypto_openssl.a -l:librte_crypto_scheduler.a -l:librte_crypto_virtio.a -l:librte_compress_octeontx.a -l:librte_compress_zlib.a -l:librte_regex_mlx5.a -l:librte_regex_octeontx2.a -l:librte_vdpa_ifc.a -l:librte_vdpa_mlx5.a -l:librte_event_dlb.a -l:librte_event_dlb2.a -l:librte_event_dpaa.a -l:librte_event_dpaa2.a -l:librte_event_octeontx2.a -l:librte_event_opdl.a -l:librte_event_skeleton.a -l:librte_event_sw.a -l:librte_event_dsw.a -l:librte_event_octeontx.a -l:librte_baseband_null.a -l:librte_baseband_turbo_sw.a -l:librte_baseband_fpga_lte_fec.a -l:librte_baseband_fpga_5gnr_fec.a -l:librte_baseband_acc100.a -l:librte_node.a -l:librte_graph.a -l:librte_bpf.a -l:librte_flow_classify.a -l:librte_pipeline.a -l:librte_table.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pdump.a -l:librte_power.a -l:librte_member.a -l:librte_lpm.a -l:librte_latencystats.a -l:librte_kni.a -l:librte_jobstats.a -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -Wl,--no-whole-archive -Wl,--export-dynamic -lpcap -Wl,--as-needed -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lpcap -L/usr/usr/lib64 -lmlx5 -lpthread -L/usr/usr/lib64 -lpthread -libverbs -lpthread -lcrypto -ldl -pthread -lz -lmlx4 -lpthread -L/usr/usr/lib64 -libverbs -lpthread -lelf -lz  -lssl -lcrypto -lz -ldl -lpthread -lrt -lstdc++ -lnuma -lm -lpcap
ln -sf l3fwd-static build/l3fwd
[root@localhost dpdk-l3fwd]# 

View the dynamic libraries that need to be linked to run executable programs

[root@localhost dpdk-l3fwd]# ldd build/l3fwd
	linux-vdso.so.1 (0x00007ffe60ff5000)
	libpcap.so.1 => /lib64/libpcap.so.1 (0x00007fd942a66000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fd9426e4000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fd9424e0000)
	libnuma.so.1 => /lib64/libnuma.so.1 (0x00007fd9422d4000)
	libmlx5.so.1 => /lib64/libmlx5.so.1 (0x00007fd942081000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fd941e61000)
	libibverbs.so.1 => /lib64/libibverbs.so.1 (0x00007fd941c41000)
	libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007fd941758000)
	libz.so.1 => /lib64/libz.so.1 (0x00007fd941541000)
	libmlx4.so.1 => /lib64/libmlx4.so.1 (0x00007fd941334000)
	libelf.so.1 => /lib64/libelf.so.1 (0x00007fd94111b000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fd940d56000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fd942cb1000)
	libnl-route-3.so.200 => /lib64/libnl-route-3.so.200 (0x00007fd940ad0000)
	libnl-3.so.200 => /lib64/libnl-3.so.200 (0x00007fd9408ad000)
[root@localhost dpdk-l3fwd]# 

It can be seen that the links are all dynamic libraries under the system, not dpdk related.

Statically compiled executable program size 19MB~

[root@localhost dpdk-l3fwd]# ll -h build/l3fwd-static 
-rwxr-xr-x. 1 root root 19M Dec 30 10:19 build/l3fwd-static

3.2 dynamic compilation

all: shared
.PHONY: shared static
shared: build/$(APP)-shared
	ln -sf $(APP)-shared build/$(APP)
static: build/$(APP)-static
	ln -sf $(APP)-static build/$(APP)

Compilation process

[root@localhost dpdk-l3fwd]# make 
OBJS1:/home/dpdk-l3fwd/main.o /home/dpdk-l3fwd/app/l3fwd_lpm.o /home/dpdk-l3fwd/app/l3fwd_em.o /home/dpdk-l3fwd/app/l3fwd_event_internal_port.o /home/dpdk-l3fwd/app/l3fwd_event.o /home/dpdk-l3fwd/app/l3fwd_event_generic.o
PKG:PKG_CONFIG_PATH=/home/dpdk-l3fwd/pkgconfig
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/main.o /home/dpdk-l3fwd/main.c  
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/app/l3fwd_lpm.o /home/dpdk-l3fwd/app/l3fwd_lpm.c  
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/app/l3fwd_em.o /home/dpdk-l3fwd/app/l3fwd_em.c  
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/app/l3fwd_event_internal_port.o /home/dpdk-l3fwd/app/l3fwd_event_internal_port.c  
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/app/l3fwd_event.o /home/dpdk-l3fwd/app/l3fwd_event.c  
cc -c -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API -o /home/dpdk-l3fwd/app/l3fwd_event_generic.o /home/dpdk-l3fwd/app/l3fwd_event_generic.c  
cc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wold-style-definition -Wpointer-arith -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral -Wformat-security -Wundef -Wwrite-strings -Wno-unused-function -Wno-implicit-fallthrough -O3 -I/home/dpdk-l3fwd/pkgconfig/include -include rte_config.h -march=native -I/usr/usr/include  -I/home/dpdk-l3fwd/include -I/home/dpdk-l3fwd/include/dpdk -I/home/dpdk-l3fwd/app -std=gnu99 -fPIC -DPIC -DUSE_DPDK -DALLOW_EXPERIMENTAL_API /home/dpdk-l3fwd/main.o /home/dpdk-l3fwd/app/l3fwd_lpm.o /home/dpdk-l3fwd/app/l3fwd_em.o /home/dpdk-l3fwd/app/l3fwd_event_internal_port.o /home/dpdk-l3fwd/app/l3fwd_event.o /home/dpdk-l3fwd/app/l3fwd_event_generic.o -o build/l3fwd-shared  -Wl,--as-needed -L/home/dpdk-l3fwd/pkgconfig/lib -lrte_node -lrte_graph -lrte_bpf -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_regexdev -lrte_rawdev -lrte_pdump -lrte_power -lrte_member -lrte_lpm -lrte_latencystats -lrte_kni -lrte_jobstats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry -lrte_kvargs  -lssl -lcrypto -lz -ldl -lpthread -lrt -lstdc++ -lnuma -lm -lpcap 
ln -sf l3fwd-shared build/l3fwd
[root@localhost dpdk-l3fwd]# 

View the dynamic libraries that need to be linked to run executable programs

[root@localhost dpdk-l3fwd]# 
[root@localhost dpdk-l3fwd]# ldd build/l3fwd
	linux-vdso.so.1 (0x00007ffc19355000)
	librte_lpm.so.21 => /lib64/librte_lpm.so.21 (0x00007f3654d7a000)
	librte_eventdev.so.21 => /lib64/librte_eventdev.so.21 (0x00007f3654b4f000)
	librte_hash.so.21 => /lib64/librte_hash.so.21 (0x00007f365493d000)
	librte_cmdline.so.21 => /lib64/librte_cmdline.so.21 (0x00007f3654733000)
	librte_ethdev.so.21 => /lib64/librte_ethdev.so.21 (0x00007f3654487000)
	librte_net.so.21 => /lib64/librte_net.so.21 (0x00007f365427f000)
	librte_mbuf.so.21 => /lib64/librte_mbuf.so.21 (0x00007f365406d000)
	librte_mempool.so.21 => /lib64/librte_mempool.so.21 (0x00007f3653e63000)
	librte_eal.so.21 => /lib64/librte_eal.so.21 (0x00007f3653b73000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f3653953000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f365358e000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f3654f81000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f365320c000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f3653008000)
	libnuma.so.1 => /lib64/libnuma.so.1 (0x00007f3652dfc000)
	librte_kvargs.so.21 => /lib64/librte_kvargs.so.21 (0x00007f3652bf9000)
	librte_telemetry.so.21 => /lib64/librte_telemetry.so.21 (0x00007f36529f0000)
	librte_ring.so.21 => /lib64/librte_ring.so.21 (0x00007f36527ec000)
	librte_rcu.so.21 => /lib64/librte_rcu.so.21 (0x00007f36525e7000)
	librte_meter.so.21 => /lib64/librte_meter.so.21 (0x00007f36523e4000)
	librte_timer.so.21 => /lib64/librte_timer.so.21 (0x00007f36521e0000)
	librte_cryptodev.so.21 => /lib64/librte_cryptodev.so.21 (0x00007f3651fcf000)

It can be seen that it depends on the dynamic library related to dpdk.

The size of dynamically compiled executable program is only 102KB

[root@localhost dpdk-l3fwd]# ll -h build/l3fwd-shared 
-rwxr-xr-x. 1 root root 102K Dec 30 10:15 build/l3fwd-shared

4. Summary

The difference between static compilation and dynamic compilation can be searched and viewed by yourself~

dpdk-l3fwd can be compiled separately. The source code has been compressed and uploaded. It can be downloaded and used if necessary dpdk-l3fwd

Like it~( ❤ ω ❤) (

Keywords: Linux dpdk

Added by sgoku01 on Fri, 31 Dec 2021 18:32:54 +0200