BumbleBee: build, deliver and run eBPF program smoothly like silk

Address: https://www.ebpf.top/post/bumblebee

1. Preface

not long ago, Solo.io The company announced open source on its official website blog, with a name of BumbleBee New project. The project focuses on simplifying the threshold for building eBPF tools and optimizing the use experience by packaging eBPF programs into OCI image , building, distributing, and running eBPF programs that bring a consistent experience with using Docker.

The purpose of BumbleBee is to let us focus on writing eBPF code, which is responsible for automatically generating the code function of user space related to eBPF program, including loading eBPF program and displaying the data of eBPF program as log, index and histogram.

So why do we need the BumbleBee project to manage the eBPF program? This needs to start from the characteristics of eBPF technology.

2. Challenges in building and distributing eBPF tools

eBPF technology is known as the biggest change of operating system in recent 50 years, which solves the dilemma of slow upstream development, merger and distribution of Linux kernel. eBPF technology provides the kernel with the ability to realize the customized function of the kernel without upstream. At present, it has been widely used in many fields such as observability, network and security. Especially in the cloud native technology trend, the ability of eBPF technology is becoming more and more important, such as the current popular Cilium project.

However, the development, construction and distribution of eBPF has always been a high threshold work. The community has successively launched front-end binding work such as BCC and BPFTrace, which greatly reduces the threshold for writing and using eBPF technology. However, the source code delivery mode of these tools requires the installation of supporting compilation environment on the Linux system running eBPF program, which brings a lot of trouble for distribution, At the same time, the problem of kernel adaptation can only be verified at runtime, which is not conducive to discovering and solving the problem in advance.

In recent years, in order to solve the problem of distribution and operation of eBPF programs in different kernel versions, the community has launched the CO-RE function ("compile once, run everywhere") based on BTF technology, which not only realizes the distribution of binary bytecode through eBPF to a certain extent, but also solves the problem of transplantation running on different kernels, However, there is no unified and concise way to package and distribute eBPF binary code. In addition to the eBPF program, we also need to write various codes for loading the eBPF program and reading the data generated by the eBPF program, which often involves copying and pasting the source code to solve some problems.

In addition, libbpf-bootstrap Generating relevant scaffold code through bpftool tool solves the problem of common code duplication to a certain extent, but it provides limited help in building, distributing and running eBPF programs.

3. Introduction to bumblebee

The BumbleBee project is exactly what Solo Enterprise service grid GLOO mesh The project was born to facilitate the application of eBPF technology, which is used to solve the repetitive challenges encountered in building, distributing and running eBPF programs,

At present, the project is still in its early stage (current version 0.0.9), and the functional scenarios (Network and file system) provided are limited. However, the idea of building OCI image based on specific template capabilities provides us with an efficient and concise implementation in managing eBPF programs, which is worthy of our attention.

Pre dependency of using BumbleBee tool: the operating system of running eBPF has enabled BTF support, and the eBPF code also needs to use CO-RE related functions. For CO-RE related technologies, please refer to here.

BumbleBee provides a sense of experience consistent with Docker. The following figure is a high-level schematic diagram of Docker, and the BumbleBee tool fully refers to this process.

3.1 construction

Let ebblee focus on the process of building ebblee's code, and let your ebblee focus on the process of building ebblee's code. BumbleBee's eBPF code is packaged into an OCI standard image so that it can be distributed in the infrastructure.

The following command can be used to convert the eBPF program probe C is directly compiled and packaged into an image_ probe:v1 .

$ bee build probe.c username/my_probe:v1

3.2 release

Using the packaging capabilities of BTF and OCI, the eBPF code written by BumbleBee is portable and can be embedded into the existing publishing workflow. By pushing the image built by eBPF code to any image warehouse conforming to OCI standard, it can be released to other users.

The following commands realize the function of publishing the image to the image warehouse. When using, you can directly use bee run to run based on the image.

# Push
$ bee push username/my_probe:v1 

# Pull
$ bee pull username/my_probe:v1 

3.3 operation

Using the CLI interface provided by BumbleBee and the images stored in the image warehouse, we can quickly run elsewhere. BumbleBee not only builds user space code, but also uses eBPF map to display log, indicator and histogram information. BumbleBee uses the BTF format self review capability to know which data types need to be displayed.

$ bee run my_probe:v1

Let's use a complete example to experience the convenience that BumbleBee brings us to manage eBPF programs.

4. Complete experience

4.1 bee installation

First of all, we need a Linux operating system that supports BTF kernel. It is recommended to directly use ubuntu 2110 version. The installed kernel already supports BTF by default. If you choose to use Vagrant to manage virtual machines, it is provided in the BumbleBee warehouse Vagrantfile Files can be used directly. Or you can use mulipass The tool directly starts a ubuntu 2110 version of the system.

The script provided by the warehouse is used here. Of course, it can also be installed directly through the git clone warehouse.

In order to quickly experience and avoid permission problems in some scenarios, it is recommended to install directly with root user.

ubuntu@ubuntu21-10:~# curl -sL https://run.solo.io/bee/install | BUMBLEBEE_VERSION=v0.0.9 sh
Attempting to download bee version v0.0.9
Downloading bee-linux-amd64...
Download complete!, validating checksum...
Checksum valid.
bee was successfully installed πŸŽ‰

Add the bumblebee CLI to your path with:
  export PATH=$HOME/.bumblebee/bin:$PATH

Now run:
  bee init     # Initialize simple eBPF program to run with bee
Please see visit the bumblebee website for more info:  https://github.com/solo-io/bumblebee

After installation, the main commands of bee are as follows:

# bee --help
  bee [command]

Available Commands:
  build       Build a BPF program, and save it to an OCI image representation.
  completion  generate the autocompletion script for the specified shell
  describe    Describe a BPF program via it's OCI ref
  help        Help about any command
  init        Initialize a sample BPF program
  login       Log in so you can push images to the remote server.
  run         Run a BPF program file or OCI image.

  -c, --config stringArray   path to auth configs
      --config-dir string    Directory to bumblebee configuration (default "/root/.bumblebee")
  -h, --help                 help for bee
      --insecure             allow connections to SSL registry without certs
  -p, --password string      registry password
      --plain-http           use plain http and not https
      --storage string       Directory to store OCI images locally (default "/root/.bumblebee/store")
  -u, --username string      registry username
  -v, --verbose              verbose output

Use "bee [command] --help" for more information about a command.

4.2 Bee init generates eBPF program

The Bee init command can generate eBPF code scaffolding through the problem Wizard mode. Its function is somewhat similar to libbpf bootstrap, but it is easier to use through the wizard.

$ export PATH=$HOME/.bumblebee/bin:$PATH
# ebpf-test && cd ebpf-test
# bee init
Use the arrow keys to navigate: ↓ ↑ β†’ ←  
? What language do you wish to use for the filter: # Step select the language in which to write eBPF code
  β–Έ C   # Currently only C is supported, and Rust may support it in the future
---------------------------------------------  # Step 2 select eBPF program type
 INFO  Selected Language: C
Use the arrow keys to navigate: ↓ ↑ β†’ ←
? What type of program to initialize: 
  β–Έ Network        # Select the type of eBPF program. Currently, Network and File System are supported
    File system    # The generated templates correspond to TCP_ Connect and open functions
--------------------------------------------- # Step 3 select the map type 
 INFO  Selected Language: C
 INFO  Selected Program Type: Network
Use the arrow keys to navigate: ↓ ↑ β†’ ←
? What type of map should we initialize: 
  β–Έ RingBuffer     
 --------------------------------------------- # Step 4 select the map export type
 INFO  Selected Language: C
 INFO  Selected Program Type: Network
 INFO  Selected Map Type: HashMap
Use the arrow keys to navigate: ↓ ↑ β†’ ←  
? What type of output would you like from your map:
  β–Έ print    # The presentation of map data, such as log printing, counting or index export
 ---------------------------------------------  # Step 5 eBPF program saves the file name
 INFO  Selected Language: C 
 INFO  Selected Program Type: Network
 INFO  Selected Map Type: HashMap
 INFO  Selected Output Type: print
βœ” BPF Program File Location: probe.c  
 ---------------------------------------------- # Finally complete the entire code generation wizard
 INFO  Selected Language: C
 INFO  Selected Program Type: Network
 INFO  Selected Map Type: HashMap
 INFO  Selected Output Type: print
 INFO  Selected Output Type: BPF Program File Location probe.c
 SUCCESS  Successfully wrote skeleton BPF program  
# ls -hl
total 4.0K
-rw-rw-r-- 1 ubuntu ubuntu 2.0K Feb 11 11:33 probe.c

Probe generated by init command C the file format is generally as follows:

#include "vmlinux.h"
#include "bpf/bpf_helpers.h"
#include "bpf/bpf_core_read.h"
#include "bpf/bpf_tracing.h"
#include "solo_types.h"

// 1. Change the license if necessary 
char __license[] SEC("license") = "Dual MIT/GPL";

struct event_t {
	// 2. Add ringbuf struct data here.
} __attribute__((packed));

// This is the definition for the global map which both our
// bpf program and user space program can access.
// More info and map types can be found here: https://www.man7.org/linux/man-pages/man2/bpf.2.html
struct {
	__uint(max_entries, 1 << 24);
	__uint(type, BPF_MAP_TYPE_RINGBUF);
	__type(value, struct event_t);
} events SEC(".maps.print");

int BPF_KPROBE(tcp_v4_connect, struct sock *sk)
	// Init event pointer
	struct event_t *event;

	// Reserve a spot in the ringbuffer for our event
	event = bpf_ringbuf_reserve(&events, sizeof(struct event_t), 0);
	if (!event) {
		return 0;

	// 3. set data for our event,
	// For example:
	// event->pid = bpf_get_current_pid_tgid();

	bpf_ringbuf_submit(event, 0);

	return 0;

Based on the generated code template, we need to fill in our own logic. This is not the key point. Skip the relevant code first. The complete code can be found in the official website Start document View in.

4.3 building eBPF program

Docker or container engine of docker type shall be used in the construction process, which shall be installed in advance.

# apt  install docker.io  # Install docker

# bee build probe.c my_probe:v1
 SUCCESS  Successfully compiled "probe.c" and wrote it to "probe.o"
 SUCCESS  Saved BPF OCI image to my_probe:v1 

In the whole construction process, we don't need to involve related compilation commands such as lang. we just need to input the eBPF program file name and the expected image through the bee build command. After compilation, the binary bytecode probe of eBPF program O is automatically added to the mirror_ Probe: in V1, we can use bee tag to redefine the image warehouse.

4.4 release eBPF procedure

We can publish the image warehouse through the bee tag and push subcommands.

# bee tag my_probe:v1 dwh0403/my_probe:v1
# bee login
# bee push dwh0403/my_probe:v1

Take a look at the above commands. Is there a sense of deja vu?

4.5 running eBPF program

After the image is built, it can be run locally directly through bee run. After running, bee will automatically start the TUI interface to display the map content in the eBPF program we write. The automatically generated map name has some special suffixes, which are used by the program in bee TUI user space to read the data in the corresponding map for display, such as SEC(".maps.print") in the generated code template, Indicates that the map is used for printing.

# bee run my_probe:v1
 SUCCESS  Fetching program from registry: my_probe:v1
 SUCCESS  Loading BPF program and maps into Kernel
 SUCCESS  Linking BPF functions to associated probe/tracepoint
 INFO  Rendering TUI..

5. Summary

So far, we have completed the experience of the whole project function. The bee init tool can help us generate the eBPF code framework through the wizard mode. Although the function is still a little thin, it is a fast and convenient way to use our specific scenarios.

Be build / push / run and other subcommands greatly simplify the steps of compiling commands, packaging images, publishing images and running images, which is very easy to use, greatly reduces the repeated cost of building, publishing and running eBPF programs, and has to praise the author's ideas.

Because the tools generated through bee are based on specific scenes and have limited function richness, they are not suitable for writing eBPF programs and user space programs with rich functions in complex situations, but the overall idea of their construction, release and operation (even some basic functions) can be directly used or used for reference.

6. References

Keywords: Operation & Maintenance Docker Container tools ebpf

Added by DannyTip on Mon, 14 Feb 2022 03:00:34 +0200