cilium/ebpf is one of the best Golang BPF libraries currently on the market. It can read, modify and load eBPF programs and maps and attach them to various hooks in the Linux Kernel.
dropbox/goebpf, another Golang eBPF library,
cilium/ebpf is advanced in compiling eBPF code directly from Go’s
go generate ability. The library provides a handy tool called
bpf2go that manages the compiling process of eBPF code as well as the generation of Golang bindings. Due to the generation of Golang bindings, its user doesn’t need to even know where the eBPF objects reside in. It conceals the detail of managing elf object files and gives its user the Golang interface to interact with directly.
Sample TC BPF code
We use the snippet below to illustrate the process of composing, compiling, and loading a TC BPF program. The BPF program is written in C language, and for the curious readers who wonder how a BPF code is compiled into an ELF file, I will give the equivocal llvm command for building.
The sample code can be found in my Github Repo:
Helper Header Files
BPF framework has provided a bunch of helper functions that can be invoked in BPF code directly to call for some specific OS abilities, such as redirecting a packet to another interface. The full list of these functions can be found in
man 7 bpf-helpers manual. It is worth mentioning that the list varies from version to version of the Linux Kernel. So it is better to consult your kernel’s source code before taking it.
The good news is that the Linux Kernel provides a script to generate the C header files specific to that version automatically:
# Run this command at the root of Linux Kernel source.
make -C tools/lib/bpf
Run the command above at the root of your kernel source, and the header files will be generated as
tools/lib/bpf directory. User should use
tools/lib/bpf/bpf_helpers.h header file instead of referencing
bpf_helper_defs.h directly. The
tools/lib/bpf/bpf_helpers.h file defines many macros and constants that are essential to any BPF code, and it includes
bpf_helper_defs.h as well.
Save the snippet above as bpf.c, and then compile it into elf object file directly by
clang -O2 -emit-llvm -c bpf.c -o - | llc -march=bpf -filetype=obj -o bpf.o
Inspect the output binary file:
bpf.o: ELF 64-bit LSB relocatable, eBPF, version 1 (SYSV), not stripped
Load the TC object binary by
It is handy to attach a TC BPF binary to a specific interface by using
tc command manually. We have to create a
clsact Qdisc for our target interface firstly:
tc qdisc show dev eth0
Then, attach our BPF binary to that interface’s ingress direction as a
tc filter add dev eth0 ingress bpf object-file bpf.o section tc_prog direct-action
Check if it is successful:
tc filter show dev eth0 ingress
To view the log message dumped by BPF code, read
/sys/kernel/debug/tracing/trace_pipe file by
cat or other tools.
Compile BPF code by
Create a new Golang module, and put the BPF C files in
go mod init ebpf
go get github.com/cilium/ebpf/cmd/bpf2go
The Golang Part
Generate BPF object files by running
go generate. Then, Four new files can be found in the same directory named as
ebpf_bpfel.o respectively. The last char,
b, of file names stands for the byte-order the file caters for. The
.go files contain bindings to BPF programs and maps. The
.o files are BPF ELF object binaries.
Load BPF into the Kernel
It is time to write our main function which attaches the BPF objects we just generated to the interface. The same as above, the ingress direction of eth0 is chosen as our target.
Load BPF programs and maps in BPF object binary into the kernel is pretty simple:
tcObjects type and
loadTcObjects() function are generated by
loadTcObjects() function loads the programs and maps in BPF object file into the kernel. This function will fail if the BPF code cannot pass the verification of the BPF verifier in the kernel space.
Every BPF program and map loaded into the kernel are granted a handler, i.e. a file descriptor, which is an identifier to that BPF object. The file descriptor is an integer index the same as an opened file. For our scenario, to get the file descriptor of our
tcObjects, use the line of code below.
progFd := objs.tcPrograms.TcMain.FD()
progFd := objs.TcMain.FD()
Attach BPF to interface by netlink
Attaching the BPF program to a tc classifier is different from attaching BPF program to Kprobe point. To attach a BPF program to an interface, we have to ask
netlink for help.
netlink is a communication method provided by the Linux Kernel to bridge the userspace and kernel space for high-efficiency data transfer.
ip command and its relatives, e.g.
tc command, are
netlink based. To Attach our BPF program to the
eth0 interface, we need to take advantage of
netlink as well.
There has already been a mature Golang library named
netlink that provides almost full functional netlink abilities to communicate with the kernel.
go get github.com/vishvananda/netlink
As we have operated before, a
clsact Qdisc must be created as an attach point. If you followed my doc from the beginning, the
clsact Qdisc should have been created. If so, please delete it to prevent getting errors afterward:
tc qdisc del dev eth0 clsact
Then, the Golang code. I have commented on the code detailedly to tell the process.
Put all together, compile and run
The complete code of
main.go is pasted below with comments stripped:
Compile and run it on your Linux host:
go generate && go build -o a.out main.go tc_bpfel.go && sudo ./a.out