Ashok G

Kernel, Traced · 6.12, x86_64 · Draft

0 — Set up the machine

Requirements:

  • An x86-64 Linux host with KVM, about 30 GB of free disk and a few hours for the first build.
  • Build tools: gcc, make, flex, bison, bc, the libelf and libssl headers, pahole, and qemu-system-x86_64.

If you only have a Mac or a machine without KVM, you can still do most of the logic, but QEMU will be slow and any timing you measure will mean nothing (QEMU docs).

Step 1: get the exact source

Download the same version these chapters were checked against:

curl -LO https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.12.tar.xz
tar xf linux-6.12.tar.xz && cd linux-6.12

These chapters were read at tag v6.12. The current 6.12 longterm release has moved on since then, so small details can differ on later 6.12.y kernels.

Step 2: configure it

Start from the default config, add the virtual machine drivers, then turn on tracing and BPF:

make x86_64_defconfig
make kvm_guest.config
scripts/config -e DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT -e DEBUG_INFO_BTF \
  -e FTRACE -e FUNCTION_TRACER -e FUNCTION_GRAPH_TRACER \
  -e KPROBES -e KPROBE_EVENTS -e UPROBE_EVENTS \
  -e BPF -e BPF_SYSCALL -e BPF_JIT -e BPF_JIT_ALWAYS_ON -e BPF_EVENTS -e BPF_LSM \
  -e SCHED_DEBUG -e SCHED_CLASS_EXT -e SECCOMP -e SECCOMP_FILTER \
  --set-str LSM "landlock,lockdown,yama,loadpin,safesetid,bpf"
make olddefconfig

make olddefconfig silently drops any option whose dependencies aren’t met. So check that the important ones survived:

for o in DEBUG_INFO_BTF FUNCTION_GRAPH_TRACER BPF_LSM SCHED_CLASS_EXT SCHED_DEBUG; do
  printf "%s=" $o; scripts/config -s $o; done

Each line must print y. If DEBUG_INFO_BTF is missing, install pahole.

Step 3: build it

make -j"$(nproc)" bzImage

You should end up with arch/x86/boot/bzImage.

Step 4: get a disk

Use a real disk image, because chapters 2 and 6 need ext4 on a virtio disk:

curl -LO https://cloud.debian.org/images/cloud/trixie/latest/debian-13-nocloud-amd64.qcow2
qemu-img resize debian-13-nocloud-amd64.qcow2 10G

Step 5: boot it

qemu-system-x86_64 -enable-kvm -cpu host -smp 2 -m 4G -nographic \
  -kernel arch/x86/boot/bzImage \
  -append "console=ttyS0 root=/dev/vda1 rw nokaslr" \
  -drive file=debian-13-nocloud-amd64.qcow2,if=virtio,format=qcow2 \
  -nic user,model=virtio-net-pci

Check the image’s root partition with lsblk, and change root= if it isn’t vda1. Press Ctrl-a then x to quit QEMU.

Inside the guest, install the tools:

apt-get update && apt-get install bpftrace trace-cmd build-essential

Step 6: verify everything

uname -r                                  # expect 6.12.0
ls /sys/kernel/btf/vmlinux                # BTF present
cat /sys/kernel/tracing/current_tracer    # expect nop
cat /sys/kernel/security/lsm              # must include bpf
cat /sys/kernel/mm/transparent_hugepage/enabled
bpftrace --version

Write these results down. Tool versions matter: bpftrace behaves differently across releases, and Debian 13 ships 0.23.2 (Debian). These chapters use the args.field form.

The one rule for every experiment

Before you attach to any function, check that it exists on your build:

grep -w NAME /sys/kernel/tracing/available_filter_functions

If it prints nothing, the function was inlined or is marked untraceable. This happens a lot, and it isn’t an error in your setup. Many of the functions in these chapters are static, so they appear or vanish depending on the compiler and config. A symbol’s presence doesn’t guarantee a probe will attach either, so treat every failure as information.

If something goes wrong

What you seeLikely causeFix
No available_filter_functionsFUNCTION_TRACER was droppedRerun the check in Step 2
“unable to mount root”Wrong root=, or no VIRTIO_BLKCheck lsblk, and rerun make kvm_guest.config
-enable-kvm failsNo KVM on the hostRemove -enable-kvm -cpu host, and ignore timings
bpftrace “could not resolve” a structDEBUG_INFO_BTF is offInstall pahole, reconfigure and rebuild
Boot hangs waiting for rootThe image may need an initramfsNot yet tested. Try virtme-ng, with a disk added for chapters 2 and 6

Next

Start with chapter 1. For new engineers, a good order is 1, 2, 5, 6, 3, 4, because chapters 2, 5 and 6 build the same habit and chapter 4 has the most ways to mislead you. Then take 7 to 12 in order.