3 — fork() and exec(): making a process

What does the kernel copy when a process forks, what does it merely share, and what exactly happens when the child calls execve?

Requirements: the machine from chapter 0, with bpf in /sys/kernel/security/lsm. It helps to read chapter 5 first.

The fork path

glibc’s fork() is clone() with SIGCHLD | CLONE_CHILD_SETTID | CLONE_CHILD_CLEARTID (arch-fork.h). It arrives at kernel_clone (kernel/fork.c), which calls copy_process. Keep the source open and step through it:

1. dup_task_struct copies the task_struct and allocates a kernel stack, 16 KiB by default or 32 KiB with KASAN (page_64_types.h).
2. copy_creds handles credentials. sched_fork sets up scheduling state, and task_fork_fair prepares the child, though it doesn’t place it yet.
3. copy_files copies the fd table. The child gets the same struct files, so offsets are shared.
4. copy_fs, copy_sighand, copy_signal follow.
5. copy_mm calls dup_mmap, usually the expensive one. Every VMA is copied. copy_page_range copies page tables, marking writable anonymous pages read-only on both sides, which sets up copy-on-write. vma_needs_copy can skip some VMAs entirely (memory.c). CLONE_VM skips all this, and that’s what makes a thread.
6. copy_namespaces, copy_io, copy_thread come next. copy_thread sets the child’s rax to 0, which is why fork() returns 0 there.
7. alloc_pid gets a PID, and the child joins its parent’s cgroup through cgroup_post_fork (chapter 11).

Back in kernel_clone, the sched_process_fork tracepoint fires first, and then wake_up_new_task runs. That function reaches activate_task and place_entity, which is where the child is actually placed (fork.c).

The exec path

do_execveat_common (fs/exec.c) calls alloc_bprm, which opens the file. do_execveat_common then copies the arguments and environment into new stack pages. Next:

1. bprm_execve reaches exec_binprm and search_binary_handler, which tries the registered formats. security_bprm_check runs there. On ELF files, load_elf_binary (fs/binfmt_elf.c) takes over.
2. load_elf_binary reads the headers and finds PT_INTERP, the dynamic loader, if any.
3. begin_new_exec is the point of no return. It kills other threads with de_thread, and exec_mmap releases the old address space. do_close_on_exec closes O_CLOEXEC files before signal handlers are reset. Credentials change later, in commit_creds (exec.c).
4. setup_new_exec and setup_arg_pages build the stack. Each PT_LOAD segment is mapped, and the interpreter too.
5. create_elf_tables writes the auxiliary vector, including AT_PHDR, AT_ENTRY and AT_RANDOM.
6. start_thread sets the instruction pointer to the entry point and the stack pointer to the new stack. The return to user mode lands in a different program, and sched_process_exec fires.

Hands-on: trace it

1. Measure fork latency:

   bpftrace -e 'kprobe:copy_process { @s[tid] = nsecs; }
     kretprobe:copy_process /@s[tid]/ { @us = hist((nsecs - @s[tid]) / 1000); delete(@s[tid]); }'

2. See how it scales. Write f.c, which maps and touches N MiB, forks, and has the child _exit(0). Run it at 10 and 1024, and graph each:

   cd /sys/kernel/tracing
   grep -w -E 'dup_mmap|copy_page_range' available_filter_functions
   echo copy_process > set_graph_function; echo 6 > max_graph_depth
   echo function_graph > current_tracer; ~/f 1024; cp trace /tmp/f1g.txt
   echo nop > current_tracer; echo > set_graph_function

   Look for: where the time goes. If dup_mmap is missing on your build, read copy_page_range instead.

3. Prove the order of events:

   bpftrace -e 'tracepoint:sched:sched_process_fork { printf("fork tp %d\n", args.child_pid); }
                kprobe:wake_up_new_task { printf("wake\n"); }
                tracepoint:sched:sched_prepare_exec { printf("prepare_exec\n"); }
                kprobe:begin_new_exec { printf("begin_new_exec\n"); }
                tracepoint:sched:sched_process_exec { printf("exec tp\n"); }'

   Expected: fork tp before wake, then begin_new_exec, prepare_exec and exec tp. Run bpftrace -l first to confirm sched_prepare_exec exists.

4. Count mappings. Compare the count with readelf -lW /bin/true | grep LOAD, plus the interpreter’s segments:

   bpftrace -e 'kprobe:vm_mmap /comm == "true"/ { @ = count(); }'

   Don’t assume one mapping per segment. Count, then explain the number.

Hands-on: change it

Write an LSM-BPF program that refuses to run one file.

1. Generate vmlinux.h:

   bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h

2. Write a libbpf program with SEC("lsm/bprm_check_security") that returns -EPERM when the file’s inode matches one you choose.

3. Attach it, then try to run that file.

4. Expected: execve should return EPERM, the old process should survive to see the error, and kprobe:begin_new_exec should not fire. The refusal comes after the file was opened and before the point of no return.

5. Compare it with the seccomp block from chapter 1: that’s earlier and blunter, with no knowledge of the file.

If it fails

• Attach succeeds but nothing is blocked: bpf isn’t in the active LSM list.
• Odd load_elf_binary results: some builds have native and compat versions.
• Mapping counts look high: a dynamic binary includes the loader’s mappings.

The takeaway

Copy what must diverge; share what won’t; defer the rest. fork shares the fd table’s files, the signal handlers, and (until written) every page; it copies only the metadata needed to let the two processes go their own way. exec then discards almost all of it. The design is a worked example of choosing, for each piece of state, between copy, share and copy-on-write: the decision every cache, every snapshot and every container image makes.