1 — The cross-toolchain

Requirements: chapter 0, a few hours, and these tarballs in /lo/sources, with hashes checked: binutils 2.47, GCC 16.2.0, Linux 7.3 and musl 1.2.6 plus upstream patches.

Why it takes five steps

There’s a chicken-and-egg problem. A full compiler needs a C library, and the C library needs a compiler. So you build a minimal compiler first, use it to build the library, then rebuild the compiler.

Why cross-compile at all? LFS puts it well: “anything that is cross-compiled cannot depend on the host environment” (LFS).

Two pieces provide the isolation:

• The target name can never equal the host’s, so the build has to be a cross build.
• --with-sysroot makes GCC look in /lo/rootfs for headers and libraries, instead of /usr (GCC docs).

Step 1: set up the shell

export LO=/lo TGT=x86_64-lo-linux-musl
export PATH=$LO/cross/bin:$PATH LC_ALL=POSIX
mkdir -p $LO/rootfs/usr/include $LO/build

Re-run this in every new shell. A missing PATH is the commonest cause of failures here.

Step 2: build binutils

cd $LO/build && tar xf $LO/sources/binutils-2.47.tar.xz
mkdir binutils-build && cd binutils-build
../binutils-2.47/configure --prefix=$LO/cross --target=$TGT \
  --with-sysroot=$LO/rootfs --disable-nls --disable-werror \
  --disable-multilib --enable-deterministic-archives
make -j"$(nproc)" && make install

Expected: $TGT-ld --print-sysroot prints /lo/rootfs.

Step 3: build GCC, first pass

This compiler handles C only and has no C library. --with-newlib tells libgcc to leave out anything that needs one (LFS):

cd $LO/build && tar xf $LO/sources/gcc-16.2.0.tar.xz
cd gcc-16.2.0 && ./contrib/download_prerequisites && cd ..
mkdir gcc1 && cd gcc1
../gcc-16.2.0/configure --prefix=$LO/cross --target=$TGT --with-sysroot=$LO/rootfs \
  --with-newlib --without-headers --disable-shared --disable-threads \
  --disable-multilib --disable-libatomic --disable-libgomp --disable-libquadmath \
  --disable-libssp --disable-libsanitizer --enable-languages=c
make -j"$(nproc)" all-gcc all-target-libgcc
make install-gcc install-target-libgcc

Expected: $TGT-gcc -print-libgcc-file-name prints a path to a libgcc.a that exists.

Step 4: install the kernel headers

Userland and the kernel have to agree on the ABI, and this is where they do:

cd $LO/build && tar xf $LO/sources/linux-7.3.tar.xz && cd linux-7.3
make ARCH=x86_64 headers_install INSTALL_HDR_PATH=$LO/rootfs/usr

Expected: ls $LO/rootfs/usr/include/linux/version.h succeeds.

Step 5: build musl

cd $LO/build && tar xf $LO/sources/musl-1.2.6.tar.gz && cd musl-1.2.6
# apply upstream security patches here
./configure --host=$TGT --prefix=/usr --syslibdir=/lib \
  CROSS_COMPILE=$TGT- LIBCC="$($TGT-gcc -print-libgcc-file-name)"
make -j"$(nproc)" && make DESTDIR=$LO/rootfs install

Expected: ls -l $LO/rootfs/lib/ld-musl-x86_64.so.1 shows a link to libc.so, and ls $LO/rootfs/usr/lib/crt1.o succeeds.

Step 6: build GCC, second pass

Now that there’s a C library, you can build the full compiler:

cd $LO/build && mkdir gcc2 && cd gcc2
../gcc-16.2.0/configure --prefix=$LO/cross --target=$TGT --with-sysroot=$LO/rootfs \
  --enable-languages=c,c++ --enable-shared --enable-threads=posix --enable-tls \
  --disable-multilib --disable-libsanitizer --disable-gnu-indirect-function \
  --enable-initfini-array --enable-libstdcxx-time=rt
make -j"$(nproc)" && make install

GCC 16 defaults to C++20 (GCC 16 changes), so older C++ packages may need an explicit -std=.

Step 7: test it

cd /lo/out
printf '#include <stdio.h>\nint main(void){puts("hello");return 0;}\n' > hello.c
$TGT-gcc hello.c -o hello-dynamic
$TGT-gcc -static hello.c -o hello-static
$TGT-readelf -l hello-dynamic | grep interpreter
./hello-static
$LO/rootfs/lib/ld-musl-x86_64.so.1 ./hello-dynamic

Expected: the readelf line names /lib/ld-musl-x86_64.so.1, and both programs print hello.

Don’t use ldd on the dynamic binary. Your host’s glibc ldd won’t understand it.

If it fails

If you get stuck, a prebuilt toolchain from the musl-cross-make fork lets you continue. It targets x86_64-linux-musl, so the prefix changes, and its versions lag those here.

Observe: what does a compiler do?

gcc is a driver. It runs other programs to do the work. Watch it:

1. Start a trace, filtering on the first 15 characters, because comm holds no more than that (sched.h):

   sudo bpftrace -e 'tracepoint:syscalls:sys_enter_execve /comm == "x86_64-lo-linux" || comm == "cc1" || comm == "as" || comm == "collect2" || comm == "ld"/ { printf("%s execs %s\n", comm, str(args.filename)); }'

2. In another shell, run $TGT-gcc hello.c -o hello-dynamic.

3. Look for: the compiler running cc1, then the assembler, then the linker stage. Note the real paths. Don’t assume the order until you’ve seen it.

4. Now watch cc1 search for headers. Count the failed openat calls. That’s the cost of a long -I list.

Principle: isolation is a boundary you can name

A private target name and a sysroot turn “don’t accidentally use the host” from a discipline into a structure. The same move shows up everywhere good systems isolate things: a separate namespace, a separate account, a separate network. Crossing the boundary takes an explicit act rather than an omitted flag. If your isolation depends on everyone remembering something, it isn’t isolation.