8 — Networking by hand

Requirements: root on your host VM, with iproute2. BusyBox ip may lack netns, which hasn’t been checked, so start on the host. The kernel needs VETH and INET.

Hands-on: two namespaces and a cable

A veth pair is a virtual cable: what goes in one end comes out the other (veth(4)).

Step 1: create and wire

ip netns add a
ip netns add b
ip link add va type veth peer name vb
ip link set va netns a
ip link set vb netns b
ip -n a addr add 10.0.0.1/24 dev va
ip -n b addr add 10.0.0.2/24 dev vb
ip -n a link set lo up
ip -n b link set lo up
ip -n a link set va up
ip -n b link set vb up

Expected: ip -n a -br addr shows va UP with its address. ip -n a route shows a connected 10.0.0.0/24 route. You didn’t add that; the kernel did when you added the address.

Step 2: test it

ip netns exec a ping -c1 10.0.0.2

Expected: one reply, 0% loss.

Step 3: fail without sending anything

ip netns exec a ping -c1 8.8.8.8

Expected: “Network is unreachable”. Confirm with ip netns exec a ip -s link show va that TX counters didn’t rise. The kernel refused locally, because no route exists.

Step 4: add a default route

ip -n a route add default via 10.0.0.2

Expected: the same ping is now sent, then times out at b. Same command, different failure.

Step 5: trace one connection

Kernel function names aren’t a stable interface. dev_queue_xmit is an inline, so you have to probe __dev_queue_xmit (netdevice.h).

In terminal 1:

bpftrace -e '
tracepoint:syscalls:sys_enter_connect { printf("connect by %s\n", comm); }
kprobe:tcp_v4_connect { printf(" tcp_v4_connect\n"); }
kprobe:__ip_queue_xmit { printf("  __ip_queue_xmit\n"); }
kprobe:ip_output { printf("  ip_output\n"); }
kprobe:__dev_queue_xmit { printf("  __dev_queue_xmit %s\n", ((struct sk_buff *)arg0)->dev->name); }'

In terminal 2:

ip netns exec b nc -l -p 8000 &
ip netns exec a sh -c 'echo hi | nc -w1 10.0.0.2 8000'

Look for: tcp_v4_connect after the connect line, with the transmit functions nested inside it. The SYN goes out during tcp_v4_connect, through tcp_connect, so these aren’t separate steps. Kernel, Traced chapter 7 follows this path in the source.

Note that sys_enter_connect also fires for AF_UNIX sockets and UDP connect(), so it isn’t quite “every connection”.

Step 6: set the resolver

There’s no 127.0.0.53 here. That’s systemd-resolved’s address, and you don’t run it. musl reads /etc/resolv.conf directly, so being deliberate means writing that file yourself:

printf 'nameserver 10.0.0.2\n' > /etc/resolv.conf

Expected: strace -e trace=openat,connect,sendto nslookup example.com shows the file being read, and a UDP send to that address. Do this in your own system or a namespace, not on a host whose DNS you need.

Step 7: clean up

ip netns del a; ip netns del b

Expected: ip netns list is empty, and the veth pair has gone too.

For a real NIC, use BusyBox’s udhcpc, which needs a script. There’s an example in examples/udhcp.

If it fails

What you see	Cause	Fix
`File exists`	Leftovers from an earlier run	Run Step 7
`Cannot find device "va"`	It moved namespace	Use `ip -n a`
“Network is unreachable” when you expect success	Missed an address or link-up step	Check Step 1
No reply to ping	The other side is down	Check `vb`
`Unknown device type`	`CONFIG_VETH` missing	Rebuild
bpftrace “probe not found”	The function is inlined	`grep -w NAME /sys/kernel/tracing/available_filter_functions`

Principle: end-to-end: the ends are where the meaning is

The kernel between connect() and the wire knows addresses and ports. It does not know why. That’s in the process that called, and in the server that answers. Saltzer, Reed and Clark’s end-to-end argument says functions that need that “why” (correctness, security, meaning) can only be fully implemented at the ends; the middle can help but can’t substitute.