7 — Storage and the page cache

Requirements: chapter 5’s kernel options, veritysetup and mkfs.erofs. Run these on your host VM with sudo, using copies of your images.

The pieces

• The page cache keeps file data in memory after a read (mm concepts).
• ext4 is a writable filesystem.
• EROFS is a read-only image filesystem, designed for images that are “bit-for-bit identical across all deployments” (erofs.rst).
• dm-verity checks blocks against a hash tree “on demand” as they’re read from disk. If a block can’t be verified up to the root hash, the I/O fails (dm-verity).

Decision: an immutable root, because “what’s on the disk” should be a hash, not a history. Your layout is a mount tree: / read-only, with /var and /tmp writable. Mount namespaces come in chapter 9.

Hands-on: watch the cache

1. Create a test file, then empty the cache. Run sync first, because drop_caches skips dirty data (sysctl docs):

   dd if=/dev/urandom of=/lo/out/test.bin bs=1M count=50
   sync; echo 3 | sudo tee /proc/sys/vm/drop_caches

2. In terminal 1, count read requests only. Writes such as journal commits would otherwise muddy the result:

   sudo bpftrace -e 'tracepoint:block:block_rq_issue /strcontains(args.rwbs, "R")/ { @blk = count(); }'

3. In terminal 2, run cat /lo/out/test.bin > /dev/null. Stop terminal 1 with Ctrl-C and note @blk.

4. Expected: a non-zero count.

5. Repeat steps 2 and 3 without dropping the cache.

6. Expected: a count near zero. Anything left over is probably other activity on the system, so explain it rather than calling it a failure.

If strcontains isn’t available on your bpftrace, filter the output afterwards.

Hands-on: protect an image with dm-verity

1. Build a read-only image and its hash tree:

   cd /lo/out
   mkfs.erofs -T 0 -U 11111111-2222-3333-4444-555555555555 root.erofs /lo/rootfs
   truncate -s 64M hash.img
   veritysetup format root.erofs hash.img | tee verity.txt

   Expected: verity.txt has a “Root hash” line. That hash is your trust anchor.

2. Verify and open it:

   ROOT=$(awk '/Root hash:/{print $3}' verity.txt)
   veritysetup verify root.erofs hash.img "$ROOT"; echo "exit=$?"
   sudo veritysetup open root.erofs vroot hash.img "$ROOT"
   sudo mkdir -p /mnt/v && sudo mount -o ro /dev/mapper/vroot /mnt/v
   ls /mnt/v

   Expected: exit 0, and your files are listed.

3. Tamper with a copy. Never point dd at a real disk:

   sudo umount /mnt/v && sudo veritysetup close vroot
   cp root.erofs bad.erofs
   printf 'X' | dd of=bad.erofs bs=1 seek=1048576 conv=notrunc
   veritysetup verify bad.erofs hash.img "$ROOT"; echo "exit=$?"

   Expected: a non-zero exit. If your image is under 1 MiB, lower seek.

4. Open bad.erofs the same way and read every file. Look for: failed reads, and a verity corruption message in dmesg. Write down the exact error, because no source here names it.

If it fails

• Device or resource busy: a stale mapping. Run sudo veritysetup close vroot.
• can't read superblock: no erofs support, or the wrong device.
• “Verification failed” on a good image: the image changed after format, or the root hash is wrong.
• The tampering isn’t noticed: the page cache may be serving good data. Close it, run sync, drop caches, and reopen.

Principle: caches change behaviour, not just speed

The second read wasn’t faster; it was a different operation that happened to return the same bytes. The disk path wasn’t used at all, and that includes verity’s check. Every cache in every system does this: it changes the failure modes, the consistency and the ordering guarantees of the path it sits on. A design that treats a cache as “the same thing but faster” will be surprised exactly when it can least afford to be.