Inside the GPU
Follow one kernel from silicon to token.
PyTorch hides a machine. Underneath it are 132 streaming multiprocessors, a register file that makes context switches free, and a memory bus that decides how fast a model can talk. This track goes through that machine in order: the hardware first, then the CUDA execution model, then the tricks that get past it, then eBPF, which finally lets Linux see inside.
It ends where the hardware meets real workloads: why decode speed depends on memory bandwidth rather than FLOPS, why FlashAttention exists, and how the whole stack gets packaged.
This track assumes some C and a rough idea of how a CPU pipeline works. The Linux tracks and this one are built to meet in the middle: page tables, DMA, the BPF verifier and the scheduler all show up on both sides.
Chapters
- 1From pixels to computeBefore CUDA, doing math on a GPU meant lying to it. How a graphics pipeline became a general-purpose processor.Draft
- 2Inside the siliconRead a real GPU's inventory, follow its memory hierarchy, and provoke a bank conflict you can measure.Draft
- 3The CUDA execution modelThreads, blocks and grids, what a warp really does at an if statement, and a lab that measures the cost of divergence.Draft
- 4Below the threadCompile one addition from C++ to machine code, decode its scheduling bits, then program the warp directly.Draft
- 5Where eBPF meets CUDALinux can barely see a GPU. Attach eBPF to the CUDA runtime and driver, decode a kernel launch, and read what the research tools do.Draft
- 6Why LLMs need all of thisWork out the ratio behind LLM inference by hand, see why batching changes everything, and test it on a real model.Draft
- 7Packaging the whole stackKernel, drivers, FUSE, eBPF fabrics, model weights and agents shipped as one immutable appliance.Draft