7 — Packaging the whole stack

Modern AI infrastructure has moved away from fragile, imperative, script-heavy provisioning (“dependency hell”) toward immutable, declarative, and pre-packaged full-stack appliances. Instead of treating the Linux operating system, network drivers, storage mounts, model weights, and agent orchestration as loosely coupled operational tiers, platform engineering teams now compile and ship them as unified Open Container Initiative (OCI) artifacts, bootable OS images, and hardware-isolated microVMs.

This architectural evolution is structured as a 5-Layer Composable AI Stack:

1. The OS & Kernel Layer: Bootable containers (bootc) and distroless agent-ready Linux distributions (such as Fedora Hummingbird and RHEL AI) that version the Linux kernel, GPU drivers, and root filesystem as a single immutable OCI image.
2. The Storage & VFS Layer: Userspace filesystems (FUSE) and range-request OCI indexers (SOCI and eStargz) that eliminate multi-minute container pull times by streaming multi-gigabyte model weights and remote object storage directly into memory on demand.
3. The Networking & Telemetry Layer: Kernel-native eBPF datapaths (Cilium 1.20 netkit) and containerized RDMA/RoCEv2 network operators that bypass host iptables and TCP/IP stacks to provide line-rate GPU-to-GPU communication and synchronous kernel-level runtime enforcement (Tetragon).
4. The Agentic & Model Packaging Layer: Declarative agent servers (LangGraph CLI) that compile state graphs, vector checkpointers, and A2A/MCP endpoints into self-contained appliances, paired with CNCF ModelPack OCI artifacts and NVIDIA NIM Blueprints.
5. The Cutting-Edge Frontier: Sub-second GPU memory checkpoint/restore (CRIUgpu + cuda-checkpoint), hardware-isolated Firecracker microVM agent sandboxes (E2B), containerized Docker MCP Catalog tool servers, and cycle-driven GPU cluster schedulers (KAI Scheduler and Kueue).

1. The Foundation: Pre-Packaging Linux & the Kernel Layer

Historically, provisioning an AI compute node required installing a base Linux distribution, manually aligning kernel headers, compiling DKMS modules for NVIDIA drivers, configuring CUDA toolkits, and fighting configuration drift across cluster nodes. Today, the entire operating system is packaged using the exact same primitives as an application container.

Image-Mode Linux & Bootable Containers (bootc)

Using bootc (Bootable Containers) and ostree, platform engineers write a standard Containerfile (or Dockerfile) that includes the Linux kernel, systemd, hardware drivers, and system libraries, and build it into a standard OCI image.

• Atomic Transactions & Rollbacks: When deployed to bare metal or cloud VMs, the root filesystem (/) is mounted as strictly read-only via Composefs/OSTree, while mutable state is isolated exclusively to /var and /etc. System updates are pulled from standard container registries and staged in the background, applying atomically on reboot. This guarantees zero partial-update states and instant rollbacks if a driver or kernel update degrades GPU or eBPF performance.
• Pre-Compiled Driver Alignment: By baking the exact Linux kernel version alongside pre-compiled NVIDIA GPU kernel modules (nvidia.ko, nvidia-uvm.ko, nvidia-peermem.ko), Mellanox OFED (MOFED) networking drivers, and the NVIDIA Container Toolkit directly into the bootc image during CI/CD, runtime DKMS compilation errors and kernel-header mismatches are completely eliminated.

# Example: Building a Pre-Packaged Bootable AI Host OS with bootc
FROM quay.io/fedora/fedora-bootc:42

# 1. Install pre-compiled kernel modules, NVIDIA drivers, and containerd snapshotters
RUN dnf install -y \
    kernel-modules-extra \
    kmod-nvidia-open-dkms \
    nvidia-container-toolkit \
    containerd-stargz-grpc \
    cilium-cli \
    && dnf clean all

# 2. Pre-configure containerd for FUSE eStargz lazy-loading and NVIDIA runtime
COPY ./etc/containerd/config.toml /etc/containerd/config.toml
COPY ./etc/modules-load.d/nvidia-rdma.conf /etc/modules-load.d/nvidia-rdma.conf

# 3. Enable core services at boot
RUN systemctl enable containerd stargz-snapshotter nvidia-persistenced

# Validate bootable container layout
RUN bootc container lint

Distroless & Agent-First Linux Distributions

• Fedora Hummingbird: Designed specifically for cloud-native and autonomous AI workloads, Fedora Hummingbird applies a distroless, near-zero-CVE philosophy to both application containers and the underlying host OS via bootc. It ships a catalog of 49 distroless base images (and 157 variants including FIPS-validated and multi-architecture builds) covering Python, Rust, Go, Node.js, and PostgreSQL. Each image strips out package managers, shells, and unnecessary binaries — drastically shrinking the attack surface if an AI agent is compromised — while pairing every build with machine-readable vulnerability feeds and comprehensive Software Bills of Materials (SBOMs).
• RHEL AI & InstructLab Appliances: Enterprise distributions like RHEL AI package the OS, fine-tuning frameworks (InstructLab), vLLM/Triton serving engines, and hardware-specific accelerations (AMD ROCm, Intel Gaudi, NVIDIA Hopper/Blackwell) into ready-to-boot cloud AMIs, QCOW2 images, and bare-metal ISOs.

2. The Data & Filesystem Layer: Packaging FUSE for Zero-Copy AI

In AI workloads, container images and model artifacts regularly range from 15 GB to over 500 GB. In a traditional container runtime (containerd or CRI-O), the node must download the entire compressed tarball and unpack every layer to local disk before the container process can even execute main(). FUSE (Filesystem in Userspace) has become the linchpin technology for bypassing this I/O bottleneck.

Lazy-Loading OCI Container Snapshotters (eStargz & AWS SOCI)

Instead of waiting for full image downloads, modern AI clusters pre-package FUSE-based containerd snapshotters directly into the node OS:

• Stargz Snapshotter (eStargz): Converts standard OCI images into seekable gzip archives (nerdctl image convert --estargz --oci). The stargz-store daemon mounts the container rootfs as a FUSE filesystem backed by the remote container registry. In production benchmarks deploying a 30B parameter Qwen3 model on vLLM, switching from a standard OCI image to an eStargz FUSE-mounted image slashed Kubernetes PodReady startup times from 4 minutes down to 33 seconds.
• AWS SOCI (Seekable OCI): Rather than reformatting the image layer itself, SOCI builds an external range-request index artifact (zTOC) stored in the registry alongside the untouched OCI image. Empirical benchmarks demonstrate a decisive performance crossover at 80% access density — because containerized Python/CUDA environments typically read only 15–25% of their bundled files during initialization, lazy-loading via FUSE HTTP Range requests consistently outperforms parallel full pulls.

# /etc/containerd/config.toml - Enabling FUSE Lazy-Loading via Stargz Snapshotter
version = 2

[plugins."io.containerd.grpc.v1.cri".containerd]
  snapshotter = "stargz"
  disable_snapshot_annotations = false

[proxy_plugins]
  [proxy_plugins.stargz]
    type = "snapshot"
    address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"

Object Storage FUSE Connectors & Memory-Mapped Weights

Rather than baking multi-hundred-gigabyte model weights into container layers or copying terabytes of training shards onto local NVMe drives, AI pipelines pre-package specialized FUSE clients inside their runtime environments:

• Mountpoint for Amazon S3 & Cloud Storage FUSE (GCS FUSE): High-throughput FUSE clients written in Rust and Go translate standard POSIX open(), lseek(), and sequential/random read() system calls directly into parallelized multi-part S3/GCS REST API calls. Modern releases of Mountpoint for S3 include strict memory usage bounds and tiered read-only metadata/data caching across local NVMe (--cache) or shared cluster memory.
• Zero-Copy safetensors via FUSE + mmap(): Legacy PyTorch .bin (pickle) files required reading the entire file sequentially into host RAM before deserialization. Modern weight formats like Hugging Face safetensors store a lightweight JSON header in the first few kilobytes defining exact byte offsets and shapes for every tensor. When PyTorch or vLLM executes mmap() on a safetensors file sitting on a FUSE mount, the Linux kernel triggers page faults only for the specific byte ranges requested — streaming tensor slices directly from object storage into host memory (and onward to GPU VRAM via pinned memory / GPUDirect) without ever writing a full intermediate copy to local disk.

3. The Network Fabric: Pre-Packaging High-Speed AI Networking

AI workloads stress network fabrics across two distinct planes:

1. East-West GPU-to-GPU Traffic: Massive collective communications (AllReduce, AllGather via NCCL) during distributed tensor/pipeline parallel inference and training.
2. North-South & East-West Agentic API Traffic: High-concurrency HTTP/2, gRPC, WebSocket, and MCP tool calls, vector database queries, and streaming SSE token responses.

Both network planes can now be packaged declaratively at the kernel and operator levels.

Kernel-Native eBPF Networking (Cilium 1.20 & netkit)

Traditional Kubernetes networking relies on iptables or nftables, where rule evaluation overhead scales poorly as Pod and Service counts grow, adding tail latency to microservice-heavy agent architectures.

• Pre-Packaged eBPF CNI: Modern AI platforms bundle Cilium as the default CNI. By intercepting sockets (sockops) and processing packets directly inside the Linux kernel via eBPF maps, Cilium bypasses the legacy netfilter stack, delivering a 30–40% throughput improvement and significantly lower tail latencies.
• Automatic netkit Device Selection: With Cilium 1.20 running on Linux kernels 6.8+, setting bpf.datapathMode=auto instructs the Cilium agent to probe the host kernel at startup and automatically replace traditional virtual ethernet (veth) pairs with netkit — a kernel-native BPF-programmable network device. Unlike veth, which incurs softirq backlog processing when crossing from container namespace to host namespace, netkit executes BPF programs directly inside the transmit routine without context switching.
• Sidecarless Service Mesh: By combining Cilium or Istio Ambient Mesh with eBPF socket redirection, L7-aware policies (HTTP/gRPC rate limiting, mTLS identity verification, and egress gateway routing) are enforced at the node level, eliminating the need to package and run resource-heavy Envoy sidecar proxies inside every AI container.

Containerized RDMA, RoCEv2, and GPUDirect

For multi-node inference (such as running DeepSeek-R1 671B or Llama-4 across multiple nodes via Tensor and Expert Parallelism), standard TCP/IP over Ethernet introduces unacceptable CPU serialization bottlenecks. Using the NVIDIA Network Operator, the entire high-speed networking stack is packaged as declarative Kubernetes Custom Resources (NicClusterPolicy):

• Containerized Driver Lifecycle: Automatically deploys Mellanox OFED (MOFED) drivers, GPUDirect RDMA kernel modules (nvidia-peermem), and RDMA Shared Device Plugins inside privileged DaemonSet containers.
• Secondary High-Speed Networks: Uses Multus CNI and SR-IOV device plugins to attach dedicated Virtual Functions (VFs) directly into AI inference pods. This enables containers to establish direct InfiniBand or RoCEv2 (RDMA over Converged Ethernet) channels where SmartNICs/SuperNICs (e.g., ConnectX-7 / BlueField-3) read and write directly to GPU VRAM over the PCIe switch, completely bypassing the host CPU and OS kernel.

4. Packaging AI Frameworks: LangGraph & OCI Model Artifacts

At the application layer, AI frameworks have transitioned from loose collections of Python scripts into standardized, compilable server appliances and registry-native artifacts.

Declarative Agent Appliances with langgraph-cli

LangChain’s LangGraph CLI (langgraph-cli) turns stateful, cyclic, multi-actor agent workflows into turnkey, production-ready Docker appliances without requiring developers to write boilerplate FastAPI servers, database migration scripts, or distributed queue workers.

By defining a single langgraph.json manifest in the repository root, executing langgraph build -t my-org/research-agent:v1 compiles a complete, self-contained Agent Server OCI image:

{
  "$schema": "https://langgra.ph/schema.json",
  "dependencies": [
    "langchain_openai",
    "langchain_anthropic",
    "./src/research_agent"
  ],
  "graphs": {
    "research_agent": "./src/research_agent/graph.py:graph"
  },
  "env": "./.env",
  "python_version": "3.12",
  "base_image": "langchain/langgraph-api:3.12-wolfi",
  "api_version": "0.2",
  "auth": {
    "path": "./src/research_agent/auth.py:auth_handler"
  },
  "encryption": {
    "path": "./src/research_agent/security.py:blob_encryptor"
  },
  "checkpointer": {
    "ttl": {
      "strategy": "delete",
      "sweep_interval_minutes": 60,
      "default_ttl": 10080
    }
  },
  "store": {
    "index": {
      "dims": 1536,
      "embed": "openai:text-embedding-3-small",
      "fields": ["metadata.summary", "content.observations"]
    },
    "ttl": {
      "refresh_on_read": true,
      "default_ttl": 43200
    }
  },
  "http": {
    "disable_mcp": false,
    "disable_a2a": false,
    "disable_meta": false,
    "disable_ui": true,
    "cors": {
      "allow_origins": ["https://internal.enterprise.ai"]
    }
  },
  "webhooks": {
    "UrlPolicy": {
      "allowed_domains": ["hooks.enterprise.ai"],
      "disable_loopback": true,
      "require_https": true
    }
  }
}

When packaged via langgraph build, the resulting container automatically bundles and configures:

1. Deterministic Dependency Resolution: Native integration with uv (UvLockPlan) and pip to deterministically resolve both remote PyPI packages and local monorepo dependencies, alongside custom dockerfile_lines for OS-level libraries.
2. Pre-Packaged State & Semantic Memory (StoreConfig & CheckpointerConfig): Built-in PostgreSQL (langgraph-checkpoint-postgres) and Redis persistence layers that manage thread state checkpointing, automatic Time-To-Live (TTLConfig / ThreadTTLConfig) garbage collection sweeps, custom at-rest state encryption (EncryptionConfig), and automatic vector embedding generation (IndexConfig) on specific JSON paths for long-term cross-thread memory.
3. Standardized Agent Protocols (MCP & A2A): The packaged server natively exposes REST and streaming SSE endpoints for /runs, /threads, /assistants, and /store, alongside built-in protocol endpoints for the Model Context Protocol (/mcp) (exposing the graph itself as an MCP tool server) and Agent-to-Agent (/a2a/*) interoperability. Developers can harden the appliance for production by toggling route disable flags (disable_meta, disable_ui, disable_mcp, disable_a2a) and strict webhook SSRF/egress validation policies (allowed_domains, disable_loopback) directly in langgraph.json.

OCI Model Packaging: CNCF ModelPack & Docker Model Runner

Historically, storing LLMs inside Docker images meant wrapping a 40 GB weight file into a compressed .tar.gz filesystem layer — causing massive disk bloat (requiring 2x disk space to download and extract the tarball) and tightly coupling the model weights to a specific inference engine container build.

• CNCF ModelPack & KitOps: Accepted into the CNCF Sandbox in June 2025 (spearheaded by Jozu alongside Red Hat, PayPal, Ant Group, and ByteDance), ModelPack is the vendor-neutral specification for packaging AI/ML models, datasets, code, and configuration (Kitfile / Modelfile) as native OCI Artifacts. Using KitOps (kit pack / kit unpack), enterprises push and pull versioned model artifacts through their existing container registries (Harbor, AWS ECR, Quay, Artifactory) with existing RBAC, Cosign cryptographic signatures, and air-gapped security controls.
• Docker Model Runner OCI Spec: Docker introduced specialized OCI artifact media types — such as application/vnd.docker.ai.model.config.v0.1+json and application/vnd.docker.ai.gguf.v3 — that store raw model weights (such as GGUF or safetensors) as uncompressed, non-TAR layers. Because the layer is stored as a raw byte stream rather than a tar archive, containerd and local inference runners can mount or mmap() the raw weight file directly from the local registry blob cache without extraction overhead.

# Example Kitfile (CNCF ModelPack / KitOps) for Packaging an OCI Model Artifact
manifestVersion: 1.0.0
package:
  name: qwen3-30b-a3b-instruct
  version: 1.2.0
  description: Production Qwen3 MoE model packaged with LoRA adapters and system prompt
model:
  name: qwen3-30b-a3b-fp8
  path: ./weights/model.safetensors
  license: Apache-2.0
  framework: safetensors
code:
  - path: ./tokenizers/
    description: Fast Rust tokenizer configuration
datasets:
  - name: enterprise-eval-set
    path: ./eval/golden_prompts.parquet

Turnkey Inference Microservices: NVIDIA NIM & AI Blueprints

For hardware-optimized deployment, NVIDIA NIM (Inference Microservices) pre-packages foundation models (such as Llama-3.3-Nemotron, DeepSeek-R1, FLUX, or Cosmos) with pre-compiled TensorRT-LLM engines, vLLM/Triton runtimes, and dynamic LoRA adapter loaders into a single container exposing standard OpenAI-compatible APIs. Expanding on this, NVIDIA AI Blueprints package multi-agent orchestration, NeMo Guardrails, vector databases, and multiple NIMs into deployable Helm charts and reference architectures.

5. Cutting-Edge AI Industry Features You Can Now Pre-Package

Beyond Linux, FUSE, networking, and LangGraph, five major architectural innovations are reshaping how AI systems are packaged, secured, and scaled in production.

5.1 GPU Process Checkpointing & Pre-Compiled Kernel Caches (Sub-Second Cold Starts)

Even when FUSE eliminates container image download time, starting a 70B+ LLM in vLLM or SGLang still incurs a 40-to-120-second cold start due to Python module imports, reading weights into VRAM, PyTorch torch.compile Inductor kernel compilation (30–90s), and CUDA graph capture (10–30s). Two packaging techniques eliminate this startup tax:

1. Pre-Packaged Inductor & CUDA Graph Caches: Because torch.compile (TorchInductor), Triton kernels, and vLLM CUDA graph capture produce deterministic artifacts for a specific model, tensor-parallel degree, and GPU microarchitecture (e.g., Hopper sm_90 or Blackwell sm_100), teams now execute a warmup pass during CI/CD and package the resulting compilation cache directory (TORCHINDUCTOR_CACHE_DIR and ~/.cache/vllm/torch_compile_cache) directly into an OCI layer or shared read-only NVMe volume, saving 40–120 seconds on every cold boot.
2. Full Process & VRAM Snapshotting (CRIUgpu + cuda-checkpoint): Using CRIU (Checkpoint/Restore In Userspace) 3.19+ paired with NVIDIA’s cuda-checkpoint utility (Driver 550/565+) and the Kubernetes ContainerCheckpoint API, platforms like NVIDIA Dynamo Snapshot, Parasail, and Cedana capture a running inference engine after weights are loaded into VRAM and CUDA graphs are captured. When autoscaling from zero, the runtime restores the frozen process tree and DMA-transfers the serialized GPU memory state directly back into VRAM, bringing vLLM replicas online in under 1 second (3x–20x faster than cold initialization).

5.2 Hardware-Isolated MicroVM Agent Sandboxes (Firecracker & gVisor)

Because autonomous AI agents dynamically synthesize and execute untrusted Python, Bash, and Node.js code — and interact with external websites via headless browsers — running agent tool execution inside standard Docker containers exposes the shared host Linux kernel to container escape vulnerabilities.

• Pre-Packaged Firecracker MicroVMs: Platforms like E2B, AWS AgentCore, Vercel Sandbox, and Fly.io package entire Linux root filesystems, language runtimes, scientific libraries, and headless Chromium browsers into Firecracker microVM templates (Dockerfile → ext4 rootfs + uncompressed vmlinux kernel). Each agent session boots its own dedicated hardware-virtualized KVM guest kernel in ~150 milliseconds.
• Memory-Preserving Pause & Snapshot Forking: Advanced microVM runtimes allow orchestrators to pause an agent’s microVM (pause/resume) while keeping guest RAM and open file descriptors intact in object storage indefinitely while waiting for human-in-the-loop approval. Furthermore, snapshot-based forking allows an orchestrator to take a single warmed microVM state (with datasets already loaded into Pandas/Polars in RAM) and instantly clone up to 20 identical parallel microVMs in milliseconds via Copy-on-Write (CoW) memory pages — enabling parallel tree-of-thought or Monte Carlo code execution from an identical starting state.
• Proxy-Level Secret Injection & Network Gating: Rather than packaging API keys as environment variables inside the sandbox (where prompt-injected code or malicious packages could dump /proc/self/environ), modern sandboxes inject credentials dynamically at the outbound network proxy boundary or hold outbound sockets in escrow until policy verification completes, keeping raw secrets completely outside the microVM memory space.

5.3 Containerized Model Context Protocol (MCP) Catalogs & Gateways

When Anthropic’s Model Context Protocol (MCP) first launched, developers routinely executed MCP tool servers directly on their host machines using npx -y @untrusted/mcp-server or uvx — granting third-party packages unrestricted access to local filesystems, SSH keys, cloud credentials, and local network interfaces.

• The Docker MCP Catalog & Toolkit: MCP servers are now packaged, distributed, and governed as cryptographically signed OCI containers with attached SBOMs and provenance attestations. Through the Docker MCP Catalog and MCP Gateway, each tool server (GitHub, PostgreSQL, Slack, Kubernetes, Brave Search) runs inside an isolated container with strict read-only root filesystems, scoped bind mounts, and explicit network egress policies.
• Dynamic Tool Loading (mcp-find / mcp-add): Pre-loading 50+ MCP tool schemas into an LLM’s system prompt at startup bloats token usage and degrades tool-calling accuracy. Containerized MCP Gateways expose lightweight meta-primitives (mcp-find, mcp-add, and mcp-exec) that allow an agent to search the registry, spin up a containerized MCP server on demand for a single task, and tear it down upon completion.

5.4 eBPF Kernel-Level AI Runtime Security (Tetragon & Connection Escrow)

When deploying autonomous agents with shell and network capabilities, traditional userspace Host Intrusion Detection Systems (HIDS) are fundamentally reactive — they parse asynchronous audit logs after a malicious binary has executed or sensitive data has been exfiltrated.

• Packaged TracingPolicy CRDs: By pre-packaging Cilium Tetragon into the Kubernetes node image, security teams deploy declarative Kubernetes TracingPolicy Custom Resource Definitions (CRDs) that attach eBPF probes directly to Linux kernel functions (kprobes, LSM hooks, and sockops). If a compromised AI agent attempts an unauthorized action — such as reading /etc/shadow, executing a binary outside an approved directory, or opening an outbound socket to an unverified IP — the kernel intercepts the syscall and terminates the process (SIGKILL) or blocks the socket synchronously before packets leave the host.

5.5 Topology-Aware & Cycle-Driven GPU Cluster Schedulers (KAI Scheduler & Kueue)

Standard Kubernetes (kube-scheduler) places pods one by one in an event-driven loop — an architecture that breaks down for distributed AI workloads where a 16-GPU training or inference job is useless unless all 16 pods start simultaneously (gang scheduling) and reside on the same NVLink domain or InfiniBand leaf switch (topology awareness).

• KAI Scheduler (Open-Sourced Run:ai Engine): Donated to the CNCF Sandbox following NVIDIA’s acquisition of Run:ai, KAI Scheduler can be packaged into any Kubernetes cluster to replace pod-by-pod scheduling with cycle-driven snapshot scheduling. It groups related pods into a PodGroup, freezes cluster state at sub-second intervals to simulate optimal gang-scheduling placement in memory, and natively manages fractional GPU sharing (NvFractions), Dynamic Resource Allocation (DRA), and priority-based quota reclamation across departments.
• Kueue & KEDA Admission Pipelines: Packaged alongside Kueue (for multi-tenant job queueing, fair-sharing cohorts, and spot-instance preemption recovery) and KEDA (for scaling vLLM/LangGraph replicas based on Prometheus metrics like KV-cache utilization and token-queue depth rather than CPU utilization), platform teams can ship an entire self-optimizing AI cluster control plane via a single GitOps Helm bundle.

6. Reference Architecture: The Modern AI Packaging Matrix

7. Practical Quickstart Checklist for Platform Teams

To transition an existing AI platform toward this pre-packaged appliance model, adopt the following phased rollout:

1. Standardize Agent Packaging (Week 1): Replace custom FastAPI agent wrappers with declarative langgraph.json manifests compiled via langgraph build, enabling built-in PostgreSQL checkpointing, /mcp exposure, and /a2a endpoints.
2. Decouple Weights from Code (Week 2): Stop baking .safetensors or .gguf files into application Dockerfiles. Package weights separately as OCI artifacts using KitOps / CNCF ModelPack or mount them directly from S3/GCS using Mountpoint for S3 with mmap().
3. Enable FUSE Lazy-Loading on Nodes (Week 3): Configure containerd with the Stargz (eStargz) or AWS SOCI snapshotter on GPU nodes to eliminate multi-minute image pull bottlenecks during autoscaling.
4. Harden Execution & Networking (Week 4): Upgrade cluster CNI to Cilium 1.20 with bpf.datapathMode=auto (netkit), deploy Tetragon TracingPolicy CRDs to restrict agent syscalls/egress at the kernel level, and move untrusted agent code execution into Firecracker microVMs or containerized Docker MCP servers.
5. Eliminate Cold-Start Compile Tax (Week 5): Persist TORCHINDUCTOR_CACHE_DIR and vLLM CUDA graph caches into your deployment artifacts, and evaluate CRIU + cuda-checkpoint for sub-second replica scale-from-zero.