As large language model (LLM) inference increasingly processes sensitive information and proprietary model context across personal, enterprise, and regulated settings, data must be processed inside a trusted environment. NVIDIA Confidential Computing (CC) provides a pathway for running these workloads securely using memory-encrypted confidential virtual machines (CVMs), confidential GPUs, and encrypted NVIDIA NVLink . This enables running production AI inference on trusted hardware.

Inference frameworks such as NVIDIA TensorRT LLM deliver best-in-class AI inference by combining framework-level optimizations with NVIDIA accelerated computing. However, when these frameworks run in a CC-enabled environment, secure execution changes assumptions behind memory movement, timing, scheduling, and multi-GPU communication. These changes introduce performance overhead if the runtime does not adapt.…