Teams customize their models to hit their targets for latency, speed, memory, and compute. With the open NVIDIA Nemotron family of models, developers can find the right-sized model for their needs.

The new Nemotron 3.5 Lightning NVFP4 checkpoint, for example, preserves accuracy while unlocking up to 4x faster throughput. It’s compressed down to 22 GB from the 66 GB full precision checkpoint by quantizing many of its weights to 4 bits.

To compress models to NVFP4 , post-training quantization (PTQ) is a common method that covers most needs. But if you want to reach high throughput with tighter memory, more aggressive quantization is needed. Quantization-aware distillation (QAD) is an optimal choice in this case. Training Nemotron 3.5 Lightning to adapt to quantization noise with QAD produced an NVFP4 checkpoint that uses less memory and delivers higher throughput while preserving accuracy.

This post demonstrates how QAD improves the Nemotron 3.5 Lightning model using NVIDIA Model Optimizer .…