NVIDIA's Nemotron 3.5 Bolt 74B-A9B 24GB is a 74B agent model that reads text and images. Only about 9B of its parameters (the numbers a model learns in training) work on each token, and a token is roughly a piece of a word. Most are stored at about 2 bits each, so the file is 22.0 GB.

We served it on one A40, a 48 GB data-center card on our cloud, with llama.cpp, the open-source program most people use to run models on one machine. The image reader was on, and so was the drafter, a small helper model that speeds up writing. It used about 27 GB. For one user it wrote 136 tokens a second with the drafter and 70 without.

Three choices matter more than the model: what the 24 GB covers, which llama.cpp you compile, and when the drafter pays off. Every figure sits in our Nemotron 3.5 Bolt deployment write-up with the full measurements .

The 24 GB is a text-only budget for one user

NVIDIA's sum comes to 23.83 GB.…