This article provides a step by step guide to serving Gemma 4 E2B and E4B, Google's quantization-aware trained (QAT) small models, on an NVIDIA L4 in an AWS EC2 G6 instance with llama.cpp, and compares the result with the same models on two small local GPUs: an 8 GB RTX 3050 desktop card and a 4 GB GTX 1650 Ti laptop card. Every card runs the same llama.cpp build, the same model files, the same drafter and the same tests. The measures are generation speed for one user and for many, what the drafter adds on each card and why, answer quality, and cost.

https://github.com/xbill9/gemma4-dev

For one user, the 24 GB L4 is 1.2-1.3x an 8 GB desktop card, with the same answers.…