This article provides a step by step guide to running Gemma 4 E2B, Google's quantization-aware trained (QAT) build, on a 4 GB NVIDIA GTX 1650 Ti laptop GPU with plain PyTorch and Hugging Face Transformers. A small OpenAI-compatible server is built to run the result like any other local model server.

https://github.com/xbill9/gemma4-dev

Can plain PyTorch run Gemma 4 on a 4 GB GPU? Yes, and faster than llama.cpp at chat-length context , once the model stays 4-bit from disk to arithmetic:

GTX 1650 Ti, 4 GB, Gemma 4 E2B Decode Fits the card

16-bit weights — ❌ no, 3.58 GiB of layers alone

Transformers' own 4-bit path — ❌ no, out of memory on the first forward pass

4-bit layers, unpacked by PyTorch one layer at a time 0.58 tok/s ✅ yes

4-bit end to end, this article 87.8 tok/s ✅ yes, with the full 131,072-token context

llama.cpp, same card, same day 82.1 tok/s ✅ yes

Every weight stays on the 4-bit grid Google trained, and the speed comes from doing the 4-bit arithmetic in kernels that read the packed…