xbill
Author ProfileClaim This Author Profile
Prove ownership by publishing #HashtagPLUS and this profile link on your author page or an article under your byline. A moderator or admin will review the request before it merges into your real HashtagPLUS username.
🌐 dev.toSource
On a 4 GB GTX 1650 Ti, Gemma 4 E2B does not fit in 16-bit, and Transformers' own 4-bit path runs out of memory on its first forward pass. Kept in Google's QAT 4-bit end to end, with a few Triton kernels and a CUDA graph, plain PyTorch decodes at 88 tok/s, 1.07x llama.cpp on the s
🌐 dev.toSource
Five Colab notebooks take Gemma 4 E2B on one v5e chip from 5 tokens per second under Hugging Face generate to 50 with one compiled decode step, and to the full 131,072-token context with a ring cache, against a JAX engine on the same chip.
🌐 dev.toSource
A full LineageOS 24.0 build outgrows a 32 GB home machine. A 192-core EC2 instance runs the long build, the 266 GB tree comes home at the same path, and the home machine packages the zip next to the phone. Claude Code drives both ends.
🌐 dev.toSource
From Dev.to - rust: Deploying a Rust MCP Server to Amazon Lambda with Gemini CLI