#Gguf

Beyond Python: Benchmarking Modern SLMs (Gemma 4, Qwen 3 & Phi-4) on CPU with Pure Rust...

ByteShape's full ShapeLearn release of Qwen 3.8 27B: five GGUF models on the measured quality-speed frontier across six GPUs, plus MTP and DFlash2 speculative decoding.

Running Gemma 4 26B-A4B and Qwen3.6 35B-A3B locally with llama.cpp, MTP speculative decoding, multimodal support, and PI as a coding agent.

I've been working on a side project called aether , a Rust LLM inference engine that can load GGUF...

Q4_K_M cuts model size 75% with minimal quality loss — but when should you use Q5, Q6, or Q8 instead? We benchmarked every quant level on real hardware and measured the actual accuracy tradeoffs.

Meet Tian AI — the open-source, completely offline AI assistant for Android that runs entirely on your phone via Termux. No cloud, no data leaks, no subscriptions.

From Dev.to - tutorial: How to Deploy Llama 3.2 7B with GGUF Quantization on a $5/Month DigitalOcean Droplet: Sub-1GB Memory Inference

Meet Tian AI — the open-source, completely offline AI assistant for Android that runs entirely on your phone via Termux. No cloud, no data leaks, no subscriptions.

From Dev.to - android: Meet Tian AI: Your Completely Offline AI Assistant for Android