I’ve spent the last few days building Ferrox , a pure-Rust inference engine for running open LLMs locally — dense models and Mixture-of-Experts, on CPU, Apple Metal, or CUDA. No bindings to llama.cpp or ggml, no wrapping an existing runtime. Every kernel, every loader, every scheduling decision written from scratch. The obvious question is “why, when llama.cpp already exists and is excellent.” The honest answer: I wanted to understand inference at a level deeper than “run the binary,” and I wanted a project where every performance claim had to be earned against a real, well-known baseline rather than asserted. What Ferrox actually is At its core, Ferrox loads a GGUF file — the same quantized model format llama.cpp uses — and runs inference on it. Two ways to use it: A CLI , ferrox , with llama.cpp-compatible flags. Point it at a model, get a completion. A server , ferrox-server , that speaks the OpenAI chat-completions API.…