It just occurred to me I'm two months into running my own AI instead of paying for one. I started off with Ollama in Docker with a web UI in front of it, then moved everything to Magnitude on my Mac mini after a few weeks because the wait on long prompts was driving me nuts. At the beginning I had no clue what quantization even meant, and I still don't fully.
I'll admit most of the setup help came from Claude, which is a bit ironic.
There's a lot of models I tried and dropped, these are the ones that stayed. Qwen 3.6 35B for coding questions, a small Gemma for summaries, and a tiny one for titles and commit messages. On the M4 Pro the 35B does about 120 tok/s which still feels unreal to me for something sitting on my desk.
Curious what the next two months bring.
submitted by