Menu

Two months in, still extremely happy with self-hosting my own AI
📰

Two months in, still extremely happy with self-hosting my own AI

Reading 0:00
15s threshold
Two months in, still extremely happy with self-hosting my own AI

It just occurred to me I'm two months into running my own AI instead of paying for one. I started off with Ollama in Docker with a web UI in front of it, then moved everything to Magnitude on my Mac mini after a few weeks because the wait on long prompts was driving me nuts. At the beginning I had no clue what quantization even meant, and I still don't fully.

I'll admit most of the setup help came from Claude, which is a bit ironic.

There's a lot of models I tried and dropped, these are the ones that stayed. Qwen 3.6 35B for coding questions, a small Gemma for summaries, and a tiny one for titles and commit messages. On the M4 Pro the 35B does about 120 tok/s which still feels unreal to me for something sitting on my desk.

Curious what the next two months bring.

submitted by /u/gato5fication
[link] [comments]