For the last few months I've been building Yamis , an AI assistant whose model runs on the user's own computer. No subscription, no prompts and files sitting on someone else's AI server. Here is what I learned along the way – including the parts that were much harder than I expected.
Why run the AI at home?
Three reasons kept coming up:
Cost. Lots of people pay a monthly AI subscription while a perfectly capable PC sits idle next to them.
Privacy. Mails, documents, half-finished ideas – people are rightly uneasy about sending all of that to a company where nobody really knows what happens with it.
The hardware is already there. Many gaming PCs and workstations have enough graphics memory to run a good open model. Even without a big GPU, mixture-of-experts models run surprisingly well on CPU and RAM.
The problem: setting this up is still a job for experts. Which model fits my machine? Which quantization? How do I reach it from my phone without opening my router to the internet?…