Menu

How do you keep your local model collection organized across machines?

Reading 0:00
15s threshold

Between GGUFs, safetensors, LoRAs and a few quants of the same model, my homelab model folder is a junk drawer. Each machine has its own Hugging Face cache, and I can't easily tell which box has which revision.

What's working for you? Shared HF cache on a NAS, a mirror proxy, rsync scripts, something else? One thing I've been doing: since I already run a local container registry, I pack each model with KitOps (kit pack with a small Kitfile listing the weights and which quant it is), push it to the registry, and kit unpack on whichever box needs it. The tag tells me exactly which quant I tested.

It works, but it's probably overkill for some setups, so I'm curious what the simpler options look like.

submitted by /u/iamjessew
[link] [comments]