Disclosure: I'm the maintainer of Kamaji, and I lent Anish the H100s and the bare-metal infrastructure. He wrote and ran everything himself, so treat me as biased and verify the numbers.
Anish Bista wrote up a lab combining Kamaji (a control plane per customer, for isolation) with Run:ai/KAI projects (per team, for GPU sharing).
We decided to work together on this topic because it's a question that comes up a lot when prospects aim to sell GPUs to more than one customer: separate clusters solve " can they see my workloads?" but not " my fraud team's GPUs sit idle all night while my NLP team queues ".
Anish ran it on two bare-metal servers with one H100 80GB each (the journey to find them could be worth another interesting blog post) and tested borrowing and reclaiming between teams, fractional GPU memory limits, scheduling rules, and whether a busy tenant can use another tenant's idle card (it can't).
Some findings are less flattering than the title, which is why I like it: the idle-GPU timeout only…