Menu

1 / 3

Ensuring Balanced GPU Allocation in Kubernetes Clusters with Time-Based Fairshare

NVIDIA Technical Blog·Ekin Karabulut·7 months ago
#cTLDq7Bb
Reading 0:00
15s threshold

NVIDIA Run:ai v2.24 introduces time-based fairshare, a new scheduling mode that brings fair-share scheduling with time awareness for over-quota resources to Kubernetes clusters. This capability, built on the open source KAI Scheduler that powers NVIDIA Run:ai , addresses a long-standing challenge in shared GPU infrastructure.

Consider two teams with equal priority sharing a cluster. Team A continuously submits smaller jobs, while Team B needs to run a larger job that requires more resources. Every time resources free up, the smaller jobs from Team A fit immediately and get scheduled. The larger job from Team B continues to wait for enough resources to become available. Before that happens, the next small job from Team A claims the freed capacity. The result: although both teams have identical priority and entitlements, Team A runs job after job while the job from Team B sits in the queue indefinitely.

Time-based fairshare solves this problem by giving the scheduler memory.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.