Introducing HAMi GPU Descheduler: Automated GPU Defragmentation for Kubernetes
GPUs are the most expensive resource in a Kubernetes cluster, so most teams share them. With HAMi, a single GPU can be split into virtual GPUs and assigned to multiple Pods, each with its own memory and compute isolation.
Sharing, however, introduces a problem that every long-running cluster eventually hits: GPU fragmentation. Pods come and go through autoscaling, rolling updates, and failed jobs, and each leaves a small slice of GPU memory behind. One day a Pod asks for a whole GPU, the cluster reports plenty of free memory, yet the Pod stays Pending. The capacity is there, but it is scattered in pieces too small to use.
This post explains why GPU fragmentation happens, why it is hard to fix by hand, and how HAMi GPU Descheduler automates the fix.