Skip to main content

18 posts tagged with "GPU"

View All Tags

HAMi v2.11 Remote GPU in Practice: Run CUDA Anywhere

· 17 min read

HAMi v2.11.0 introduces Remote GPU access, allowing CPU-only machines to use GPUs on other machines. CUDA workloads and GPU resources can now run on separate nodes.

Running CUDA or vLLM on Kubernetes usually means scheduling the Pod onto a GPU node. With Remote GPU, the Pod can run on a separate CPU node and use a GPU Server over the network. The client needs neither a local GPU nor an NVIDIA driver.

HAMi Joins the Vultr Open Source Alliance

· 3 min read
HAMi Community

The HAMi community is pleased to announce that HAMi has joined the newly launched Vultr Open Source Alliance as one of its six founding members, alongside Spinifex, Upbound, Qdrant, Valkey, and NetBird.

The Alliance was announced on November 10, 2026. It brings together open source projects and infrastructure providers to help organizations deploy and operate open source technologies with the performance, scale, and control of production-grade cloud infrastructure.

HAMi v2.11.0 Release: Remote GPU Scheduling, AMD GPU Sharing, Ascend ENPU, NUMA Refit, and Flexible MIG at Scale

· 28 min read
HAMi Community

The HAMi community is proud to announce the official release of HAMi v2.11.0.

Highlights of this release:

  • Remote GPU: run CUDA and vLLM Pods on GPU-less, driver-less client nodes, using GPUs served over the network by Lupine GPU servers.
  • AMD Instinct GPU sharing: AMD GPUs (ROCm) are now first-class in HAMi, with memory limits by size or percentage, compute limits, NUMA- and XGMI-aware placement, scheduling policies and queue priorities, all enforced at the container level.
  • Ascend ENPU mode: policy-based isolation (fixed-share, elastic, best-effort) as a third partitioning mode alongside templates and hami-core, plus a major hami-vnpu-core performance overhaul.
  • NUMA refit: kubelet's Topology Manager and HAMi's scheduler now agree, so CPU affinity and GPU placement land on the same NUMA node.
  • Flexible MIG grows up: CDI injection for dynamic MIG devices, placement scoring with backtracking, and per-Pod profile preferences.
  • A harder scheduler: security-audit-driven hardening of the scheduler extender and webhook, plus Filter-path performance gains.
  • Platform and chart: OpenShift and SELinux support, a restructured devices.VENDOR chart, and node capacity resource reporting.
  • Website overhaul: reorganized information architecture, a redrawn Concepts section, llms.txt, and restructured installation and upgrade guides.

In total, v2.11.0 merges more than 220 pull requests from over 50 contributors, including 29 first-time contributors.

This article walks through the major updates in v2.11.0.

HAMi at ContainerDays Singapore 2026: From GPU Sharing to AI Factory

· 5 min read
HAMi Community

ContainerDays & AI Context Singapore 2026 was held on October 27-28 at Suntec Singapore, bringing together practitioners in cloud native, AI infrastructure, and platform engineering. HAMi Community Manager Jimmy Song and HAMi Contributor Reza Jelveh (both at Dynamia) presented a hands-on workshop and a conference talk, and HAMi appeared in three other sessions at the event.

ContainerDays Singapore 2026

KubeCon China 2026 Recap: Keynotes, a Lightning Talk, and a Case Study Award

· 6 min read
HAMi Community

KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 wrapped up on September 9 at the Shanghai International Convention Center. It was HAMi's first KubeCon China since the project moved to CNCF Incubating, and the community brought two keynotes, a lightning talk, a production session, a booth, and a case study award for China Merchants Bank announced from the keynote stage.

Slides for all four talks are available on the KubeCon China 2026 event page.

Composable GPU Scheduling Policies: How mutex, binpack, spread, and numa Work Together

· 9 min read

HAMi has always offered per-Pod GPU scheduling policies through the hami.io/gpu-scheduler-policy annotation: binpack to pack workloads onto as few cards as possible, spread to distribute them, mutex (new in v2.10.0) to demand an exclusive card. Until now, the annotation accepted exactly one value.

Real clusters rarely want just one behavior at a time. A typical production wish list looks like this: pack inference replicas tightly to leave whole cards free, but keep each Pod's GPUs on the same NUMA node for bandwidth, and give the latency-critical tier cards of its own. That is three policies in a single sentence. Before v2.10.0 you had to pick one and give up the rest.

v2.10.0 closes this gap: hami.io/gpu-scheduler-policy now accepts an ordered, comma-separated list of policies, so filter-style and sort-style policies compose (#2621, @mesutoezdil, closes #2010). This post explains how the combination actually works, and how to adopt and verify it. If you prefer to learn by doing, the companion Lab 14: Composable GPU Scheduling Policies on GKE walks through every scenario below on a real cluster.

HAMi at KubeCon China 2026: Two Keynotes, Two Talks, and a Booth in Shanghai

· 5 min read
HAMi Community

From September 7 to 9, KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 will take place at the Shanghai International Convention Center. This will be HAMi's first KubeCon China appearance since the project was accepted as a CNCF Incubating project in July.

The HAMi community is bringing two keynotes, a lightning talk, a technical session, and a project booth to the show: from PD-disaggregated heterogeneous inference deployment, to dynamic MIG partitioning in five minutes, to a production story of GPU virtualization at thousand-GPU scale. If you are attending, come say hi.

HAMi v2.10.0 Release: Flexible MIG, Composable Scheduling, and an Expanded Accelerator Ecosystem

· 14 min read
HAMi Community

The HAMi community is proud to announce the official release of HAMi v2.10.0. This release advances HAMi on three fronts: more flexible scheduling policies, broader heterogeneous accelerator coverage, and a richer scheduler ecosystem.

v2.10.0 introduces dynamic Flexible MIG, a new mutex scheduling policy, a long-requested NUMA-aware sort fix, composable scheduler policies, gang-scheduling (PodGroup) support, and correct init-container resource accounting. On the device side it adds AMD MI300X and Biren support, heterogeneous Ascend management that lets template-based vNPU and HAMi-core nodes coexist in one cluster, and vNPU HAMi-core monitoring. It also debuts a KAI Scheduler + HAMi-core integration through the new KAI Resource Isolator companion project.

This article walks through the major updates in v2.10.0.

HAMi at KubeCon + CloudNativeCon India 2026: Bringing GPU Sharing to the Community

· 6 min read
HAMi Community

Held on June 18-19, 2026, in Mumbai, India, KubeCon + CloudNativeCon India 2026 brought together cloud native practitioners, platform engineers, AI infrastructure teams, and open source contributors from across the ecosystem. As AI emerged as one of the conference's defining themes, HAMi showcased how Kubernetes-native GPU sharing helps organizations maximize accelerator utilization while maintaining workload isolation and operational flexibility.

From the opening keynote to live booth demonstrations and technical discussions with engineering teams, the event highlighted a growing industry focus: making expensive GPU infrastructure practical for multi-tenant AI workloads.

CNCFHAMi is a CNCF Incubating project