Photo by Louis Reed on Unsplash. Source: https://unsplash.com/photos/refill-of-liquid-on-tubes-pwcKF7L4-no (Unsplash License).

NVIDIA now ships the stack underneath an inference model as a versioned recipe. The number that matters is 41.

  • AI Cluster Runtime snapshots a cluster, generates a recipe, renders Helm values and manifests, deploys them, then validates the result before the application reaches the node pool.
  • A fully specialized recipe carries up to 268 configuration values across 16 components, including the GPU Operator, the DRA driver, KAI Scheduler, Grove and NVSentinel.
  • Switching intent from training to inference swaps 5 components and changes 41 values from the same base.
  • Dynamo 1.4 drops NATS as the default, because frontends and workers now discover each other through Kubernetes resources.
  • Two routing paths ship, the Dynamo frontend router and the Gateway API Inference Extension with Dynamo’s endpoint picker, and both need live KV-cache events to route well.

A green validation proves the runtime matches a tested version. It says nothing about your tokens per second. Read the full analysis.

By Ivan Tarin

Ivan Tarin is a Principal Product Marketing Manager at SUSE, where he owns go-to-market strategy and positioning for a seven-product cloud-native portfolio spanning Kubernetes, virtualization, storage, security, and observability. A former full-stack developer who shipped production code for enterprise and public-sector clients including U.S. national laboratories, Ivan translates complex infrastructure and AI technology into messaging that lands with developers, platform teams, and enterprise buyers. He has presented at KubeCon, SUSECON, and AWS Developer Week, and is currently pursuing an MS in Artificial Intelligence at the University of Colorado Boulder.

Leave a Reply

Your email address will not be published. Required fields are marked *

Get the next one before it is old news

Independent analysis of cloud-native infrastructure, Kubernetes and data center economics. No vendor spin.