Kubernetes as AI Infrastructure: Google Cloud, llm-d, and the CNCF

Kubernetes becoming a first-class citizen for LLM serving signals a fundamental shift in how we architect data platforms. What we're seeing is the consolidation of compute orchestration—rather than maintaining separate infrastructure stacks for batch ETL, real-time analytics, and model serving, teams can now treat LLM workloads as just another containerized service. This directly reduces operational complexity, though it introduces new challenges around GPU resource management and inference cost optimization that many teams haven't solved yet. The CNCF's involvement matters because it legitimizes Kubernetes as the standard platform for AI workloads across clouds, making vendor lock-in less likely. For data engineers, this means your existing Kubernetes expertise becomes immediately valuable for LLM operations. My recommendation: invest time understanding GPU scheduling and inference optimization patterns now—teams still treating model serving as separate from data infrastructure are creating unnecessary silos. The convergence is inevitable; getting ahead of it positions you as an architect rather than a firefighter.