Scaling & Orchestration

The recurring question “Pulumi or Kubernetes?” is a category error — they stack:

Because each customer is an entire organization — a hospital or medical school with hundreds or thousands of users — operating in one region on their own pinned version, the deployment unit collapses to a single-tenant cluster per customer. This dissolves several hard problems at once:

• Version independence — Org A on 1.4.5 and Org B on 1.4.9 are just different clusters with different Helm release versions. No mixed-version cluster, which is exactly what LiveKit's “homogeneous instances” model wants.
• Data residency — the cluster lives in the customer's region, full stop.
• Isolation — a single-tenant cluster is the strongest tenant boundary there is, ideal for HIPAA/PHI, and one customer's blast radius can't reach another's.
• Trivial routing — no cross-region selector needed; the control plane is a simple org → cluster lookup.

Grounded in LiveKit's own Kubernetes and distributed docs, the media plane has specific requirements:

• Host networking, one pod per node. LiveKit pods need direct network access so the node's rtc.udp/tcp ports are handled by the server — which limits it to one LiveKit pod per node (other workloads may co-reside).
• TLS at the ingress. The official Helm chart terminates signal-connection TLS at the cloud LB (GKE LB, AWS ALB via the Load Balancer Controller, or nginx-ingress + cert-manager). A separate SSL cert is still needed for the embedded TURN/TLS server.
• Redis coordinates the cluster. Nodes report load to Redis; new rooms are placed on the least-loaded node; a room is pinned to a single node.
• Sim & agent workers run as ordinary autoscaled pods alongside the media node pool.

Each customer upgrades on their own schedule by rolling their cluster's Helm release to a new image version. LiveKit's native connection draining makes this safe: on SIGTERM a node keeps its active rooms running, accepts new participants into those rooms, rejects brand-new rooms, and shuts down only once empty. The Helm chart sets terminationGracePeriodSeconds to 5 hours, so a live simulation session is never cut off by an upgrade.

A full cluster per customer is the right unit for high-security customers, but it duplicates a fixed per-cluster baseline (control plane, Redis, ingress, observability, and HA-floor nodes) for every org — which dominates cost for smaller customers. So the same control plane and Pulumi templates offer two tiers (ADR-0027). Crucially, version independence and logical isolation are preserved in both; the difference is physical isolation and cost.

This is the cleanest justification for infrastructure-as-code: adding a new organization is provisioning a new parameterized stack.