--- title: "What Kubernetes actually gives you, and what it does not" date: "2022-10-01T05:00:00.000Z" categories: "Cloud native" canonical: "https://xraph.com/writing/what-kubernetes-actually-gives-you" --- # Abstract A reconciliation loop over declared state is a genuinely good idea. It is not a distributed systems solution, and treating it as one produces confident, broken deployments. ## The idea that works You declare desired state. A controller compares it to observed state and acts to close the gap, continuously, forever. That loop is the whole design, and it comes from a decade of running containers at scale before Kubernetes existed [2] [1]. What it buys is that the system converges after disruption without anybody running a script. A node dies and the pods are rescheduled. A container exits and it is restarted. Nobody wrote that recovery logic for their particular application, and that is real value. ## What people assume it also gives them **Correct application state.** Kubernetes restarts a crashed process. If the crash left a half-written record, the restart does nothing about it. Consistency is the application's problem and no orchestrator changes that [5]. **Meaningful health.** The default probes check that a process responds. They report nothing about whether it can do its job unless you write checks that mean something, and a service that reports healthy while every downstream call fails is a service that will not be routed around. **Zero-downtime deploys.** Rolling updates handle scheduling. They do not make the old version shut down cleanly and they do not make your schema change backward compatible. **Resource efficiency.** Bin packing is only as good as the requests and limits you set, and most requests are set by copying another service's manifest. ## The cost that is rarely counted Kubernetes has a large surface, and the surface has to be learned by whoever operates it at three in the morning. A team of four running a cluster is spending a meaningful fraction of its capacity on the cluster rather than on the product. That is a reasonable trade above a certain scale and a bad one below it. The threshold in my experience is roughly where you have more services than people, or where you genuinely need multi-tenancy or bin packing across heterogeneous workloads. ## Where the model genuinely helps The reconciliation pattern is worth more than the platform. Declared state, observed state, a controller closing the gap, and convergence rather than one-shot commands is a good design for anything that manages instances over time, and I have reused it in contexts with no Kubernetes involved. The cloud native definition is broader than one orchestrator for this reason [3]. The valuable properties are observability, resilience and manageability, and the orchestrator is one way of getting them [4]. # References 1. Brendan Burns, Brian Grant, David Oppenheimer, Eric Brewer, John Wilkes, "Borg, Omega, and Kubernetes", ACM Queue, vol. 14, no. 1, 2016 2. Abhishek Verma et al., "Large-scale Cluster Management at Google with Borg", European Conference on Computer Systems (EuroSys), 2015 3. Cloud Native Computing Foundation, "CNCF Cloud Native Definition v1.0", CNCF, 2018 4. Betsy Beyer, Chris Jones, Jennifer Petoff, Niall Richard Murphy, "Site Reliability Engineering: How Google Runs Production Systems", O'Reilly Media, 2016 5. Seth Gilbert, Nancy Lynch, "Brewer's Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services", ACM SIGACT News, vol. 33, no. 2, pp. 51-59, 2002 --- *Unpublished draft. Not peer reviewed and not submitted to a venue.*