Stateless vs. Stateful: Deploying Databases with StatefulSets
Kubernetes was originally built for stateless microservices—applications that can be killed, restarted, and cloned at will without losing any important data. But what happens when you need to run a database like PostgreSQL or MongoDB inside your cluster? Databases care deeply about data persistence, network identity, and the exact order in which they are started and stopped.
To solve this, Kubernetes provides the StatefulSet. A StatefulSet guarantees predictable pod naming, stable storage, and ordered deployment, making it the only safe way to run a database natively in Kubernetes.
- The architectural difference between stateless (Deployments) and stateful (StatefulSets) workloads.
- Why you should never use a standard Deployment for a database.
- How StatefulSets maintain stable network identities and attach to dedicated storage volumes.
- How to write and apply a production-ready PostgreSQL StatefulSet manifest.
1. Stateless vs. Stateful Workloads
Before diving into YAML configurations, we must understand how Kubernetes treats data and identity differently based on the resource type you choose.
| Feature | Deployments (Stateless) | StatefulSets (Stateful) |
|---|---|---|
| Pod Identity | Random hashes (e.g., web-7b98d-x4zq) |
Predictable, ordered (e.g., db-0, db-1) |
| Storage (Volumes) | Shared. All replicas usually mount the same volume. | Dedicated. Each replica gets its own unique Persistent Volume (PVC). |
| Deployment Order | Simultaneous. All pods spin up at the same time. | Sequential. Pod 1 waits for Pod 0 to be ready. |
| Best Used For | Web servers, APIs, background workers. | PostgreSQL, Redis, Kafka, Elasticsearch. |
2. The Problem with Deployments for Databases
Many junior engineers try to deploy databases using a standard Kubernetes `Deployment` combined with a Persistent Volume Claim (PVC). This leads to catastrophic failure during scaling or updates.
If you scale a database Deployment to 3 replicas, Kubernetes spins up three identical pods randomly named `db-pod-abc`, `db-pod-xyz`, and `db-pod-123`. Because they are part of a Deployment, they will all try to attach to the exact same Persistent Volume at the same time. Most block storage systems (like AWS EBS) do not allow multiple instances to write to the same disk simultaneously. The database gets corrupted, or the new pods simply fail to start.
Furthermore, if a pod crashes and restarts, it gets a completely new random name and IP address, breaking any master-replica replication logic you had configured in your database.
3. Enter the StatefulSet: PostgreSQL Example
StatefulSets solve these problems by providing two magical features: Stable Network Identity (via a Headless Service) and Volume Claim Templates (which stamp out unique storage disks for every single pod).
Writing the YAML Manifest
Example 1: PostgreSQL StatefulSet and Headless Service
apiVersion: v1
kind: Service
metadata:
name: postgres-svc
spec:
clusterIP: None # This makes it a "Headless" Service
selector:
app: postgres
ports:
- port: 5432
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-db
spec:
serviceName: "postgres-svc" # Must match the Headless Service name
replicas: 3
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:15
env:
- name: POSTGRES_PASSWORD
value: "supersecret"
ports:
- containerPort: 5432
volumeMounts:
- name: data-volume
mountPath: /var/lib/postgresql/data
# The Magic: Creates a unique PVC for each Pod
volumeClaimTemplates:
- metadata:
name: data-volume
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 10Gi
Expected output after applying (kubectl get pods,pvc)
NAME READY STATUS RESTARTS AGE pod/postgres-db-0 1/1 Running 0 2m pod/postgres-db-1 1/1 Running 0 1m pod/postgres-db-2 1/1 Running 0 30s NAME STATUS VOLUME CAPACITY pvc/data-volume-postgres-db-0 Bound pvc-x 10Gi pvc/data-volume-postgres-db-1 Bound pvc-y 10Gi pvc/data-volume-postgres-db-2 Bound pvc-z 10Gi
4. Architecture Visualized
To truly understand StatefulSets, you must visualize how the Headless Service routes traffic and how storage is mapped uniquely.
StatefulSet Execution & Data Flow
A Headless Service routes internal traffic directly to individual Pods. Each Pod strictly attaches to its own designated Persistent Volume Claim.
5. Step-by-Step Execution Breakdown
When you run kubectl apply -f postgres-statefulset.yaml, Kubernetes executes the deployment with extreme caution. It does not rush like a standard Deployment.
6. Hands-On Practice
The best way to understand stateful behavior is to break things deliberately inside a test cluster (like Minikube or Docker Desktop).
Create a Service for your database but ensure it doesn't get a random IP address load-balancer.
Hint: Set clusterIP: None in your Service specification.
Scale down your StatefulSet from 3 replicas to 1 using kubectl scale sts postgres-db --replicas=1.
Expected result: Watch the pods terminate. Kubernetes will gracefully terminate `postgres-db-2` first, then `postgres-db-1`. It always scales down in reverse order.
Write a row of data into `postgres-db-0`. Then, explicitly delete the pod using kubectl delete pod postgres-db-0.
Hint: Wait for Kubernetes to recreate the pod automatically. Query the database again. Your row of data will still be there because the PVC lifecycle is completely independent of the Pod lifecycle.
7. Interview Questions
If you are applying for a DevOps or Senior Backend Engineering role, you will almost certainly be asked about handling data in Kubernetes.
Answer: Deployments are designed for stateless applications. They generate random pod names and typically share a single storage volume. If a database scales, multiple pods will attempt to write to the same disk simultaneously, causing data corruption. Furthermore, replicas need stable network identities to sync with a Master node; random pod hashes break this replication.
Answer: A Headless Service is a Kubernetes Service created with `clusterIP: None`. Instead of acting as a load balancer that routes traffic to random pods, it creates explicit DNS records for every single pod in the StatefulSet (e.g., `postgres-db-0.postgres-svc.default.svc.cluster.local`). This allows database replicas to securely find and communicate with the specific Master pod.
Answer: The PVCs are left intact. Deleting a StatefulSet deletes the pods, but Kubernetes intentionally leaves the underlying storage volumes untouched to prevent accidental data loss. You must manually delete the PVCs if you actually want to destroy the data.
Conclusion
Running databases in Kubernetes was historically considered taboo, but the StatefulSet architecture has matured significantly. By explicitly handling pod identity, network routing (via Headless Services), and dedicated persistent volumes (via VolumeClaimTemplates), you can safely orchestrate stateful applications. Master these concepts, and you transition from simply running containers to architecting enterprise-grade distributed systems.
Official Documentation
- Kubernetes Documentation – StatefulSets
- Kubernetes Tutorials – Run a Replicated Stateful Application
Upgradix by Shrash Studio – Learn Technology. Build Your Future.