etcd Cluster Lifecycle Management

This guide demonstrates how to manage an etcd cluster's operational state in KubeBlocks, including:

• Stopping the cluster to conserve resources
• Starting a stopped cluster
• Restarting cluster components

Prerequisites

Before proceeding, verify your environment meets these requirements:

• A functional Kubernetes cluster (v1.21+ recommended)
• kubectl v1.21+ installed and configured with cluster access
• Helm installed (installation guide)
• KubeBlocks installed (installation guide)
• etcd Add-on installed and an etcd cluster running (see Quickstart)

Stopping the Cluster

Stopping an etcd cluster in KubeBlocks will:

1. Terminate all running pods
2. Retain persistent storage (PVCs)
3. Maintain cluster configuration

apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: etcd-stop
  namespace: demo
spec:
  clusterName: etcd-cluster
  type: Stop

kubectl apply -f https://raw.githubusercontent.com/apecloud/kubeblocks-addons/refs/heads/main/examples/etcd/stop.yaml

kubectl patch cluster etcd-cluster -n demo --type='json' -p='[
  {
    "op": "add",
    "path": "/spec/componentSpecs/0/stop",
    "value": true
  }
]'

Verify Cluster Stop

kubectl get cluster etcd-cluster -n demo -w

NAME           CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS     AGE
etcd-cluster                        Delete               Stopping   3m
etcd-cluster                        Delete               Stopped    3m42s

Verify no running pods:

kubectl get pods -n demo -l app.kubernetes.io/instance=etcd-cluster

No resources found in demo namespace.

Confirm persistent volumes remain:

kubectl get pvc -n demo -l app.kubernetes.io/instance=etcd-cluster

NAME                       STATUS   VOLUME       CAPACITY   ACCESS MODES   STORAGECLASS    AGE
data-etcd-cluster-etcd-0   Bound    pvc-xxx      20Gi       RWO            kb-default-sc   5m
data-etcd-cluster-etcd-1   Bound    pvc-xxx      20Gi       RWO            kb-default-sc   5m
data-etcd-cluster-etcd-2   Bound    pvc-xxx      20Gi       RWO            kb-default-sc   5m

Starting the Cluster

apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: etcd-start
  namespace: demo
spec:
  clusterName: etcd-cluster
  type: Start

kubectl apply -f https://raw.githubusercontent.com/apecloud/kubeblocks-addons/refs/heads/main/examples/etcd/start.yaml

kubectl patch cluster etcd-cluster -n demo --type='json' -p='[
  {
    "op": "remove",
    "path": "/spec/componentSpecs/0/stop"
  }
]'

Verify Cluster Start

kubectl get cluster etcd-cluster -n demo -w

NAME           CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS     AGE
etcd-cluster                        Delete               Updating   6m
etcd-cluster                        Delete               Running    6m47s

kubectl get pods -n demo -l app.kubernetes.io/instance=etcd-cluster -L kubeblocks.io/role

NAME                  READY   STATUS    RESTARTS   AGE    ROLE
etcd-cluster-etcd-0   2/2     Running   0          55s    follower
etcd-cluster-etcd-1   2/2     Running   0          42s    follower
etcd-cluster-etcd-2   2/2     Running   0          28s    leader

Restarting the Cluster

Restart recreates pods without full cluster stop, useful for:

• Resource refresh
• Troubleshooting

apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: etcd-restart
  namespace: demo
spec:
  clusterName: etcd-cluster
  type: Restart
  restart:
  - componentName: etcd

Apply it:

kubectl apply -f https://raw.githubusercontent.com/apecloud/kubeblocks-addons/refs/heads/main/examples/etcd/restart.yaml

Verify Restart Completion

kubectl get ops etcd-restart -n demo -w

NAME           TYPE      CLUSTER        STATUS    PROGRESS   AGE
etcd-restart   Restart   etcd-cluster   Running   0/3        10s
etcd-restart   Restart   etcd-cluster   Running   1/3        36s
etcd-restart   Restart   etcd-cluster   Running   2/3        68s
etcd-restart   Restart   etcd-cluster   Running   3/3        100s
etcd-restart   Restart   etcd-cluster   Succeed   3/3        2m20s

kubectl get pods -n demo -l app.kubernetes.io/instance=etcd-cluster -L kubeblocks.io/role

NAME                  READY   STATUS    RESTARTS   AGE   ROLE
etcd-cluster-etcd-0   2/2     Running   0          94s   follower
etcd-cluster-etcd-1   2/2     Running   0          62s   follower
etcd-cluster-etcd-2   2/2     Running   0          28s   leader

Cleanup

kubectl delete cluster etcd-cluster -n demo
kubectl delete ns demo