Topologies
Operations
Backup And Restores
Custom Secret
Monitoring
This guide explains how to decommission (take offline) a specific Pod in a MySQL cluster managed by KubeBlocks. Decommissioning a Pod allows precise control over cluster resources without disrupting the cluster's overall functionality. This is particularly useful for workload rebalancing, node maintenance, or addressing specific failures.
In traditional StatefulSet-based deployments, Kubernetes lacks the ability to decommission specific Pods. StatefulSets ensure the order and identity of Pods, and scaling down always removes the Pod with the highest ordinal number (e.g., scaling down from 3 replicas removes Pod-2
first). This limitation prevents precise control over which Pod to take offline, which can complicate maintenance, workload distribution, or failure handling.
KubeBlocks overcomes this limitation by enabling administrators to decommission specific Pods directly. This fine-grained control ensures high availability and allows better resource management without disrupting the entire cluster.
Before proceeding, ensure the following:
kubectl create ns demo
namespace/demo created
Deploy a 3-node MySQL semi-synchronous cluster (1 primary, 2 replicas):
kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
name: example-mysql-cluster
namespace: demo
spec:
clusterDef: mysql
topology: semisync
terminationPolicy: Delete
componentSpecs:
- name: mysql
serviceVersion: 8.0.35
replicas: 3
resources:
limits:
cpu: '0.5'
memory: 0.5Gi
requests:
cpu: '0.5'
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
EOF
Monitor the cluster status until it transitions to the Running state:
kubectl get cluster example-mysql-cluster -n demo -w
Example Output:
NAME CLUSTER-DEFINITION TERMINATION-POLICY STATUS AGE
example-mysql-cluster mysql Delete Creating 8s
example-mysql-cluster mysql Delete Running 2m41s
Once the cluster status becomes Running, your MySQL cluster is ready for use.
List the Pods in the cluster to verify all three Pods are running:
kubectl get pods -n demo -l app.kubernetes.io/instance=example-mysql-cluster
Example Output (3 Pods):
NAME READY STATUS RESTARTS AGE
example-mysql-cluster-mysql-0 4/4 Running 0 4m30s
example-mysql-cluster-mysql-1 4/4 Running 0 4m30s
example-mysql-cluster-mysql-2 4/4 Running 0 49s
To decommission a specific Pod (e.g., 'example-mysql-cluster-mysql-1'), you can use one of the following methods:
Create an OpsRequest to mark the Pod as offline:
kubectl apply -f - <<EOF
apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
name: example-mysql-cluster-decommission-ops
namespace: demo
spec:
clusterName: example-mysql-cluster
type: HorizontalScaling
horizontalScaling:
- componentName: mysql
scaleIn:
onlineInstancesToOffline:
- 'example-mysql-cluster-mysql-1' # Specifies the instance names that need to be taken offline
EOF
Check the progress of the decommissioning operation:
kubectl describe ops example-mysql-cluster-decommission-ops -n demo
Example Output:
Status:
Phase: Succeed
Progress: 1/1
...
Alternatively, update the Cluster resource directly to decommission the Pod:
kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
name: example-mysql-cluster
namespace: demo
spec:
clusterDef: mysql
topology: semisync
terminationPolicy: Delete
componentSpecs:
- name: mysql
serviceVersion: 8.0.35
replicas: 2 # <----- Reduce replicas from 3 to 2
offlineInstances:
- example-mysql-cluster-mysql-1 # <----- Specify Pod to be decommissioned
resources:
limits:
cpu: '0.5'
memory: 0.5Gi
requests:
cpu: '0.5'
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
EOF
After applying the updated configuration, verify the remaining Pods in the cluster:
kubectl get pods -n demo -l app.kubernetes.io/instance=example-mysql-cluster
Example Output:
NAME READY STATUS RESTARTS AGE
example-mysql-cluster-mysql-0 4/4 Running 0 6m38s
example-mysql-cluster-mysql-2 4/4 Running 0 6m38s
In this guide, you learned:
By leveraging KubeBlocks, you can manage MySQL clusters with fine-grained control, ensuring high availability and flexibility for maintenance and workload distribution.