Decommission a Specific Pod in KubeBlocks-Managed Redis Clusters

This guide explains how to decommission (take offline) specific Pods in Redis clusters managed by KubeBlocks. Decommissioning provides precise control over cluster resources while maintaining availability. Use this for workload rebalancing, node maintenance, or addressing failures.

Why Decommission Pods with KubeBlocks?

In traditional StatefulSet-based deployments, Kubernetes lacks the ability to decommission specific Pods. StatefulSets ensure the order and identity of Pods, and scaling down always removes the Pod with the highest ordinal number (e.g., scaling down from 3 replicas removes Pod-2 first). This limitation prevents precise control over which Pod to take offline, which can complicate maintenance, workload distribution, or failure handling.

KubeBlocks overcomes this limitation by enabling administrators to decommission specific Pods directly. This fine-grained control ensures high availability and allows better resource management without disrupting the entire cluster.

Prerequisites

Before proceeding, ensure the following:

• Environment Setup:
  • A Kubernetes cluster is up and running.
  • The kubectl CLI tool is configured to communicate with your cluster.
  • KubeBlocks CLI and KubeBlocks Operator are installed. Follow the installation instructions here.
• Namespace Preparation: To keep resources isolated, create a dedicated namespace for this tutorial:

kubectl create ns demo
namespace/demo created

Deploy a Redis Cluster

KubeBlocks uses a declarative approach for managing Redis Replication Clusters. Below is an example configuration for deploying a Redis Replication Cluster with two components, redis and redis sentinel.

Apply the following YAML configuration to deploy the cluster:

apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: redis-replication
  namespace: demo
spec:
  terminationPolicy: Delete
  clusterDef: redis
  topology: replication
  componentSpecs:
    - name: redis
      serviceVersion: "7.2.4"
      disableExporter: false
      replicas: 2
      resources:
        limits:
          cpu: '0.5'
          memory: 0.5Gi
        requests:
          cpu: '0.5'
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: ""
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
    - name: redis-sentinel
      replicas: 3
      resources:
        limits:
          cpu: '0.5'
          memory: 0.5Gi
        requests:
          cpu: '0.5'
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: ""
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi

Verifying the Deployment

Monitor the cluster status until it transitions to the Running state:

kubectl get cluster redis-replication -n demo -w

Expected Output:

NAME                CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS    AGE
redis-replication   redis                Delete               Running   3m49s

Check the pod status and roles:

kubectl get pods -l app.kubernetes.io/instance=redis-replication -L  kubeblocks.io/role -n demo

Expected Output:

NAME                                 READY   STATUS    RESTARTS   AGE     ROLE
redis-replication-redis-0            3/3     Running   0          3m38s   primary
redis-replication-redis-1            3/3     Running   0          3m16s   secondary
redis-replication-redis-sentinel-0   2/2     Running   0          4m35s
redis-replication-redis-sentinel-1   2/2     Running   0          4m17s
redis-replication-redis-sentinel-2   2/2     Running   0          3m59s

Once the cluster status becomes Running, your Redis cluster is ready for use.

Decommission a Pod

Expected Workflow:

1. Replica specified in onlineInstancesToOffline is removed
2. Pod terminates gracefully
3. Cluster transitions from Updating to Running

To decommission a specific Pod (e.g., 'redis-replication-redis-1'), you can use one of the following methods:

apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: redis-replication-decommission-ops
  namespace: demo
spec:
  clusterName: redis-replication
  type: HorizontalScaling
  horizontalScaling:
  - componentName: redis
    scaleIn:
      onlineInstancesToOffline:
        - 'redis-replication-redis-1'  # Specifies the instance names that need to be taken offline

kubectl get ops redis-replication-decommission-ops -n demo -w

NAME                                 TYPE                CLUSTER             STATUS    PROGRESS   AGE
redis-replication-decommission-ops   HorizontalScaling   redis-replication   Succeed   1/1        71s

apiVersion: apps.kubeblocks.io/v1
kind: Cluster
spec:
  componentSpecs:
    - name: redis
      replicas: 1       # explected replicas after decommission
      offlineInstances:
        - redis-replication-redis-1   # <----- Specify Pod to be decommissioned
 ...

Verify the Decommissioning

After applying the updated configuration, verify the remaining Pods in the cluster:

kubectl get pods -n demo -l app.kubernetes.io/instance=redis-replication

Example Output:

NAME                      READY   STATUS    RESTARTS   AGE
redis-replication-redis-0 3/3     Running   0          33m33s

Summary

Key takeaways:

• Traditional StatefulSets lack precise Pod removal control
• KubeBlocks enables targeted Pod decommissioning
• Two implementation methods: OpsRequest or Cluster API

This provides granular cluster management while maintaining availability.