Redis Cluster Switchover

A switchover is a planned operation that transfers the primary role from one Redis instance to another. Unlike failover which occurs during failures, switchover provides:

• Controlled role transitions
• Minimal downtime (typically a few hundred milliseconds)
• Predictable maintenance windows

Switchover is ideal for:

• Node maintenance/upgrades
• Workload rebalancing
• Testing high availability
• Planned infrastructure changes

Prerequisites

Before proceeding, ensure the following:

• Environment Setup:
  • A Kubernetes cluster is up and running.
  • The kubectl CLI tool is configured to communicate with your cluster.
  • KubeBlocks CLI and KubeBlocks Operator are installed. Follow the installation instructions here.
• Namespace Preparation: To keep resources isolated, create a dedicated namespace for this tutorial:

kubectl create ns demo
namespace/demo created

Deploy a Redis Cluster

KubeBlocks uses a declarative approach for managing Redis Replication Clusters. Below is an example configuration for deploying a Redis Replication Cluster with two components, redis and redis sentinel.

Apply the following YAML configuration to deploy the cluster:

apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: redis-replication
  namespace: demo
spec:
  terminationPolicy: Delete
  clusterDef: redis
  topology: replication
  componentSpecs:
    - name: redis
      serviceVersion: "7.2.4"
      disableExporter: false
      replicas: 2
      resources:
        limits:
          cpu: '0.5'
          memory: 0.5Gi
        requests:
          cpu: '0.5'
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: ""
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
    - name: redis-sentinel
      replicas: 3
      resources:
        limits:
          cpu: '0.5'
          memory: 0.5Gi
        requests:
          cpu: '0.5'
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: ""
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi

Verifying the Deployment

Monitor the cluster status until it transitions to the Running state:

kubectl get cluster redis-replication -n demo -w

Expected Output:

NAME                CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS    AGE
redis-replication   redis                Delete               Running   3m49s

Check the pod status and roles:

kubectl get pods -l app.kubernetes.io/instance=redis-replication -L  kubeblocks.io/role -n demo

Expected Output:

NAME                                 READY   STATUS    RESTARTS   AGE     ROLE
redis-replication-redis-0            3/3     Running   0          3m38s   primary
redis-replication-redis-1            3/3     Running   0          3m16s   secondary
redis-replication-redis-sentinel-0   2/2     Running   0          4m35s
redis-replication-redis-sentinel-1   2/2     Running   0          4m17s
redis-replication-redis-sentinel-2   2/2     Running   0          3m59s

Once the cluster status becomes Running, your Redis cluster is ready for use.

Check Roles

List the Pods and their roles (primary or secondary):

kubectl get pods -n demo -l app.kubernetes.io/instance=redis-replication,apps.kubeblocks.io/component-name=redis -L kubeblocks.io/role

Example Output:

NAME                      READY   STATUS    RESTARTS   AGE     ROLE
redis-replication-redis-0   4/4     Running   0          9m59s   primary
redis-replication-redis-1   4/4     Running   0          11m     secondary

Performing a Planned Switchover

To initiate a planned switchover, create an OpsRequest resource as shown below:

apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: redis-switchover-ops
  namespace: demo
spec:
  clusterName: redis-replication
  type: Switchover
  switchover:
  - componentName: redis
    instanceName: redis-replication-redis-0

apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: redis-switchover-targeted
  namespace: demo
spec:
  clusterName: redis-replication
  type: Switchover
  switchover:
  - componentName: redis
    # Specifies the instance whose role will be transferred.
    # A typical usage is to transfer the leader role in a consensus system.
    instanceName: redis-replication-redis-0
    # If CandidateName is specified, the role will be transferred to this instance.
    # The name must match one of the pods in the component.
    # Refer to ComponentDefinition's Swtichover lifecycle action for more details.
    candidateName: redis-replication-redis-1

Monitoring the Switchover

Monitor the switchover progress:

kubectl get ops redis-switchover-ops -n demo -w

Expected Result:

NAME                   TYPE         CLUSTER             STATUS    PROGRESS   AGE
redis-switchover-ops   Switchover   redis-replication   Succeed   1/1        33s

Verify the Switchover

After the switchover is executed, the specified instance will be promoted to the primary role, while the previously primary instance will take on the secondary role.

kubectl get pods -n demo -l app.kubernetes.io/instance=redis-replication,apps.kubeblocks.io/component-name=redis -L kubeblocks.io/role

Expected Output:

NAME                      READY   STATUS    RESTARTS   AGE     ROLE
redis-replication-redis-0   4/4     Running   0          19m59s  secondary
redis-replication-redis-1   4/4     Running   0          21m     primary

In this example:

• Pod 'redis-replication-redis-1' has been promoted to the primary role.
• Pod 'redis-replication-redis-0' has transitioned to the secondary role.

Troubleshooting

Common Switchover Issues

If the switchover operation gets stuck, check these resources:

# Check agent logs on both current primary and candidate
kubectl logs -n demo <primary-pod> -c kbagent
kubectl logs -n demo <candidate-pod> -c kbagent

# Check cluster events for errors
kubectl get events -n demo --field-selector involvedObject.name=redis-replication

# Check kubeblocks logs
kubectl -n kb-system logs deploy/kubeblocks

Summary

This guide demonstrated how to:

1. Deploy a Redis HA cluster
2. Perform both automatic and targeted Switchover
3. Verify role transitions

Key takeaways:

• Switchover enables controlled maintenance with minimal downtime (~100-500ms)
• KubeBlocks provides declarative operations for reliable role transitions
• Always verify:
  • Cluster status immediately after switchover
  • Application connectivity
  • Replication health
• Check logs for troubleshooting:
  • KubeBlocks operator (kb-system namespace)
  • kbagent on database pods