PostgreSQL Cluster Switchover

A switchover is a planned operation that transfers the primary role from one PostgreSQL instance to another. Unlike failover which occurs during failures, switchover provides:

• Controlled role transitions
• Minimal downtime (typically a few hundred milliseconds)
• Predictable maintenance windows

Switchover is ideal for:

• Node maintenance/upgrades
• Workload rebalancing
• Testing high availability
• Planned infrastructure changes

Prerequisites

Before proceeding, ensure the following:

• Environment Setup:
  • A Kubernetes cluster is up and running.
  • The kubectl CLI tool is configured to communicate with your cluster.
  • KubeBlocks CLI and KubeBlocks Operator are installed. Follow the installation instructions here.
• Namespace Preparation: To keep resources isolated, create a dedicated namespace for this tutorial:

kubectl create ns demo
namespace/demo created

Deploy a PostgreSQL Cluster

KubeBlocks uses a declarative approach for managing PostgreSQL clusters. Below is an example configuration for deploying a PostgreSQL cluster with 2 replicas (1 primary, 1 replicas).

Apply the following YAML configuration to deploy the cluster:

apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: pg-cluster
  namespace: demo
spec:
  terminationPolicy: Delete
  clusterDef: postgresql
  topology: replication
  componentSpecs:
    - name: postgresql
      serviceVersion: 16.4.0
      disableExporter: true
      replicas: 2
      resources:
        limits:
          cpu: "0.5"
          memory: "0.5Gi"
        requests:
          cpu: "0.5"
          memory: "0.5Gi"
      volumeClaimTemplates:
        - name: data
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi

Verifying the Deployment

Monitor the cluster status until it transitions to the Running state:

kubectl get cluster pg-cluster -n demo -w

Expected Output:

NAME         CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS     AGE
pg-cluster   postgresql           Delete               Creating   50s
pg-cluster   postgresql           Delete               Running    4m2s

Once the cluster status becomes Running, your PostgreSQL cluster is ready for use.

Check Roles

List the Pods and their roles (primary or secondary):

kubectl get pods -n demo -l app.kubernetes.io/instance=pg-cluster -L kubeblocks.io/role

Example Output:

NAME                      READY   STATUS    RESTARTS   AGE     ROLE
pg-cluster-postgresql-0   4/4     Running   0          9m59s   primary
pg-cluster-postgresql-1   4/4     Running   0          11m     secondary

Performing a Planned Switchover

To initiate a planned switchover, create an OpsRequest resource as shown below:

apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: pg-switchover-ops
  namespace: demo
spec:
  clusterName: pg-cluster
  type: Switchover
  switchover:
  - componentName: postgresql
    instanceName: pg-cluster-postgresql-0

apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: pg-switchover-targeted
  namespace: demo
spec:
  clusterName: pg-cluster
  type: Switchover
  switchover:
  - componentName: postgresql
    # Specifies the instance whose role will be transferred.
    # A typical usage is to transfer the leader role in a consensus system.
    instanceName: pg-cluster-postgresql-0
    # If CandidateName is specified, the role will be transferred to this instance.
    # The name must match one of the pods in the component.
    # Refer to ComponentDefinition's Swtichover lifecycle action for more details.
    candidateName: pg-cluster-postgresql-1

Monitoring the Switchover

Monitor the switchover progress:

kubectl get ops pg-switchover-ops -n demo -w

Expected Result:

NAME                TYPE         CLUSTER      STATUS    PROGRESS   AGE
pg-switchover-ops   Switchover   pg-cluster   Succeed   1/1        17s

Verify the Switchover

After the switchover is executed, the specified instance will be promoted to the primary role, while the previously primary instance will take on the secondary role.

kubectl get pods -n demo -l app.kubernetes.io/instance=pg-cluster -L kubeblocks.io/role

Expected Output:

NAME                      READY   STATUS    RESTARTS   AGE     ROLE
pg-cluster-postgresql-0   4/4     Running   0          19m59s  secondary
pg-cluster-postgresql-1   4/4     Running   0          21m     primary

In this example:

• Pod 'pg-cluster-postgresql-1' has been promoted to the primary role.
• Pod 'pg-cluster-postgresql-0' has transitioned to the secondary role.

Troubleshooting

Common Switchover Issues

If the switchover operation gets stuck, check these resources:

# Check agent logs on both current primary and candidate
kubectl logs -n demo <primary-pod> -c kbagent
kubectl logs -n demo <candidate-pod> -c kbagent

# Check cluster events for errors
kubectl get events -n demo --field-selector involvedObject.name=pg-cluster

# Check kubeblocks logs
kubectl -n kb-system logs deploy/kubeblocks

Summary

This guide demonstrated how to:

1. Deploy a PostgreSQL HA cluster
2. Perform both automatic and targeted Switchover
3. Verify role transitions

Key takeaways:

• Switchover enables controlled maintenance with minimal downtime (~100-500ms)
• KubeBlocks provides declarative operations for reliable role transitions
• Always verify:
  • Cluster status immediately after switchover
  • Application connectivity
  • Replication health
• Check logs for troubleshooting:
  • KubeBlocks operator (kb-system namespace)
  • kbagent on database pods