Skip to main content
Version: Preview

High availability

KubeBlocks integrates the open-source Patroni solution to realize high availability and adopts Noop as the switch policy.

Before you start

  • Install kbcli.

  • Install KubeBlocks: You can install KubeBlocks by kbcli or by Helm.

  • Create a PostgreSQL Replication Cluster.

  • Check the Switch Policy and the role probe.

    • Check whether the switch policy is Noop.

      kubectl get cluster pg-cluster -o yaml
      >
      spec:
      componentSpecs:
      - name: postgresql
      componentDefRef: postgresql
      switchPolicy:
      type: Noop
    • Check whether the following role probe parameters exist to verify the role probe is enabled.

      kubectl get cd postgresql -o yaml
      >
      probes:
      roleProbe:
      failureThreshold: 3
      periodSeconds: 2
      timeoutSeconds: 1

Steps

  1. View the initial status of the PostgreSQL cluster.

    kbcli cluster describe pg-cluster

    PostgreSQL cluster original status

    Currently, pg-cluster-postgresql-0 is the primary pod and pg-cluster-postgresql-1 is the secondary pod.

  2. Simulate a primary pod exception.

    # Enter the primary pod
    kubectl exec -it pg-cluster-postgresql-0 -- bash

    # Delete the data directory of PostgreSQL to simulate an exception
    root@postgres-postgresql-0:/home/postgres# rm -fr /home/postgres/pgdata/pgroot/data
  3. View logs to observe how the roles of pods switch when an exception occurs.

    # View the primary pod logs
    kubectl logs pg-cluster-postgresql-0

    In the logs, the leader lock is released from the primary pod and an HA switch occurs.

    2023-04-18 08:06:52,338 INFO: Lock owner: pg-cluster-postgresql-0; I am pg-cluster-postgresql-0
    2023-04-18 08:06:52,460 INFO: Leader key released
    2023-04-18 08:06:52,552 INFO: released leader key voluntarily as data dir empty and currently leader
    2023-04-18 08:06:52,553 INFO: Lock owner: pg-cluster-postgresql-1; I am pg-cluster-postgresql-0
    2023-04-18 08:06:52,553 INFO: trying to bootstrap from leader 'pg-cluster-postgresql-1'
    # View secondary pod logs
    kubectl logs pg-cluster-postgresql-1

    In the logs, the original secondary pod has obtained the lock and become the leader.

    2023-04-18 08:07:14,441 INFO: no action. I am (pg-cluster-postgresql-1), the leader with the lock
    2023-04-18 08:07:24,354 INFO: no action. I am (pg-cluster-postgresql-1), the leader with the lock
  4. Connect to the PostgreSQL cluster to view the replication information.

    kbcli cluster connect pg-cluster
    postgres=# select * from pg_stat_replication;

    PostgreSQL replication info

    From the output, pg-cluster-postgresql-0 has been assigned as the secondary's pod.

  5. Describe the cluster and check the instance role.

    kbcli cluster describe pg-cluster

    PostgreSQL cluster status after HA

    After the failover, pg-cluster-postgresql-0 becomes the secondary pod and pg-cluster-postgresql-1 becomes the primary pod.