Deploying a Redis Sharding Cluster (Cluster Mode) with KubeBlocks

Redis Cluster distributes data across multiple nodes (shards) using hash-based partitioning, allowing horizontal scaling for both reads and writes.

Use Cases

• Large-scale applications requiring high throughput.
• Distributed caching and session storage.
• Write-heavy workloads (e.g., real-time analytics).

Prerequisites

Before proceeding, ensure the following:

• Environment Setup:
  • A Kubernetes cluster is up and running.
  • The kubectl CLI tool is configured to communicate with your cluster.
  • KubeBlocks CLI and KubeBlocks Operator are installed. Follow the installation instructions here.
• Namespace Preparation: To keep resources isolated, create a dedicated namespace for this tutorial:

kubectl create ns demo
namespace/demo created

Deploying the Redis Sharding Cluster

To create a redis sharding cluster (cluster mode) with 3 shards, and 2 replica for each shard:

apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: redis-sharding
  namespace: demo
spec:
  terminationPolicy: Delete
  shardings:
  - name: shard  # the name of the shard
    shards: 3  # the number of shards to create for the cluster
    template:
      name: redis
      componentDef: redis-cluster-7 # the name of the component definition for each shard
      replicas: 2 # replicas is the number of replicas to create for each shard
      resources:
        limits:
          cpu: '1'
          memory: 1Gi
        requests:
          cpu: '1'
          memory: 1Gi
      serviceVersion: 7.2.4
      volumeClaimTemplates:
      - name: data
        spec:
          accessModes:
          - ReadWriteOnce
          resources:
            requests:
              storage: 20Gi
      services:
      # The service `redis-advertised` is defined in `ComponentDefinition`
      # and it is used to parse the advertised endpoints of the Redis pods.
      - name: redis-advertised # This is a per-pod svc, and will be used to parse advertised endpoints
        podService: true
        #  - NodePort
        #  - LoadBalancer
        serviceType: NodePort

Key Configuration Details:

• shardings: Specifies a list of ShardingSpec objects that configure the sharding topology for components of a Cluster.
• shards: Specifies the number of shards to create for the cluster.
• serviceType: Specifies the service type of redis-advertised service, which is used to parse the advertised endpoints of the Redis pods. By default, the service type is NodePort. If you want to expose the service to the outside of the cluster, you can override the service type to NodePort or LoadBalancer depending on your need.

Verifying the Deployment

Check the Cluster Status

Once the cluster is deployed, check its status:

kubectl get cluster redis-sharding  -n demo -w

Expected Output:

NAME             CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS    AGE
redis-sharding                        Delete               Running   103s

Verify Component and Pod Status

Get all components working for this cluster:

kubectl get cmp -l app.kubernetes.io/instance=redis-sharding -n demo

Expected Output:

NAME                       DEFINITION                      SERVICE-VERSION   STATUS    AGE
redis-sharding-shard-5cd   redis-cluster-7-1.0.0           7.2.4             Running   2m34s
redis-sharding-shard-drg   redis-cluster-7-1.0.0           7.2.4             Running   2m34s
redis-sharding-shard-tgf   redis-cluster-7-1.0.0           7.2.4             Running   2m34s

Each component stands for a shard, with hash id as suffix.

Check pods and their roles

kubectl get pods -l app.kubernetes.io/instance=redis-sharding -L  kubeblocks.io/role -n demo

Expected Output:

NAME                         READY   STATUS    RESTARTS   AGE     ROLE
redis-sharding-shard-5cd-0   2/2     Running   0          3m55s   primary
redis-sharding-shard-5cd-1   2/2     Running   0          3m35s   secondary
redis-sharding-shard-drg-0   2/2     Running   0          3m53s   primary
redis-sharding-shard-drg-1   2/2     Running   0          3m35s   secondary
redis-sharding-shard-tgf-0   2/2     Running   0          3m54s   primary
redis-sharding-shard-tgf-1   2/2     Running   0          3m36s   secondary

There are in-total six replicas in the cluster, two (one primarily and one secondary) for each component.

Scaling Shards

Scaling-Out Shards (Add Shards)

Expected Workflow:

1. A new Component is provisioned with two replicas, one primary and one secondary.
2. Cluster status changes from Updating to Running when all the components are ready (status is Running).

apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: redis-sharding-scale-out-ops
  namespace: demo
spec:
  clusterName: redis-sharding
  type: HorizontalScaling
  horizontalScaling:
  - componentName: shard
    shards: 4

kubectl get ops redis-sharding-scale-out-ops -n demo -w

NAME                           TYPE                CLUSTER          STATUS    PROGRESS   AGE
redis-sharding-scale-out-ops   HorizontalScaling   redis-sharding   Running   0/1          35s
redis-sharding-scale-out-ops   HorizontalScaling   redis-sharding   Succeed   1/1        2m35s

apiVersion: apps.kubeblocks.io/v1
kind: Cluster
spec:
  componentSpecs:
    - name: shard
      shards: 4
# remaining fields are the same as the original cluster CR, omited for brevity
...

kubectl patch cluster redis-sharding -n demo --type=json -p='[{"op": "replace", "path": "/spec/shardings/0/shards", "value": 4}]'

Similar to scaling-out, you can also scale-in the cluster by decreasing the shards field in the Cluster resource. But make sure the shards is greater than or equal to 3.

Switchover

To switchover a shard, named redis-sharding-shard-5cd for example, you can use the following OpsRequest:

kind: OpsRequest
metadata:
  name: redis-sharding-switchover-ops
  namespace: demo
spec:
  clusterName: redis-sharding  # redis-sharding is the name of the cluster
  switchover:
  - componentObjectName: redis-sharding-shard-5cd  # componentObjectName is the name of one of the shards
    candidateName: redis-sharding-shard-5cd-0  # candidateName is the name of the candidate instance
    instanceName: redis-sharding-shard-5cd-1  # instanceName is the name of the primary instance
  type: Switchover

Cleanup

To remove all resources created during this tutorial:

kubectl delete cluster redis-sharding -n demo
kubectl delete ns demo