Topologies
Operations
Backup And Restores
Custom Secret
Monitoring
This guide demonstrates how to control pod creation, scaling, and deletion parallelism for MySQL clusters in KubeBlocks using the parallelPodManagementConcurrency
parameter. By defining the maximum number of pods that can be managed in parallel, it allows users to balance operational speed and system stability. Unlike the podManagementPolicy
in StatefulSet, which only provides two fixed options (OrderedReady
or Parallel
), parallelPodManagementConcurrency
offers more flexibility, making it ideal for both resource-sensitive and production environments.
Before proceeding, ensure the following:
kubectl create ns demo
namespace/demo created
Deploy a 2-node semi-sync MySQL cluster (1 primary, 1 secondary) and set the parallelPodManagementConcurrency
parameter to 1 to enforce sequential pod creation.
kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
name: example-mysql-cluster
namespace: demo
spec:
clusterDef: mysql
topology: semisync
terminationPolicy: Delete
componentSpecs:
- name: mysql
serviceVersion: 8.0.35
replicas: 2
parallelPodManagementConcurrency: 1
resources:
limits:
cpu: '0.5'
memory: 0.5Gi
requests:
cpu: '0.5'
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
EOF
Key Parameter:
parallelPodManagementConcurrency
parameter is set to 1, which ensures that the pods are created sequentially rather than in parallel.Run the following command to monitor the creation and initialization of the MySQL pods in the demo namespace:
kubectl get pods -n demo -w
Example Output:
Below is an example of the output when deploying the cluster. Notice how the pods are created sequentially due to the parallelPodManagementConcurrency: 1
setting:
NAME READY STATUS RESTARTS AGE
example-mysql-cluster-mysql-0 0/4 Pending 0 1s
example-mysql-cluster-mysql-0 0/4 Pending 0 3s
example-mysql-cluster-mysql-0 0/4 Init:0/5 0 3s
example-mysql-cluster-mysql-0 0/4 Init:1/5 0 9s
example-mysql-cluster-mysql-0 0/4 Init:2/5 0 10s
example-mysql-cluster-mysql-0 0/4 Init:3/5 0 11s
example-mysql-cluster-mysql-0 0/4 Init:4/5 0 12s
example-mysql-cluster-mysql-0 0/4 PodInitializing 0 13s
example-mysql-cluster-mysql-0 3/4 Running 0 14s
example-mysql-cluster-mysql-0 3/4 Running 0 15s
example-mysql-cluster-mysql-0 3/4 Running 0 18s
example-mysql-cluster-mysql-0 4/4 Running 0 19s
example-mysql-cluster-mysql-1 0/4 Pending 0 0s
example-mysql-cluster-mysql-1 0/4 Pending 0 3s
example-mysql-cluster-mysql-1 0/4 Init:0/5 0 3s
example-mysql-cluster-mysql-1 0/4 Init:1/5 0 7s
example-mysql-cluster-mysql-1 0/4 Init:2/5 0 8s
example-mysql-cluster-mysql-1 0/4 Init:3/5 0 9s
example-mysql-cluster-mysql-1 0/4 Init:4/5 0 10s
example-mysql-cluster-mysql-1 0/4 PodInitializing 0 11s
example-mysql-cluster-mysql-1 3/4 Running 0 12s
example-mysql-cluster-mysql-1 3/4 Running 0 13s
example-mysql-cluster-mysql-0 4/4 Running 0 35s
example-mysql-cluster-mysql-1 3/4 Running 0 17s
example-mysql-cluster-mysql-1 4/4 Running 0 17s
Key Observations:
parallelPodManagementConcurrency
parameter.To scale the cluster to 4 replicas, use the following configuration:
kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
name: example-mysql-cluster
namespace: demo
spec:
clusterDef: mysql
topology: semisync
terminationPolicy: Delete
componentSpecs:
- name: mysql
serviceVersion: 8.0.35
replicas: 4
parallelPodManagementConcurrency: 1
resources:
limits:
cpu: '0.5'
memory: 0.5Gi
requests:
cpu: '0.5'
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
EOF
Example Output:
kubectl get pods -n demo -w
NAME READY STATUS RESTARTS AGE
example-mysql-cluster-mysql-0 4/4 Running 0 32m
example-mysql-cluster-mysql-1 4/4 Running 0 29m
example-mysql-cluster-mysql-2 0/4 Init:0/5 0 4s
example-mysql-cluster-mysql-2 0/4 Init:0/5 0 23s
example-mysql-cluster-mysql-2 0/4 Init:1/5 0 24s
example-mysql-cluster-mysql-2 0/4 Init:1/5 0 32s
example-mysql-cluster-mysql-2 0/4 Init:2/5 0 33s
example-mysql-cluster-mysql-2 0/4 Init:2/5 0 42s
example-mysql-cluster-mysql-2 0/4 Init:3/5 0 43s
example-mysql-cluster-mysql-2 0/4 Init:4/5 0 52s
example-mysql-cluster-mysql-2 0/4 Init:4/5 0 60s
example-mysql-cluster-mysql-2 0/4 PodInitializing 0 61s
example-mysql-cluster-mysql-2 3/4 Running 0 93s
example-mysql-cluster-mysql-2 3/4 Running 0 93s
example-mysql-cluster-mysql-2 3/4 Running 0 100s
example-mysql-cluster-mysql-2 4/4 Running 0 101s
example-mysql-cluster-mysql-3 0/4 Pending 0 0s
example-mysql-cluster-mysql-3 0/4 Pending 0 3s
example-mysql-cluster-mysql-3 0/4 Init:0/5 0 3s
example-mysql-cluster-mysql-3 0/4 Init:1/5 0 12s
example-mysql-cluster-mysql-3 0/4 Init:2/5 0 13s
example-mysql-cluster-mysql-3 0/4 Init:3/5 0 14s
example-mysql-cluster-mysql-3 0/4 Init:4/5 0 15s
example-mysql-cluster-mysql-3 0/4 PodInitializing 0 16s
example-mysql-cluster-mysql-3 3/4 Running 0 17s
example-mysql-cluster-mysql-3 3/4 Running 0 18s
example-mysql-cluster-mysql-3 3/4 Running 0 21s
example-mysql-cluster-mysql-3 4/4 Running 0 21s
Key Observations:
To reduce the cluster back to 2 replicas, apply the following configuration:
kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
name: example-mysql-cluster
namespace: demo
spec:
clusterDef: mysql
topology: server
terminationPolicy: Delete
componentSpecs:
- name: mysql
serviceVersion: 8.0.35
replicas: 2
parallelPodManagementConcurrency: 1
resources:
limits:
cpu: '0.5'
memory: 0.5Gi
requests:
cpu: '0.5'
memory: 0.5Gi
volumeClaimTemplates:
- name: data
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
EOF
Example Output:
kubectl get pods -n demo -w
NAME READY STATUS RESTARTS AGE
example-mysql-cluster-mysql-0 4/4 Running 0 36m
example-mysql-cluster-mysql-1 4/4 Running 0 34m
example-mysql-cluster-mysql-2 4/4 Running 0 48s
example-mysql-cluster-mysql-3 4/4 Running 0 31s
example-mysql-cluster-mysql-3 4/4 Terminating 0 51s
example-mysql-cluster-mysql-3 0/4 Completed 0 53s
example-mysql-cluster-mysql-3 0/4 Completed 0 54s
example-mysql-cluster-mysql-3 0/4 Completed 0 54s
example-mysql-cluster-mysql-2 4/4 Terminating 0 71s
example-mysql-cluster-mysql-2 0/4 Completed 0 73s
example-mysql-cluster-mysql-2 0/4 Completed 0 74s
example-mysql-cluster-mysql-2 0/4 Completed 0 74s
Key Observations:
To remove all created resources, delete the MySQL cluster along with its namespace:
kubectl delete cluster example-mysql-cluster -n demo
kubectl delete ns demo
The parallelPodManagementConcurrency
parameter defines the maximum number of Pods that can be created or deleted in parallel.
It is similar to the podManagementPolicy
of StatefulSet. However, podManagementPolicy
in StatefulSet only provides two options: fully ordered ('OrderedReady') or fully parallel ('Parallel'). In contrast, parallelPodManagementConcurrency
offers more fine-grained control.
This parameter provides fine-grained control over Pod management, allowing you to balance speed and stability based on your application needs. Below are the key benefits:
Avoiding Resource Contention: In resource-limited clusters, creating multiple Pods in parallel can lead to resource competition, causing some Pods to remain in a Pending state. Sequential Pod creation ensures even resource allocation and prevents overwhelming the cluster.
Simplifying Debugging: Sequential creation makes it easier to identify and resolve issues. If a Pod fails to start due to an error, it can be fixed before proceeding, avoiding simultaneous failures and simplifying troubleshooting.
Improving Stability: For scaling or updates, creating Pods one at a time ensures dependencies are respected, reducing the risk of conflicts or instability in stateful applications.
Preventing Service Interruptions: Sequential creation ensures each Pod is fully ready before the next starts, maintaining availability and avoiding downtime, especially for applications with strict readiness requirements.
Through this guide, we demonstrated:
parallelPodManagementConcurrency
for sequential pod creation, scaling, and deletion.Using parallelPodManagementConcurrency
, you can optimize pod management to suit your workload's unique requirements while ensuring high availability and efficient resource usage.