Manage MySQL Service Using the Declarative Cluster API in KubeBlocks

This guide provides step-by-step instructions on how to expose a MySQL service managed by KubeBlocks, either externally or internally. You will learn how to configure external access using a cloud provider's LoadBalancer service, manage internal services, and correctly disable external exposure when no longer needed.

Prerequisites

Before proceeding, ensure the following:

Environment Setup:
- A Kubernetes cluster is up and running.
- The kubectl CLI tool is configured to communicate with your cluster.
- KubeBlocks CLI and KubeBlocks Operator are installed.
Namespace Preparation: To keep resources isolated, create a dedicated namespace for this tutorial:

kubectl create ns demo
namespace/demo created

Deploy a MySQL Semi-Synchronous Cluster

KubeBlocks uses a declarative approach for managing MySQL clusters. Below is an example configuration for deploying a MySQL cluster with 2 nodes (1 primary, 1 replicas) in semi-synchronous mode.

Cluster Configuration

kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: example-mysql-cluster
  namespace: demo
spec:
  clusterDef: mysql
  topology: semisync
  terminationPolicy: Delete
  componentSpecs:
    - name: mysql
      serviceVersion: 8.0.35
      replicas: 2
      resources:
        limits:
          cpu: '0.5'
          memory: 0.5Gi
        requests:
          cpu: '0.5'
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: ""
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
EOF

Verifying the Deployment

Monitor the cluster status until it transitions to the Running state:

kubectl get cluster example-mysql-cluster -n demo -w

Example Output:

NAME                     CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS    AGE
example-mysql-cluster   mysql                Delete               Creating   9s
example-mysql-cluster   mysql                Delete               Running    34s

View Network Services

List the Services created for the MySQL cluster:

kubectl get service -n demo

Example Services:

NAME                           TYPE           CLUSTER-IP       EXTERNAL-IP                                                                          PORT(S)                                                 AGE
example-mysql-cluster-mysql            ClusterIP      172.20.122.164   <none>                                                                               3306/TCP                                                5m16s
example-mysql-cluster-mysql-headless   ClusterIP      None             <none>                                                                               3306/TCP,3601/TCP,9104/TCP,3501/TCP,3502/TCP,9901/TCP   5m16s

Expose MySQL Service Externally or Internally

External addresses allow public internet access to the MySQL service, while internal addresses restrict access to the user’s VPC.

Service Types Comparison

Type	Use Case	Cloud Cost	Security
ClusterIP	Internal service communication	Free	Highest
NodePort	Development/testing	Low	Moderate
LoadBalancer	Production external access	High	Managed via SG

Option 1.: Using OpsRequest

To expose the MySQL service externally using a LoadBalancer, create an OpsRequest resource:

kubectl apply -f - <<EOF
apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: example-mysql-cluster-expose-enable-ops
  namespace: demo
spec:
  type: Expose
  clusterName: example-mysql-cluster
  expose:
  - componentName: mysql
    services:
    - name: internet
      # Determines how the Service is exposed. Defaults to 'ClusterIP'.
      # Valid options are 'ClusterIP', 'NodePort', and 'LoadBalancer'.
      serviceType: LoadBalancer
      # Contains cloud provider related parameters if ServiceType is LoadBalancer.
      # Following is an example for AWS EKS
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-type: nlb
        service.beta.kubernetes.io/aws-load-balancer-internal: "false"  # or "true" for an internal VPC IP
      # Specifies a role to target with the service.
      # If specified, the service will only be exposed to pods with the matching
      # role.
      roleSelector: primary
    switch: Enable
EOF

Wait for the OpsRequest to complete:

kubectl get ops example-mysql-cluster-expose-enable-ops -n demo

Example Output:

NAME                                      TYPE     CLUSTER                 STATUS    PROGRESS   AGE
example-mysql-cluster-expose-enable-ops   Expose   example-mysql-cluster   Succeed   1/1        31s

Option 2.: Using Cluster API

Alternatively, update the spec.services section in the Cluster resource to include a LoadBalancer service:

kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: example-mysql-cluster
  namespace: demo
spec:
  clusterDef: mysql
  topology: semisync
  terminationPolicy: Delete
  # expose a external service
  services:
    - annotations:
        service.beta.kubernetes.io/aws-load-balancer-type: nlb  # Use Network Load Balancer
        service.beta.kubernetes.io/aws-load-balancer-internal: "false"  # or "true" for an internal VPC IP
      componentSelector: mysql
      name: mysql-internet
      serviceName: mysql-internet
      roleSelector: primary
      spec:
        ipFamilyPolicy: PreferDualStack
        ports:
          - name: tcp-mysql
            port: 3306
            protocol: TCP
            targetPort: mysql
        type: LoadBalancer
  componentSpecs:
    - name: mysql
      serviceVersion: 8.0.35
      replicas: 2
      resources:
        limits:
          cpu: '0.5'
          memory: 0.5Gi
        requests:
          cpu: '0.5'
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: ""
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
EOF

In the above YAML configuration, a new external service is added under the services section. The service is of type LoadBalancer and is configured with annotations for AWS Network Load Balancer (NLB).

NOTE

Cloud Provider Annotations

When using a LoadBalancer service, you must include the appropriate annotations specific to your cloud provider. Below is a list of commonly used annotations for different cloud providers:

service.beta.kubernetes.io/aws-load-balancer-type: nlb  # Use Network Load Balancer
service.beta.kubernetes.io/aws-load-balancer-internal: "true"  # Use "false" for internet-facing LoadBalancer

Azure

service.beta.kubernetes.io/azure-load-balancer-internal: "true" # Use "false" for internet-facing LoadBalancer

networking.gke.io/load-balancer-type: "Internal"  # Restricts the LoadBalancer to internal VPC access only. Defaults to internet-facing if not specified.
cloud.google.com/l4-rbs: "enabled" # Optimization for internet-facing LoadBalancer

Alibaba Cloud

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "internet"  # Use "intranet" for internal-facing LoadBalancer

NOTE

The annotation service.beta.kubernetes.io/aws-load-balancer-internal determines whether the LoadBalancer is internal or internet-facing. However, this annotation cannot be dynamically modified after the Service is created.

  service.beta.kubernetes.io/aws-load-balancer-internal: "false"  # Use "true" for internal VPC IPs

If you change this annotation from "false" to "true" after the Service is created, the annotation may update in the Service object, but the LoadBalancer will still retain its public IP.

To properly modify this behavior:

First, delete the existing LoadBalancer service.
Recreate the service with the updated annotation (service.beta.kubernetes.io/aws-load-balancer-internal: "true").
Wait for the new LoadBalancer to be provisioned with the correct internal or external IP.

Wait for the Cluster status to transition to Running using the following command:

kubectl get cluster example-mysql-cluster -n demo -w

NAME                    CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS    AGE
example-mysql-cluster   mysql                Delete               Running   18m

Verify the Exposed Service

Check the service details to confirm the LoadBalancer service is created:

kubectl get services -n demo

Example Output:

NAME                                    TYPE           CLUSTER-IP       EXTERNAL-IP                                                                          PORT(S)                                                 AGE
example-mysql-cluster-mysql            ClusterIP      172.20.129.84    <none>                                                                               3306/TCP                                                4h39m
example-mysql-cluster-mysql-headless   ClusterIP      None             <none>                                                                               3306/TCP,3601/TCP,9104/TCP,3501/TCP,3502/TCP,9901/TCP   4h39m
example-mysql-cluster-mysql-internet        LoadBalancer   172.20.60.24    a1d37733683d244e0bebad8559cbf24d-3728431cad3e24b5.elb.ap-southeast-1.amazonaws.com   3306:30985/TCP                                          13s

Wait for DNS Propagation

The LoadBalancer DNS name may take 2-5 minutes to become resolvable. Verify the resolution status:

nslookup a1d37733683d244e0bebad8559cbf24d-3728431cad3e24b5.elb.ap-southeast-1.amazonaws.com

Example Output:

Server:		192.168.101.1
Address:	192.168.101.1#53

Non-authoritative answer:
Name:	a1d37733683d244e0bebad8559cbf24d-3728431cad3e24b5.elb.ap-southeast-1.amazonaws.com
Address: 54.251.110.4

Connect to MySQL Externally

Retrieve Credentials

KubeBlocks automatically creates a Secret containing the MySQL root credentials. Retrieve the MySQL root credentials:

kubectl get secrets -n demo example-mysql-cluster-mysql-account-root -o jsonpath='{.data.password}' | base64 -d
KI260UK7E9

Connect Using MySQL Client

You can now connect to the MySQL database externally (e.g., from your laptop or EC2):

mysql -h a1d37733683d244e0bebad8559cbf24d-3728431cad3e24b5.elb.ap-southeast-1.amazonaws.com -uroot -pKI260UK7E9

Disable External Exposure

Option 1.: Using OpsRequest

To disable external access, create an OpsRequest:

kubectl apply -f - <<EOF
apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: example-mysql-cluster-expose-disable-ops
  namespace: demo
spec:
  clusterName: example-mysql-cluster
  expose:
  - componentName: mysql
    services:
    - name: internet
      roleSelector: primary
      serviceType: LoadBalancer
    switch: Disable
  preConditionDeadlineSeconds: 0
  type: Expose
EOF

Wait for the OpsRequest to complete:

kubectl get ops example-mysql-cluster-expose-disable-ops -n demo

Example Output:

NAME                                       TYPE     CLUSTER                 STATUS    PROGRESS   AGE
example-mysql-cluster-expose-disable-ops   Expose   example-mysql-cluster   Succeed   1/1        12s

Option 2.: Using Cluster API

Alternatively, remove the spec.services field from the Cluster resource:

kubectl patch cluster example-mysql-cluster -n demo --type=json -p='[
  {
    "op": "remove",
    "path": "/spec/services"
  }
]'

Monitor the cluster status until it is Running:

kubectl get cluster example-mysql-cluster -n demo -w

NAME                    CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS    AGE
example-mysql-cluster   mysql                Delete               Running   23m

Verify Service Removal

Ensure that the 'example-mysql-connect-mysql-internet' Service is removed:

kubectl get service -n demo

Expected Result: The 'example-mysql-cluster-mysql-internet' Service should be removed.

Summary

This guide demonstrated how to:

Expose a MySQL service externally or internally using KubeBlocks.
Configure LoadBalancer services with cloud provider-specific annotations.
Manage external access by enabling or disabling services via OpsRequest or direct updates to the Cluster API.

KubeBlocks provides flexibility and simplicity for managing MySQL services in Kubernetes environments.

Manage MySQL Service Using the Declarative Cluster API in KubeBlocks

Prerequisites

Before proceeding, ensure the following:

Environment Setup:
- A Kubernetes cluster is up and running.
- The kubectl CLI tool is configured to communicate with your cluster.
- KubeBlocks CLI and KubeBlocks Operator are installed.
Namespace Preparation: To keep resources isolated, create a dedicated namespace for this tutorial:

kubectl create ns demo
namespace/demo created

Deploy a MySQL Semi-Synchronous Cluster

KubeBlocks uses a declarative approach for managing MySQL clusters. Below is an example configuration for deploying a MySQL cluster with 2 nodes (1 primary, 1 replicas) in semi-synchronous mode.

Cluster Configuration

kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: example-mysql-cluster
  namespace: demo
spec:
  clusterDef: mysql
  topology: semisync
  terminationPolicy: Delete
  componentSpecs:
    - name: mysql
      serviceVersion: 8.0.35
      replicas: 2
      resources:
        limits:
          cpu: '0.5'
          memory: 0.5Gi
        requests:
          cpu: '0.5'
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: ""
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
EOF

Verifying the Deployment

Monitor the cluster status until it transitions to the Running state:

kubectl get cluster example-mysql-cluster -n demo -w

Example Output:

NAME                     CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS    AGE
example-mysql-cluster   mysql                Delete               Creating   9s
example-mysql-cluster   mysql                Delete               Running    34s

View Network Services

List the Services created for the MySQL cluster:

kubectl get service -n demo

Example Services:

NAME                           TYPE           CLUSTER-IP       EXTERNAL-IP                                                                          PORT(S)                                                 AGE
example-mysql-cluster-mysql            ClusterIP      172.20.122.164   <none>                                                                               3306/TCP                                                5m16s
example-mysql-cluster-mysql-headless   ClusterIP      None             <none>                                                                               3306/TCP,3601/TCP,9104/TCP,3501/TCP,3502/TCP,9901/TCP   5m16s

Expose MySQL Service Externally or Internally

External addresses allow public internet access to the MySQL service, while internal addresses restrict access to the user’s VPC.

Service Types Comparison

Type	Use Case	Cloud Cost	Security
ClusterIP	Internal service communication	Free	Highest
NodePort	Development/testing	Low	Moderate
LoadBalancer	Production external access	High	Managed via SG

Option 1.: Using OpsRequest

To expose the MySQL service externally using a LoadBalancer, create an OpsRequest resource:

kubectl apply -f - <<EOF
apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: example-mysql-cluster-expose-enable-ops
  namespace: demo
spec:
  type: Expose
  clusterName: example-mysql-cluster
  expose:
  - componentName: mysql
    services:
    - name: internet
      # Determines how the Service is exposed. Defaults to 'ClusterIP'.
      # Valid options are 'ClusterIP', 'NodePort', and 'LoadBalancer'.
      serviceType: LoadBalancer
      # Contains cloud provider related parameters if ServiceType is LoadBalancer.
      # Following is an example for AWS EKS
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-type: nlb
        service.beta.kubernetes.io/aws-load-balancer-internal: "false"  # or "true" for an internal VPC IP
      # Specifies a role to target with the service.
      # If specified, the service will only be exposed to pods with the matching
      # role.
      roleSelector: primary
    switch: Enable
EOF

Wait for the OpsRequest to complete:

kubectl get ops example-mysql-cluster-expose-enable-ops -n demo

Example Output:

NAME                                      TYPE     CLUSTER                 STATUS    PROGRESS   AGE
example-mysql-cluster-expose-enable-ops   Expose   example-mysql-cluster   Succeed   1/1        31s

Option 2.: Using Cluster API

Alternatively, update the spec.services section in the Cluster resource to include a LoadBalancer service:

kubectl apply -f - <<EOF
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: example-mysql-cluster
  namespace: demo
spec:
  clusterDef: mysql
  topology: semisync
  terminationPolicy: Delete
  # expose a external service
  services:
    - annotations:
        service.beta.kubernetes.io/aws-load-balancer-type: nlb  # Use Network Load Balancer
        service.beta.kubernetes.io/aws-load-balancer-internal: "false"  # or "true" for an internal VPC IP
      componentSelector: mysql
      name: mysql-internet
      serviceName: mysql-internet
      roleSelector: primary
      spec:
        ipFamilyPolicy: PreferDualStack
        ports:
          - name: tcp-mysql
            port: 3306
            protocol: TCP
            targetPort: mysql
        type: LoadBalancer
  componentSpecs:
    - name: mysql
      serviceVersion: 8.0.35
      replicas: 2
      resources:
        limits:
          cpu: '0.5'
          memory: 0.5Gi
        requests:
          cpu: '0.5'
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: ""
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
EOF

NOTE

Cloud Provider Annotations

When using a LoadBalancer service, you must include the appropriate annotations specific to your cloud provider. Below is a list of commonly used annotations for different cloud providers:

service.beta.kubernetes.io/aws-load-balancer-type: nlb  # Use Network Load Balancer
service.beta.kubernetes.io/aws-load-balancer-internal: "true"  # Use "false" for internet-facing LoadBalancer

Azure

service.beta.kubernetes.io/azure-load-balancer-internal: "true" # Use "false" for internet-facing LoadBalancer

networking.gke.io/load-balancer-type: "Internal"  # Restricts the LoadBalancer to internal VPC access only. Defaults to internet-facing if not specified.
cloud.google.com/l4-rbs: "enabled" # Optimization for internet-facing LoadBalancer

Alibaba Cloud

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "internet"  # Use "intranet" for internal-facing LoadBalancer

NOTE

  service.beta.kubernetes.io/aws-load-balancer-internal: "false"  # Use "true" for internal VPC IPs

If you change this annotation from "false" to "true" after the Service is created, the annotation may update in the Service object, but the LoadBalancer will still retain its public IP.

To properly modify this behavior:

First, delete the existing LoadBalancer service.
Recreate the service with the updated annotation (service.beta.kubernetes.io/aws-load-balancer-internal: "true").
Wait for the new LoadBalancer to be provisioned with the correct internal or external IP.

Wait for the Cluster status to transition to Running using the following command:

kubectl get cluster example-mysql-cluster -n demo -w

NAME                    CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS    AGE
example-mysql-cluster   mysql                Delete               Running   18m

Verify the Exposed Service

Check the service details to confirm the LoadBalancer service is created:

kubectl get services -n demo

Example Output:

NAME                                    TYPE           CLUSTER-IP       EXTERNAL-IP                                                                          PORT(S)                                                 AGE
example-mysql-cluster-mysql            ClusterIP      172.20.129.84    <none>                                                                               3306/TCP                                                4h39m
example-mysql-cluster-mysql-headless   ClusterIP      None             <none>                                                                               3306/TCP,3601/TCP,9104/TCP,3501/TCP,3502/TCP,9901/TCP   4h39m
example-mysql-cluster-mysql-internet        LoadBalancer   172.20.60.24    a1d37733683d244e0bebad8559cbf24d-3728431cad3e24b5.elb.ap-southeast-1.amazonaws.com   3306:30985/TCP                                          13s

Wait for DNS Propagation

The LoadBalancer DNS name may take 2-5 minutes to become resolvable. Verify the resolution status:

nslookup a1d37733683d244e0bebad8559cbf24d-3728431cad3e24b5.elb.ap-southeast-1.amazonaws.com

Example Output:

Server:		192.168.101.1
Address:	192.168.101.1#53

Non-authoritative answer:
Name:	a1d37733683d244e0bebad8559cbf24d-3728431cad3e24b5.elb.ap-southeast-1.amazonaws.com
Address: 54.251.110.4

Connect to MySQL Externally

Retrieve Credentials

KubeBlocks automatically creates a Secret containing the MySQL root credentials. Retrieve the MySQL root credentials:

kubectl get secrets -n demo example-mysql-cluster-mysql-account-root -o jsonpath='{.data.password}' | base64 -d
KI260UK7E9

Connect Using MySQL Client

You can now connect to the MySQL database externally (e.g., from your laptop or EC2):

mysql -h a1d37733683d244e0bebad8559cbf24d-3728431cad3e24b5.elb.ap-southeast-1.amazonaws.com -uroot -pKI260UK7E9

Disable External Exposure

Option 1.: Using OpsRequest

To disable external access, create an OpsRequest:

kubectl apply -f - <<EOF
apiVersion: operations.kubeblocks.io/v1alpha1
kind: OpsRequest
metadata:
  name: example-mysql-cluster-expose-disable-ops
  namespace: demo
spec:
  clusterName: example-mysql-cluster
  expose:
  - componentName: mysql
    services:
    - name: internet
      roleSelector: primary
      serviceType: LoadBalancer
    switch: Disable
  preConditionDeadlineSeconds: 0
  type: Expose
EOF

Wait for the OpsRequest to complete:

kubectl get ops example-mysql-cluster-expose-disable-ops -n demo

Example Output:

NAME                                       TYPE     CLUSTER                 STATUS    PROGRESS   AGE
example-mysql-cluster-expose-disable-ops   Expose   example-mysql-cluster   Succeed   1/1        12s

Option 2.: Using Cluster API

Alternatively, remove the spec.services field from the Cluster resource:

kubectl patch cluster example-mysql-cluster -n demo --type=json -p='[
  {
    "op": "remove",
    "path": "/spec/services"
  }
]'

Monitor the cluster status until it is Running:

kubectl get cluster example-mysql-cluster -n demo -w

NAME                    CLUSTER-DEFINITION   TERMINATION-POLICY   STATUS    AGE
example-mysql-cluster   mysql                Delete               Running   23m

Verify Service Removal

Ensure that the 'example-mysql-connect-mysql-internet' Service is removed:

kubectl get service -n demo

Expected Result: The 'example-mysql-cluster-mysql-internet' Service should be removed.

Summary

This guide demonstrated how to:

Expose a MySQL service externally or internally using KubeBlocks.
Configure LoadBalancer services with cloud provider-specific annotations.
Manage external access by enabling or disabling services via OpsRequest or direct updates to the Cluster API.

KubeBlocks provides flexibility and simplicity for managing MySQL services in Kubernetes environments.