Skip to main content
Version: Preview

Monitor a database

With the built-in database observability, you can observe the database health status and track and measure your database in real-time to optimize database performance. This section shows you how database monitoring tools work with KubeBlocks and how to use the function.

For Playground/test​

KubeBlocks integrates open-source monitoring components, such as Prometheus, AlertManager, and Grafana, by addons and adopts the custom apecloud-otel-collector to collect the monitoring indicators of databases and host machines. All monitoring addons are enabled when KubeBlocks Playground is deployed.

KubeBlock Playground supports the following built-in monitoring addons:

  • prometheus: it includes Prometheus and AlertManager addons.
  • grafana: it includes Grafana monitoring addons.
  • alertmanager-webhook-adaptor: it includes the notification extension addon and is used to extend the notification capability of AlertManager. Currently, the custom bots of Feishu, DingTalk, and Wechat Enterprise are supported.
  • apecloud-otel-collector: it is used to collect the indicators of databases and the host machine.

  1. View all built-in addons and make sure the monitoring addons are enabled. If the monitoring addons are not enabled, enable these addons first.

    # View all addons supported
    kbcli addon list
    ...
    grafana                        Helm   Enabled                   true                                                                                    
    alertmanager-webhook-adaptor   Helm   Enabled                   true                                                                                    
    prometheus                     Helm   Enabled    alertmanager   true 
    ...

  2. Check whether the monitoring function of the cluster is enabled. If the monitoring function is enabled, the output shows disableExporter: false.

    kubectl get cluster mycluster -o yaml
    >
    apiVersion: apps.kubeblocks.io/v1alpha1
    kind: Cluster
    metadata:
    ......
    spec:
      ......
      componentSpecs:
      ......
        disableExporter: false

    If disableExporter: false is not shown in the output, it means the monitoring function of this cluster is not enabled and you need to enable it first.

    kbcli cluster update mycluster --disable-exporter=false

  3. View the dashboard list.

    kbcli dashboard list
    >
    NAME                                 NAMESPACE   PORT    CREATED-TIME
    kubeblocks-grafana                   kb-system   13000   Jul 24,2023 11:38 UTC+0800
    kubeblocks-prometheus-alertmanager   kb-system   19093   Jul 24,2023 11:38 UTC+0800
    kubeblocks-prometheus-server         kb-system   19090   Jul 24,2023 11:38 UTC+0800

  4. Open and view the web console of a monitoring dashboard. For example,

    kbcli dashboard open kubeblocks-grafana

For production environment​

In the production environment, it is highly recommended to build your monitoring system or purchase a third-party monitoring service.

Integrate dashboard and alert rules​

Kubeblocks provides Grafana Dashboards and Prometheus AlertRules for mainstream engines, which you can obtain from the repository, or convert and customize according to your needs.

For the importing method, refer to the tutorials of your third-party monitoring service.

Enable the monitoring function for a database​

Check whether the monitoring function of the cluster is enabled. If the monitoring function is enabled, the output shows monitor: true.

kubectl get cluster mycluster -o yaml
>
apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
......
spec:
   ......
   componentSpecs:
   ......
      disableExporter: false

If disableExporter: false is not shown in the output, it means the monitoring function of this cluster is not enabled and you need to enable it first.

kbcli cluster update mycluster --disable-exporter=false

View the dashboard​

You can view the dashboard of the corresponding cluster via Grafana Web Console. For more detailed information, see the Grafana dashboard documentation.

(Optional) Enable remote write​

Remote write is an optional step and you can enable it based on your actual needs. KubeBlocks provides an addon, victoria-metrics-agent, to push the monitoring data to a third-party monitoring system compatible with the Prometheus Remote Write protocol. Compared with the native Prometheus, vmgent is lighter and supports the horizontal extension.

  1. Enable data push.

    You just need to provide the endpoint which supports the Prometheus Remote Write protocol and multiple endpoints can be supported. Refer to the tutorials of your third-party monitoring system for how to get an endpoint.

    The following examples show how to enable data push by different options.

    # The default option. You only need to provide an endpoint with no verification.
    # Endpoint example: http://localhost:8428/api/v1/write
    kbcli addon enable victoria-metrics-agent --set remoteWriteUrls={http://<remoteWriteUrl>:<port>/<remote write path>}
    # Basic Auth
    kbcli addon enable victoria-metrics-agent --set "extraArgs.remoteWrite\.basicAuth\.username=<your username>,extraArgs.remoteWrite\.basicAuth\.password=<your password>,remoteWriteUrls={http://<remoteWriteUrl>:<port>/<remote write path>}"
    # TLS
    kbcli addon enable victoria-metrics-agent --set "extraArgs.tls=true,extraArgs.tlsCertFile=<path to certifle>,extraArgs.tlsKeyFile=<path to keyfile>,remoteWriteUrls={http://<remoteWriteUrl>:<port>/<remote write path>}"
    # AWS SigV4
    kbcli addon enable victoria-metrics-agent --set "extraArgs.remoteWrite\.aws\.region=<your AMP region>,extraArgs.remoteWrite\.aws\.accessKey=<your accessKey>,extraArgs.remoteWrite\.aws\.secretKey=<your secretKey>,remoteWriteUrls={http://<remoteWriteUrl>:<port>/<remote write path>}"

  2. (Optional) Horizontally scale the victoria-metrics-agent addon.

    When the amount of database instances continues to increase, a single-node vmagent becomes the bottleneck. This problem can be solved by scaling vmagent. The multiple-node vmagent automatically divides the task of data collection according to the Hash strategy.

    kbcli addon enable victoria-metrics-agent --replicas <replica count> --set remoteWriteUrls={http://<remoteWriteUrl>:<port>/<remote write path>}

  3. (Optional) Disable the victoria-metrics-agent addon.

    kbcli addon disable victoria-metrics-agent