Monitor a database
With the built-in database observability, you can observe the database health status and track and measure your database in real-time to optimize database performance. This section shows you how database monitoring tools work with KubeBlocks and how to use the function.
For Playground/test
KubeBlocks integrates open-source monitoring components, such as Prometheus, AlertManager, and Grafana, by addons and adopts the custom apecloud-otel-collector
to collect the monitoring indicators of databases and host machines. All monitoring addons are enabled when KubeBlocks Playground is deployed.
KubeBlock Playground supports the following built-in monitoring addons:
prometheus
: it includes Prometheus and AlertManager addons.grafana
: it includes Grafana monitoring addons.alertmanager-webhook-adaptor
: it includes the notification extension addon and is used to extend the notification capability of AlertManager. Currently, the custom bots of Feishu, DingTalk, and Wechat Enterprise are supported.apecloud-otel-collector
: it is used to collect the indicators of databases and the host machine.
View all built-in addons and make sure the monitoring addons are enabled. If the monitoring addons are not enabled, enable these addons first.
# View all addons supported
kbcli addon list
...
grafana Helm Enabled true
alertmanager-webhook-adaptor Helm Enabled true
prometheus Helm Enabled alertmanager true
...Check whether the monitoring function of the cluster is enabled. If the monitoring function is enabled, the output shows
disableExporter: false
.kubectl get cluster mycluster -o yaml
>
apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
......
spec:
......
componentSpecs:
......
disableExporter: falseIf
disableExporter: false
is not shown in the output, it means the monitoring function of this cluster is not enabled and you need to enable it first.kbcli cluster update mycluster --disable-exporter=false
View the dashboard list.
kbcli dashboard list
>
NAME NAMESPACE PORT CREATED-TIME
kubeblocks-grafana kb-system 13000 Jul 24,2023 11:38 UTC+0800
kubeblocks-prometheus-alertmanager kb-system 19093 Jul 24,2023 11:38 UTC+0800
kubeblocks-prometheus-server kb-system 19090 Jul 24,2023 11:38 UTC+0800Open and view the web console of a monitoring dashboard. For example,
kbcli dashboard open kubeblocks-grafana
For production environment
In the production environment, it is highly recommended to build your monitoring system or purchase a third-party monitoring service.
Integrate dashboard and alert rules
Kubeblocks provides Grafana Dashboards and Prometheus AlertRules for mainstream engines, which you can obtain from the repository, or convert and customize according to your needs.
For the importing method, refer to the tutorials of your third-party monitoring service.
Enable the monitoring function for a database
Check whether the monitoring function of the cluster is enabled. If the monitoring function is enabled, the output shows disableExporter: false
.
kubectl get cluster mycluster -o yaml
>
apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
......
spec:
......
componentSpecs:
......
disableExporter: false
If disableExporter: false
is not shown in the output, it means the monitoring function of this cluster is not enabled and you need to enable it first.
kbcli cluster update mycluster --disable-exporter=false
View the dashboard
You can view the dashboard of the corresponding cluster via Grafana Web Console. For more detailed information, see the Grafana dashboard documentation.
(Optional) Enable remote write
Remote write is an optional step and you can enable it based on your actual needs. KubeBlocks provides an addon, victoria-metrics-agent
, to push the monitoring data to a third-party monitoring system compatible with the Prometheus Remote Write protocol. Compared with the native Prometheus, vmgent is lighter and supports the horizontal extension.
Enable data push.
You just need to provide the endpoint which supports the Prometheus Remote Write protocol and multiple endpoints can be supported. Refer to the tutorials of your third-party monitoring system for how to get an endpoint.
The following examples show how to enable data push by different options.
# The default option. You only need to provide an endpoint with no verification.
# Endpoint example: http://localhost:8428/api/v1/write
kbcli addon enable victoria-metrics-agent --set remoteWriteUrls={http://<remoteWriteUrl>:<port>/<remote write path>}# Basic Auth
kbcli addon enable victoria-metrics-agent --set "extraArgs.remoteWrite\.basicAuth\.username=<your username>,extraArgs.remoteWrite\.basicAuth\.password=<your password>,remoteWriteUrls={http://<remoteWriteUrl>:<port>/<remote write path>}"# TLS
kbcli addon enable victoria-metrics-agent --set "extraArgs.tls=true,extraArgs.tlsCertFile=<path to certifle>,extraArgs.tlsKeyFile=<path to keyfile>,remoteWriteUrls={http://<remoteWriteUrl>:<port>/<remote write path>}"# AWS SigV4
kbcli addon enable victoria-metrics-agent --set "extraArgs.remoteWrite\.aws\.region=<your AMP region>,extraArgs.remoteWrite\.aws\.accessKey=<your accessKey>,extraArgs.remoteWrite\.aws\.secretKey=<your secretKey>,remoteWriteUrls={http://<remoteWriteUrl>:<port>/<remote write path>}"(Optional) Horizontally scale the
victoria-metrics-agent
addon.When the amount of database instances continues to increase, a single-node vmagent becomes the bottleneck. This problem can be solved by scaling vmagent. The multiple-node vmagent automatically divides the task of data collection according to the Hash strategy.
kbcli addon enable victoria-metrics-agent --replicas <replica count> --set remoteWriteUrls={http://<remoteWriteUrl>:<port>/<remote write path>}
(Optional) Disable the
victoria-metrics-agent
addon.kbcli addon disable victoria-metrics-agent