Skip to main content
Version: release-0.8

Access logs

The KubeBlocks log enhancement function aims to simplify troubleshooting. With kbcli, the command line tool of KubeBlocks, you can view all kinds of logs generated by the database clusters running on KubeBlocks, such as slow logs, error logs, audit logs, and the container running logs (Stdout and Stderr). For Redis, only the running log is supported.

The KubeBlocks log enhancement function uses methods similar to kubectl exec and kubectl logs to ensure a self-closed loop and lightweight.

Before you start

  • The container image supports tail and xargs commands.
  • Install KubeBlocks: You can install KubeBlocks by kbcli or by Helm.
  • In this guide, we take the MySQL engine as an example, the operation is the same for all database engines.

Steps

  1. Enable the log enhancement function.

    • Enable this function when creating a cluster.

      • If you create a cluster by running the kbcli cluster create command, add the --enable-all-logs=true option to enable the log enhancement function. When this option is true, all the log types defined by spec.componentDefs.logConfigs in ClusterDefinition are enabled automatically.

        kbcli cluster create mysql --enable-all-logs=true mycluster
    • Update this cluster if you did not enable it when creating a cluster.

      kbcli cluster update mycluster --enable-all-logs=true -n <namespace>
      note

      The default namespace in which a cluster is created is default. If you specify a namespace when creating a cluster, fill in <namespace> with your customized one.

  2. View the supported logs.

    Run the kbcli cluster list-logs command to view the enabled log types of the target cluster and the log file details. INSTANCE of each node is displayed.

    Example

    kbcli cluster list-logs mycluster
    >
    INSTANCE LOG-TYPE FILE-PATH SIZE LAST-WRITTEN COMPONENT
    mycluster-mysql-0 error /data/mysql/log/mysqld-error.log 6.4K Feb 06, 2023 09:13 (UTC+00:00) mysql
    mycluster-mysql-0 general /data/mysql/log/mysqld.log 5.9M Feb 06, 2023 09:13 (UTC+00:00) mysql
    mycluster-mysql-0 slow /data/mysql/log/mysqld-slowquery.log 794 Feb 06, 2023 09:13 (UTC+00:00) mysql
  3. Access the cluster log file.

    Run the kbcli cluster logs command to view the details of the target log file generated by the target instance on the target cluster. You can use different options to view the log file details you need. You can also run kbcli cluster logs -h to see the examples and option descriptions.

    kbcli cluster logs -h
    Output
    Access cluster log file

    Examples:
    # Return snapshot logs from cluster mycluster with default primary instance (stdout)
    kbcli cluster logs mycluster

    # Display only the most recent 20 lines from cluster mycluster with default primary instance (stdout)
    kbcli cluster logs --tail=20 mycluster

    # Return snapshot logs from cluster mycluster with specify instance my-instance-0 (stdout)
    kbcli cluster logs mycluster --instance my-instance-0

    # Return snapshot logs from cluster mycluster with specify instance my-instance-0 and specify container
    # my-container (stdout)
    kbcli cluster logs mycluster --instance my-instance-0 -c my-container

    # Return slow logs from cluster mycluster with default primary instance
    kbcli cluster logs mycluster --file-type=slow

    # Begin streaming the slow logs from cluster mycluster with default primary instance
    kbcli cluster logs -f mycluster --file-type=slow

    # Return the specify file logs from cluster mycluster with specify instance my-instance-0
    kbcli cluster logs mycluster --instance my-instance-0 --file-path=/var/log/yum.log

    # Return the specify file logs from cluster mycluster with specify instance my-instance-0 and specify
    # container my-container
    kbcli cluster logs mycluster --instance my-instance-0 -c my-container --file-path=/var/log/yum.log
  4. (Optional) Troubleshooting.

    The log enhancement function does not affect the core process of KubeBlocks. If a configuration exception occurs, a warning shows to help troubleshoot. warning is recorded in the event and status.Conditions of the target database cluster.

    View warning information.

    • Run kbcli cluster describe <cluster-name> to view the status of the target cluster. You can also run kbcli cluster list events <cluster-name> to view the event information of the target cluster directly.

      kbcli cluster describe mycluster
      kbcli cluster list-events mycluster
    • Run kubectl describe cluster <cluster-name> to view the warning.

      kubectl describe cluster mycluster

      Example

      Status:
      Cluster Def Generation: 3
      Components:
      Replicasets:
      Phase: Running
      Conditions:
      Last Transition Time: 2022-11-11T03:57:42Z
      Message: EnableLogs of cluster component replicasets has invalid value [errora slowa] which isn't defined in cluster definition component replicasets
      Reason: EnableLogsListValidateFail
      Status: False
      Type: ValidateEnabledLogs
      Observed Generation: 2
      Operations:
      Horizontal Scalable:
      Name: replicasets
      Restartable:
      replicasets
      Vertical Scalable:
      replicasets
      Phase: Running
      Events:
      Type Reason Age From Message
      ---- ------ ---- ---- -------
      Normal Creating 49s cluster-controller Start Creating in Cluster: release-name-error
      Warning EnableLogsListValidateFail 49s cluster-controller EnableLogs of cluster component replicasets has invalid value [errora slowa] which isn't defined in cluster definition component replicasets
      Normal Running 36s cluster-controller Cluster: release-name-error is ready, current phase is Running