KubeBlocks uses a declarative approach for managing Kafka Clusters. Below is an example configuration for deploying a Kafka Cluster with 3 components
Apply the following YAML configuration to deploy the cluster:
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
name: kafka-separated-cluster
namespace: demo
spec:
terminationPolicy: Delete
clusterDef: kafka
topology: separated_monitor
componentSpecs:
- name: kafka-broker
serviceVersion: 3.3.2
services:
- name: advertised-listener
serviceType: ClusterIP # Valid options are: [ClusterIP, NodePort, LoadBalancer]
podService: true
replicas: 1
resources:
limits:
cpu: "0.5"
memory: "0.5Gi"
requests:
cpu: "0.5"
memory: "0.5Gi"
env:
- name: KB_KAFKA_BROKER_HEAP
value: "-XshowSettings:vm -XX:MaxRAMPercentage=100 -Ddepth=64"
- name: KB_KAFKA_CONTROLLER_HEAP
value: "-XshowSettings:vm -XX:MaxRAMPercentage=100 -Ddepth=64"
# Whether to enable direct Pod IP address access mode.
# - If set to 'true', Kafka clients will connect to Brokers using the Pod IP address directly.
# - If set to 'false', Kafka clients will connect to Brokers using the Headless Service's FQDN, and service `advertised-listener` must be set with "podService: true".
- name: KB_BROKER_DIRECT_POD_ACCESS
value: "false"
volumeClaimTemplates:
- name: data
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
- name: metadata
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
- name: kafka-controller
serviceVersion: 3.3.2
replicas: 1
resources:
limits:
cpu: "0.5"
memory: "0.5Gi"
requests:
cpu: "0.5"
memory: "0.5Gi"
volumeClaimTemplates:
- name: metadata
spec:
storageClassName: ""
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
- name: kafka-exporter
serviceVersion: 1.6.0
replicas: 1
resources:
limits:
cpu: "0.5"
memory: "1Gi"
requests:
cpu: "0.1"
memory: "0.2Gi"
These three components will be created strictly in controller->broker->exporter
order as defined in ClusterDefinition
.