Prerequisite

Check your permission

Fault injection requires local code permission. Make sure your access key has been granted with local code permission.

EKS

Go to IAM and click Users -> User name -> Security credentials -> Create access key and select Local code.

note

After a new access key is created, you need to set aws configure again.

GKE

Verify whether your account has permission to create Podchaos.

kubectl auth can-i create podchaos.chaos-mesh.org -n default --as "useraccont"

If the output is yes, you have the required permission.

If the output is no, follow the instructions below to solve this problem by deleting the verification process.

kubectl delete validatingwebhookconfigurations.admissionregistration.k8s.io chaos-mesh-validation-auth

If the output is reauth related error, it may relate to your GKE account permission. Reset your permission and clear the environment by running the commands below.

rm -rf .config/gcloud

gcloud init

gcloud auth application-default login

export GOOGLE_PROJECT=xxx

kubectl delete secret cloud-key-secret-gcp

Enable Chaos Mesh

Both Helm and kbcli are provided as options to deploy Chaos Mesh.

Here we use ChaosMesh v2.5.2 and the DNS server is enabled for DNS fault injection.

kbcli

For installing ChaosMesh in Containerd, run the command below.

kbcli addon enable fault-chaos-mesh

For installing ChaosMesh in k3d/k3s, run the command below.

kbcli addon enable fault-chaos-mesh --set dnsServer.create=true --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/k3s/containerd/containerd.sock

If you set taints, you can set tolerations following the commands below.

# Chaos-mesh follows the tolerations of kubeblocks by default
# You can specify tolerations for four components of fault-chaos-mesh, controllerManager, chaosDaemon, dashboard, and dnsServer, according to your needs.
# Once a component is specified with tolerations, the default toleration of all components fails. It is recommended to spcify tolerations for four components or none.
kbcli addon enable fault-chaos-mesh \
--tolerations '[{"key":"kb-controller","operator":"Equal","effect":"NoSchedule","value":"true"}]' \
--tolerations 'chaosDaemon:[{"key":"kb-controller","operator":"Equal","effect":"NoSchedule","value":"true"},{"key":"kb-data","operator":"Equal","effect":"NoSchedule","value":"true"}]' \
--tolerations 'dashboard:[{"key":"kb-controller","operator":"Equal","effect":"NoSchedule","value":"true"}]' \
--tolerations 'dnsServer:[{"key":"kb-controller","operator":"Equal","effect":"NoSchedule","value":"true"}]' 

Helm

helm repo add chaos-mesh https://charts.chaos-mesh.org
kubectl create ns chaos-mesh

For installing ChaosMesh in Containerd, run the commands below.

helm install chaos-mesh chaos-mesh/chaos-mesh -n=chaos-mesh --version 2.5.2 --set chaosDaemon.privileged=true --set dnsServer.create=true --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/containerd/containerd.sock

For installing ChaosMesh in k3d/k3s, run the commands below.

helm install chaos-mesh chaos-mesh/chaos-mesh -n=chaos-mesh --version 2.5.2 --set chaosDaemon.privileged=true --set dnsServer.create=true --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/k3s/containerd/containerd.sock