Contents

• Docker Installation
• CRI-O Installation

This is a detailed guide on how to setup high availability etcd cluster on raspberry pi using kubernetes.

1. Flash raspbianos on usb drive that you are going to use

Download Raspbian OS from official sources

Flash it using the command

# Replace <X> with the drive letter of your storage media, please refer to my previous guide for more info
# Replace <PATH TO RASPBIAN> with your system path where your image is located at
xzcat <PATH TO RASPBIAN.img.xz> | sudo dd of=/dev/sd<X> bs=64k oflag=dsync status=progress

This will be completed at around 2.1GB size, time is completely dependent upon the speeds of your storage media. See here for reference

2. change the host name to etcd-x where x is the number you are going to use

    echo etcd-<X> > /etc/hostname

3. change the locale and keyboard to en_US

    echo LANG=en_US.UTF-8 > /etc/locale.conf
    echo en_US.UTF-8 UTF-8 >> /etc/locale.gen
    locale-gen
    localectl set-locale LANG=en_US.UTF-8
    echo LC_ALL= >> /etc/locale.conf

5. Turn Swap off permanently

On Raspbian, we can turn off swap by doing

    dphys-swapfile swapoff
    systemctl disable dphys-swapfile

6. Install essentials

    apt install -y apt-transport-https curl jq tar bridge-utils iptables-persistent

4. choose your container runtime interface

Again we have two options, one is to use

Docker Installation

    curl -fsSL https://get.docker.com -o get-docker.sh
    sh get-docker.sh

CRI-O Installation

As the dockershim CRI is deprecated for Kubernetes 1.24 onwards, we are going to choose a better and more lightweight option for our usecase

curl https://raw.githubusercontent.com/cri-o/cri-o/main/scripts/get | bash -s -- -a arm64 -t v1.23.2

This script will automatically setup the CRI-O container runtime engine for our Raspbian OS.

Enable the runtime from systemd by command

    systemctl enable crio.service
    systemctl start crio.service

You can check the status of runtime by using crictl info and the version by crictl version

9. add the etcd system config file at /etc/systemd/system/kubelet.service.d/20-etcd-service-manager.conf

vi /etc/systemd/system/kubelet.service.d/20-etcd-service-manager.conf

[Service]
ExecStart=
# If you are using CRI-O as container runtime interface, add the following to your config.yaml also
#ExecStart=/usr/bin/kubelet --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock
ExecStart=/usr/bin/kubelet --address=127.0.0.1 --pod-manifest-path=/etc/kubernetes/manifests --cgroup-driver=systemd --container-runtime=remote --container-runtime-endpoint=unix:///var/run/crio/crio.sock --runtime-request-timeout=5m
Restart=always

8. add the /var/lib/kubelet/config.yaml file with proper configuration

vi /var/lib/kubelet/config.yaml

kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
address: 127.0.0.1
staticPodPath: /etc/kubernetes/manifests
cgroupDriver: systemd
runtimeRequestTimeout: 10m

systemctl daemon-reload
systemctl enable kubelet

9. install kubernetes
10. restore etcd data directory

    rsync -av /mnt/var/lib/etcd/members /var/lib/etcd/

11. edit the kubeadm.config file with initial-cluster-state=existing

12. manage your etcd cluster

More commands to manage your etcd cluster can be found at https://etcd.io/docs/v3.5/op-guide

crictl exec $(crictl ps | awk 'FNR==2{print $1;exit}') etcdctl --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key --cacert=/etc/kubernetes/pki/etcd/ca.crt --endpoints=https://192.168.131.23:2379 endpoint status -w table --cluster