Cluster configuration
Following https://www.talos.dev/v1.7
Also following the guide:
https://blog.dalydays.com/post/kubernetes-homelab-series-part-1-talos-linux-proxmox/
Configuration layout
The bulk of the talos configuration is left as default. Only select patches are applied.
The configuration is generated and output to the controlplane.yaml
and worker.yaml
files. They contain the configuration for the controlplane and worker nodes as well as the certificates and keys for the cluster. They cannot be checked into version control.
Patches
For patches we differentiate between:
- those that are applied to all (all controlplane) nodes
- those that are applied to particular nodes
We also differentiate:
- patches that are required at install-time
- patches that can be applied after installation.
Setup
For this setup we want to use a virtual IP (VIP) for the controlplane nodes. Note that this IP is only available if etcd was running to begin with. Meaning it can be used by kubectl but should not be used by talosctl itself.
-
Generate the install media with the addons we want, by visiting https://factory.talos.dev/
-
Choose a virtual controle plane ip. Nodes will auto-negotiate who actually uses that ip.
-
Generate the required files to setup the talos cluster:
talosctl gen secrets talosctl gen config <CLUSTER_NAME> https://<CONTROL_PLANE_IP>:6443 --with-secrets secrets.yaml --output-dir _out --install-image <FACTORY_IMAGE> # in this case: talosctl gen config kluster https://192.168.5.0:6443 --with-secrets secrets.yaml --output-dir _out --install-image factory.talos.dev/metal-installer/235b109dafe508dc2e3329cacf7735c8f9154f433a04da96e1bb4b70b1437b49:v1.11.0
-
This also generates a
talosconfig
at_out
which I don't want to manually reference.EXPORT TALOSCONFIG=_out/talosconfig
-
Install talos. Since my VMs are on different hypervisors their disks have different names. So I apply a per-node patch at install time:
talosctl apply-config --insecure --file _out/controlplane.yaml --nodes <NODE_IP> -p @<PATCH_FILE> # in this case talosctl apply-config --insecure --file _out/controlplane.yaml --nodes 192.168.5.1 -p @patch/controlplane.proxmox.yaml talosctl apply-config --insecure --file _out/controlplane.yaml --nodes 192.168.5.2 -p @patch/controlplane.xoa.yaml talosctl apply-config --insecure --file _out/controlplane.yaml --nodes 192.168.5.3 -p @patch/controlplane.xoa.yaml
-
We are now ready to use all these nodes and not specify
--node
every timetalosctl config endpoint 192.168.5.1 192.168.5.2 192.168.5.3 talosctl config node 192.168.5.1 192.168.5.2 192.168.5.3
-
Now we can apply the late-stage patches (common and individual)
talosctl patch mc --patch @<PATCH_FILE> --node <NODE_IP (OPTIONAL)> # in this case talosctl patch mc --patch @patch/common.yaml # no individual patches
-
Bootstrap the kubernetes cluster on a single node
talosctl bootstrap -n <NODE_IP> # in this case talosctl bootstrap -n 192.168.5.1
-
Get the
kubeconfig
:talosctl kubeconfig --node <NODE_IP> # in this case talosctl kubeconfig -n 192.168.5.1
-
Proceed to apps bootstrap: https://git.kluster.moll.re/remoll/k3s-infra