remoll/k8s-bootstrap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

nix ready, cleaner setup

Browse Source
This commit is contained in:
Remy Moll
2025-09-02 23:24:14 +02:00
parent eae471a9d5
commit ed228c1936
7 changed files with 96 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files

110
README.md
View File

@@ -2,54 +2,90 @@
> Following [https://www.talos.dev/v1.7](https://www.talos.dev/v1.7)
Also following the guide:
> https://blog.dalydays.com/post/kubernetes-homelab-series-part-1-talos-linux-proxmox/
## Configuration layout
The main talos configuration is in the `controlplane.yaml` and `worker.yaml` files. They contain the configuration for the controlplane and worker nodes as well as the certificates and keys for the cluster. **They cannot be checked into version control**.
The bulk of the talos configuration is left as default. Only select patches are applied.
Luckily, the boilerplate configuration can be generated with `talosctl gen config ...` and we are not deviating significantly from the defaults. We will only need small patches as described below.
- The main configuration is given by the `controlplane.yaml`, `worker.yaml` and `talosconfig` files. They need to be generated by `talosctl`.
- The patches are in the `patch` directory.
- Patches applied to **all** nodes are in `common.yaml`.
- Patches applied
The configuration is generated and output to the `controlplane.yaml` and `worker.yaml` files. They contain the configuration for the controlplane and worker nodes as well as the certificates and keys for the cluster. **They cannot be checked into version control**.
## High-availability setup
For this setup we want to use a virtual IP (VIP) for the controlplane nodes. This only works once etcd is running on at least one of the controlplane nodes.
### Patches
For patches we differentiate between:
- those that are applied to all (all controlplane) nodes
- those that are applied to particular nodes
- Prerequisites:
```
export TALOSCONFIG="talosconfig"
We also differentiate:
- patches that are required at install-time
- patches that can be applied after installation.
## Setup
> For this setup we want to use a virtual IP (VIP) for the controlplane nodes.
> Note that this IP is only available if etcd was running to begin with. Meaning it can be used by kubectl but should not be used by talosctl itself.
1. Generate the install media with the addons we want, by visiting https://factory.talos.dev/
1. Choose a virtual controle plane ip. Nodes will auto-negotiate who actually uses that ip.
1. Generate the required files to setup the talos cluster:
```bash
talosctl gen secrets
talosctl gen config <CLUSTER_NAME> https://<CONTROL_PLANE_IP>:6443 --with-secrets secrets.yaml --output-dir _out --install-image <FACTORY_IMAGE>
# in this case:
talosctl gen config kluster https://192.168.5.0:6443 --with-secrets secrets.yaml --output-dir _out --install-image factory.talos.dev/metal-installer/235b109dafe508dc2e3329cacf7735c8f9154f433a04da96e1bb4b70b1437b49:v1.11.0
```
- Setup a cluster on one node. I chose a vm node.
```
talosctl gen config kluster https://192.168.1.199:6443
```
```
talosctl machineconfig patch controlplane.yaml --patch @patch/common.yaml --patch @patch/controlplane.vm.yaml > patched.yaml
talosctl apply-config --insecure --nodes 192.168.1.199 --file patched.yaml
talosctl config endpoint 192.168.1.199
talosctl config node 192.168.1.199
talosctl bootstrap
1. This also generates a `talosconfig` at `_out` which I don't want to manually reference.
```bash
EXPORT TALOSCONFIG=_out/talosconfig
```
- Now modify the controlplane configuration to use a VIP, by changing all occurrences of the IP address (...199) to the VIP (...222).
1. Install talos. Since my VMs are on different hypervisors their disks have different names. So I apply a per-node patch at install time:
```bash
talosctl apply-config --insecure --file _out/controlplane.yaml --nodes <NODE_IP> -p @<PATCH_FILE>
- Apply the configuration to the rest of the controlplane nodes.
```
talosctl machineconfig patch controlplane.yaml --patch @patch/common.yaml --patch @patch/controlplane.vm.yaml > patched.yaml
talosctl apply-config --insecure --nodes 192.168.1.201 --file patched.yaml
# For the second node as well
talosctl machineconfig patch controlplane.yaml --patch @patch/common.yaml --patch @patch/controlplane.metal.yaml > patched.yaml
talosctl apply-config --insecure --nodes 192.168.1.108 --file patched.yaml
# in this case
talosctl apply-config --insecure --file _out/controlplane.yaml --nodes 192.168.5.1 -p @patch/controlplane.proxmox.yaml
talosctl apply-config --insecure --file _out/controlplane.yaml --nodes 192.168.5.2 -p @patch/controlplane.xoa.yaml
talosctl apply-config --insecure --file _out/controlplane.yaml --nodes 192.168.5.3 -p @patch/controlplane.xoa.yaml
```
- Also add the new nodes to the talosctl configuration, by manually editing the `endpoints` and `nodes` sections in the `talosconfig` file.
- Finally, get the kubeconfig file.
```
talosctl kubeconfig .
1. We are now ready to use all these nodes and not specify `--node` every time
```bash
talosctl config endpoint 192.168.5.1 192.168.5.2 192.168.5.3
talosctl config node 192.168.5.1 192.168.5.2 192.168.5.3
```
- Proceed to cluster bootstrapping.
1. Now we can apply the late-stage patches (common and individual)
```bash
talosctl patch mc --patch @<PATCH_FILE> --node <NODE_IP (OPTIONAL)>
# in this case
talosctl patch mc --patch @patch/common.yaml
# no individual patches
```
1. Bootstrap the kubernetes cluster **on a single node**
```bash
talosctl bootstrap -n <NODE_IP>
# in this case
talosctl bootstrap -n 192.168.5.1
```
1. Get the `kubeconfig`:
```bash
talosctl kubeconfig --node <NODE_IP>
# in this case
talosctl kubeconfig -n 192.168.5.1
```
1. Proceed to apps bootstrap: https://git.kluster.moll.re/remoll/k3s-infra