6f717a602f
Configures `myks` for Helm chart rendering with `ytt` overlays to manage cluster applications. Defines prototypes and environment-specific configurations for core applications including ArgoCD, Traefik, Cert-Manager, and Forgejo. Adds comprehensive documentation covering cluster setup, GitOps structure, and development environment. Integrates `direnv` for environment variable management, `gitignore` for file exclusion, and `sops` for secret encryption. Includes rendered Kubernetes manifests and ArgoCD application resources for initial deployment.
123 lines
3.1 KiB
Markdown
123 lines
3.1 KiB
Markdown
# k8s-and-chill
|
|
|
|
Private Kubernetes cluster running on 3x Hetzner CAX11 (ARM64) instances with Talos Linux, managed by myks.
|
|
|
|
## Cluster Setup
|
|
|
|
### Prerequisites
|
|
|
|
Enter the dev shell (via direnv or `nix develop`), which provides:
|
|
|
|
- `talosctl`
|
|
- `kubectl`
|
|
- `helm`
|
|
- `myks`
|
|
- `hcloud`
|
|
|
|
### Infrastructure
|
|
|
|
| Node | Public IP | Private IP | Location |
|
|
|------|-----------|------------|----------|
|
|
| ubuntu-4gb-nbg1-1 | 195.201.219.17 | 10.0.0.3 | nbg1 |
|
|
| ubuntu-4gb-nbg1-2 | 195.201.140.75 | 10.0.0.4 | nbg1 |
|
|
| ubuntu-4gb-nbg1-3 | 195.201.219.111 | 10.0.0.2 | nbg1 |
|
|
|
|
All nodes are control plane nodes (3-node HA etcd). The Kubernetes API endpoint is `https://195.201.219.111:6443`.
|
|
|
|
The nodes are connected via a Hetzner private network (`thalos-k8s`), which is used for inter-node communication.
|
|
|
|
### Installing Talos on Hetzner Cloud
|
|
|
|
The servers were originally provisioned with Ubuntu. Talos was installed by writing the Talos disk image via Hetzner rescue mode.
|
|
|
|
#### 1. Get the Talos image URL
|
|
|
|
Talos images for Hetzner Cloud are generated via the [Talos Image Factory](https://factory.talos.dev). For vanilla Talos (no extensions), get the schematic ID:
|
|
|
|
```sh
|
|
curl -sX POST https://factory.talos.dev/schematics \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"customization":{"systemExtensions":{"officialExtensions":[]}}}'
|
|
# Returns: {"id":"376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba"}
|
|
```
|
|
|
|
The image URL follows this pattern:
|
|
|
|
```
|
|
https://factory.talos.dev/image/<schematic-id>/<talos-version>/hcloud-arm64.raw.xz
|
|
```
|
|
|
|
#### 2. Enable rescue mode and reboot
|
|
|
|
For each server:
|
|
|
|
```sh
|
|
hcloud server enable-rescue <server-name> --ssh-key "<ssh-key-name>"
|
|
hcloud server reboot <server-name>
|
|
```
|
|
|
|
#### 3. Write Talos to disk
|
|
|
|
SSH into each server's rescue system and write the image:
|
|
|
|
```sh
|
|
ssh root@<server-ip> "curl -fsSL '<image-url>' | xz -d | dd of=/dev/sda bs=4M status=progress && sync"
|
|
```
|
|
|
|
#### 4. Reboot into Talos
|
|
|
|
```sh
|
|
hcloud server reboot <server-name>
|
|
```
|
|
|
|
### Bootstrapping the Cluster
|
|
|
|
#### 1. Generate machine configs
|
|
|
|
```sh
|
|
mkdir -p talos
|
|
talosctl gen config k8s-and-chill https://195.201.219.111:6443 --output talos/
|
|
```
|
|
|
|
This creates `controlplane.yaml`, `worker.yaml`, and `talosconfig`.
|
|
|
|
#### 2. Configure talosctl
|
|
|
|
```sh
|
|
export TALOSCONFIG=talos/talosconfig
|
|
talosctl config endpoint 195.201.219.111 195.201.140.75 195.201.219.17
|
|
talosctl config node 195.201.219.111 195.201.140.75 195.201.219.17
|
|
```
|
|
|
|
#### 3. Apply configs
|
|
|
|
Apply the controlplane config to each node (use `--insecure` on first apply since the nodes don't have matching certs yet):
|
|
|
|
```sh
|
|
talosctl apply-config --insecure --nodes 195.201.219.111 --file talos/controlplane.yaml
|
|
talosctl apply-config --insecure --nodes 195.201.140.75 --file talos/controlplane.yaml
|
|
talosctl apply-config --insecure --nodes 195.201.219.17 --file talos/controlplane.yaml
|
|
```
|
|
|
|
#### 4. Bootstrap etcd
|
|
|
|
Run this on exactly one node:
|
|
|
|
```sh
|
|
talosctl bootstrap --nodes 195.201.219.111
|
|
```
|
|
|
|
#### 5. Get kubeconfig
|
|
|
|
```sh
|
|
talosctl kubeconfig talos/kubeconfig --nodes 195.201.219.111
|
|
```
|
|
|
|
#### 6. Verify
|
|
|
|
```sh
|
|
export KUBECONFIG=talos/kubeconfig
|
|
kubectl get nodes -o wide
|
|
kubectl get pods -A
|
|
```
|