fe51c8c1bc
Adds a self-contained minikube environment for local development and
testing alongside the existing production env.
env: minikube
- cluster.domain: minikube (browser DNS routes *.minikube → minikube ip)
- tls issuer: mkcert (CA-signed via cert-manager mkcert ClusterIssuer)
- storageClass: standard (minikube hostpath provisioner)
- backups disabled; storagebox disabled
- excludes argocd, forgejo, hcloud-csi (manual kubectl apply for testing)
prototypes/garage:
- hand-rolled S3-compatible object store (single Deployment + PVC)
- mittwald-generated rpc_secret + admin_token (hex)
- PostSync init Job: assigns cluster layout, ensures bucket and access
key, writes ocis-s3-credentials cross-namespace into ocis ns
- idempotent: skips if k8s secret already populated; otherwise rotates
the garage key (admin API only returns secretAccessKey on create)
- cross-ns RBAC re-pinned via zz-cross-ns-rbac-fix overlay (ns.ytt.yaml
clobbers explicit namespace fields)
ocis:
- new admin-user-id init Job ensures secret.user-id is a valid UUID v4
(mittwald can't generate UUIDs; ocis-settings rejects non-UUID ids)
- mittwald no longer manages user-id; existing prod UUIDs preserved
- insecure flag (oidcIdpInsecure / ocisHttpApiInsecure / ocmInsecure)
parameterized; defaults to false; minikube sets true for self-signed
OIDC issuer URL trust
other prototypes:
- victoria-metrics-single helm values ytt-ified (storageClassName)
- grafana admin secret now generated by mittwald (was hand-created in
prod; manifest is no-op there since mittwald only fills empty fields)
flake.nix: minikube + docker + postgresql added to dev shell.
|
||
|---|---|---|
| docs | ||
| envs | ||
| prototypes | ||
| rendered | ||
| talos | ||
| .envrc | ||
| .gitattributes | ||
| .gitignore | ||
| .myks.yaml | ||
| .sops.yaml | ||
| CLAUDE.md | ||
| flake.lock | ||
| flake.nix | ||
| README.md | ||
k8s-and-chill
Private Kubernetes cluster running on 3x Hetzner CAX11 (ARM64) instances with Talos Linux, managed by myks.
Cluster Setup
Prerequisites
Enter the dev shell (via direnv or nix develop), which provides:
talosctlkubectlhelmmykshcloud
Infrastructure
| Node | Public IP | Private IP | Location |
|---|---|---|---|
| ubuntu-4gb-nbg1-1 | 195.201.219.17 | 10.0.0.3 | nbg1 |
| ubuntu-4gb-nbg1-2 | 195.201.140.75 | 10.0.0.4 | nbg1 |
| ubuntu-4gb-nbg1-3 | 195.201.219.111 | 10.0.0.2 | nbg1 |
All nodes are control plane nodes (3-node HA etcd). The Kubernetes API endpoint is https://195.201.219.111:6443.
The nodes are connected via a Hetzner private network (thalos-k8s), which is used for inter-node communication.
Installing Talos on Hetzner Cloud
The servers were originally provisioned with Ubuntu. Talos was installed by writing the Talos disk image via Hetzner rescue mode.
1. Get the Talos image URL
Talos images for Hetzner Cloud are generated via the Talos Image Factory. For vanilla Talos (no extensions), get the schematic ID:
curl -sX POST https://factory.talos.dev/schematics \
-H 'Content-Type: application/json' \
-d '{"customization":{"systemExtensions":{"officialExtensions":[]}}}'
# Returns: {"id":"376567988ad370138ad8b2698212367b8edcb69b5fd68c80be1f2ec7d603b4ba"}
The image URL follows this pattern:
https://factory.talos.dev/image/<schematic-id>/<talos-version>/hcloud-arm64.raw.xz
2. Enable rescue mode and reboot
For each server:
hcloud server enable-rescue <server-name> --ssh-key "<ssh-key-name>"
hcloud server reboot <server-name>
3. Write Talos to disk
SSH into each server's rescue system and write the image:
ssh root@<server-ip> "curl -fsSL '<image-url>' | xz -d | dd of=/dev/sda bs=4M status=progress && sync"
4. Reboot into Talos
hcloud server reboot <server-name>
Bootstrapping the Cluster
1. Generate machine configs
mkdir -p talos
talosctl gen config k8s-and-chill https://195.201.219.111:6443 --output talos/
This creates controlplane.yaml, worker.yaml, and talosconfig.
2. Configure talosctl
export TALOSCONFIG=talos/talosconfig
talosctl config endpoint 195.201.219.111 195.201.140.75 195.201.219.17
talosctl config node 195.201.219.111 195.201.140.75 195.201.219.17
3. Apply configs
Apply the controlplane config to each node (use --insecure on first apply since the nodes don't have matching certs yet):
talosctl apply-config --insecure --nodes 195.201.219.111 --file talos/controlplane.yaml
talosctl apply-config --insecure --nodes 195.201.140.75 --file talos/controlplane.yaml
talosctl apply-config --insecure --nodes 195.201.219.17 --file talos/controlplane.yaml
4. Bootstrap etcd
Run this on exactly one node:
talosctl bootstrap --nodes 195.201.219.111
5. Get kubeconfig
talosctl kubeconfig talos/kubeconfig --nodes 195.201.219.111
6. Verify
export KUBECONFIG=talos/kubeconfig
kubectl get nodes -o wide
kubectl get pods -A