# k8s-and-chill

## Project Overview
GitOps-managed Kubernetes cluster on Hetzner Cloud running Talos Linux. Uses [myks](https://github.com/mykso/myks) for Helm chart rendering with ytt overlays, targeting ArgoCD for continuous deployment.

## Cluster
- **3 Talos control-plane nodes** (CAX11 ARM64, Hetzner Cloud Nuremberg)
- Node IPs: `195.201.219.111`, `195.201.140.75`, `195.201.219.17`
- `allowSchedulingOnControlPlanes: true` (no dedicated workers)

## Domain & DNS
- **Domain**: `tr1ceracop.de` (registered at INWX)
- **DNS**: Managed at INWX with wildcard A record `*.tr1ceracop.de` pointing to node IPs
- **Forgejo**: `https://git.tr1ceracop.de`
- **ArgoCD**: `https://argocd.tr1ceracop.de`

## Deployed Applications
| App | Namespace | Notes |
|-----|-----------|-------|
| traefik | traefik | Ingress controller, DaemonSet with hostPort 80/443 |
| cert-manager | cert-manager | Let's Encrypt HTTP-01 via ClusterIssuer `letsencrypt` |
| forgejo | forgejo | Git server, SQLite, local-path PVC |
| argocd | argocd | GitOps controller |
| local-path-provisioner | local-path-storage | Default StorageClass, installed via upstream manifest |

## myks Structure
```
prototypes/         # Application templates (helm values + ytt overlays)
  argocd/
  traefik/
  cert-manager/
  forgejo/
envs/
  env-data.ytt.yaml           # Global ArgoCD config
  _env/                       # Shared overlays (annotations, secrets)
  production/
    env-data.ytt.yaml          # App list for production
    _apps/{app}/app-data.ytt.yaml  # Per-app overrides
rendered/
  envs/production/{app}/       # kubectl-ready manifests
  argocd/production/           # ArgoCD Application resources
talos/
  controlplane.yaml            # Talos machine config
  talosconfig                  # Talos client config
  kubeconfig                   # Cluster kubeconfig
```

### Prototype Pattern
Each prototype follows this structure:
- `app-data.ytt.yaml` — namespace declaration
- `vendir/vendir-data.ytt.yaml` — chart name, version, repository URL
- `vendir/base.ytt.yaml` — vendir config template (identical across all)
- `helm/{chart}.yaml` — Helm values overrides
- `ytt/ns.ytt.yaml` — Namespace resource + namespace overlay on all resources

### Key Commands
```bash
myks render                          # Render all apps
myks render production <app>         # Render single app
kubectl apply -f rendered/envs/production/<app>/ --server-side  # Deploy
```

## Kubeconfig & Talos
- `KUBECONFIG` and `TALOSCONFIG` are already set in the user's shell environment. Do not set them in commands.


## Known Issues / TODOs
- **Namespace race condition**: First `kubectl apply` of a new app often fails because namespace isn't ready. Re-apply once.
- **Traefik DaemonSet updates**: Requires `updateStrategy.rollingUpdate.maxSurge: 0` because hostPort conflicts prevent surge.
- **Forgejo Ingress API version**: Chart renders `extensions/v1beta1`, fixed via `ytt/ingress-fix.ytt.yaml` overlay to `networking.k8s.io/v1`.
- **ArgoCD**: Fully wired to Forgejo via App of Apps. Root Application in `default` project syncs `rendered/argocd/production/`. Deploy key provisioned automatically by `argocd-deploy-key-init` Job in forgejo namespace.

## Container Images
- **Never use bitnami images.** Use `alpine/k8s` or plain `alpine` for utility Jobs instead.

## Secrets
- **Never commit secrets to git.** This is a public repository.
- **All secrets must be generated in-cluster** using init Jobs (ArgoCD PreSync hooks) that create secrets if they don't already exist. See `prototypes/ocis/ytt/s3-secret-job.ytt.yaml` for the pattern.
- **External secrets** (e.g. S3 credentials) that cannot be generated must be created manually in the cluster before deploying. The init Job should validate their existence and fail fast if missing.
- When adding a new application that uses a Helm chart generating secrets, configure all `secretRefs` to point to pre-created secret names and use an init Job to generate them.
- Known external secrets (not in git, created manually):
  - `ocis/ocis-s3-credentials` — Hetzner S3 access key and secret key
  - `ocis/ocis-storagebox-credentials` — Hetzner Storage Box host, user, and SSH private key (for S3 backup to Helsinki)
  - `cert-manager/letsencrypt-account-key` — ACME account key (auto-generated by cert-manager)
