Commit graph

11 commits

Author SHA1 Message Date
Felix Wolf fe51c8c1bc feat(minikube): add minikube environment with garage S3 backend
Adds a self-contained minikube environment for local development and
testing alongside the existing production env.

env: minikube
  - cluster.domain: minikube (browser DNS routes *.minikube → minikube ip)
  - tls issuer: mkcert (CA-signed via cert-manager mkcert ClusterIssuer)
  - storageClass: standard (minikube hostpath provisioner)
  - backups disabled; storagebox disabled
  - excludes argocd, forgejo, hcloud-csi (manual kubectl apply for testing)

prototypes/garage:
  - hand-rolled S3-compatible object store (single Deployment + PVC)
  - mittwald-generated rpc_secret + admin_token (hex)
  - PostSync init Job: assigns cluster layout, ensures bucket and access
    key, writes ocis-s3-credentials cross-namespace into ocis ns
  - idempotent: skips if k8s secret already populated; otherwise rotates
    the garage key (admin API only returns secretAccessKey on create)
  - cross-ns RBAC re-pinned via zz-cross-ns-rbac-fix overlay (ns.ytt.yaml
    clobbers explicit namespace fields)

ocis:
  - new admin-user-id init Job ensures secret.user-id is a valid UUID v4
    (mittwald can't generate UUIDs; ocis-settings rejects non-UUID ids)
  - mittwald no longer manages user-id; existing prod UUIDs preserved
  - insecure flag (oidcIdpInsecure / ocisHttpApiInsecure / ocmInsecure)
    parameterized; defaults to false; minikube sets true for self-signed
    OIDC issuer URL trust

other prototypes:
  - victoria-metrics-single helm values ytt-ified (storageClassName)
  - grafana admin secret now generated by mittwald (was hand-created in
    prod; manifest is no-op there since mittwald only fills empty fields)

flake.nix: minikube + docker + postgresql added to dev shell.
2026-05-03 17:23:57 +02:00
Felix Wolf 85b8fec6b3 feat: replace secret-init Jobs with mittwald operator + cert-manager
Migrate ~180 LOC of openssl/kubectl init Jobs to declarative Secret
manifests reconciled by mittwald/kubernetes-secret-generator (random
strings, SSH keypair) and cert-manager Certificates (RSA private key +
self-signed CA chain). mittwald only fills empty fields, so existing
populated Secrets keep their current values across the migration.

Changes:

- New prototype kubernetes-secret-generator (chart 3.4.1, mittwald helm
  repo). Cluster-wide informer reconciler, no webhook -> cold-bootstrap
  safe via ArgoCD retries.
- New cert-manager selfsigned ClusterIssuer (in-cluster trust root).
  letsencrypt remains for public-DNS endpoints.
- forgejo: admin-secret Job replaced with a mittwald-annotated Secret
  (hex-encoded 24-char password). Deploy-key Job split: mittwald
  ssh-keypair Secret + slim Job that uploads pubkey to Forgejo and
  copies privkey into the argocd repo Secret.
- ocis: 13 Secrets / 16 random fields now mittwald-managed (UUIDs
  replaced with opaque random hex; ocis treats user-id as opaque). IDP
  RSA signing key, LDAP self-signed CA, and LDAP server cert produced
  by cert-manager. Per-Deployment ytt overlay remaps volume key paths
  (tls.crt -> ldap-ca.crt, tls.key -> private-key.pem, etc.) since the
  ocis chart mounts Secrets raw without items support. Old multi-secret
  s3-secret-job replaced with a slim external-secret precheck Job that
  only validates pre-created Hetzner S3/Storage Box credentials.
- Application sync-wave -10 on cert-manager and kubernetes-secret-
  generator so they install before consumers. ArgoCD selfHeal handles
  any residual races.

CLAUDE.md: remove the "all namespaces use privileged PodSecurity"
convention. Existing namespaces still carry the label and will be
audited separately.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 00:00:07 +02:00
Felix Wolf ffa171bfb0 feat: Replace Nextcloud with oCIS (ownCloud Infinite Scale)
Removes the full Nextcloud stack (PostgreSQL/CNPG, Valkey, Caddy sidecar)
and replaces it with oCIS at drive.tr1ceracop.de. oCIS is self-contained
(no external DB/cache needed) with S3ng storage backend on Hetzner Object
Storage (bucket: ocis-tr1ceracop). Chart sourced from git via vendir since
it is not published to a Helm repo.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 20:19:54 +02:00
Felix Wolf d1959dd6cf feat: Adds Nextcloud application deployment
Deploys Nextcloud using an FPM-alpine image with a Caddy sidecar for web serving.

Integrates with an external CloudNativePG cluster for PostgreSQL and a dedicated Valkey instance for caching. Configures S3-compatible object storage for file data.

Includes an initialization Job to create essential admin and Valkey secrets. Sets up Ingress for external access with automated TLS provisioning via cert-manager.

Configures local-path persistence for Nextcloud's core data to ensure state is maintained across pod restarts. Centralizes hostname configuration and migrates various Nextcloud settings to environment variables for streamlined management.

Adds ArgoCD ignore rules for `batch/Job` resource selectors and template labels, preventing spurious out-of-sync states caused by Kubernetes mutations and improving synchronization stability.
2026-04-04 19:24:50 +02:00
Felix Wolf 8af1321177 feat: Add metrics-server for pod/node resource metrics
Enables CPU/memory visibility in k9s and kubectl top by deploying
the Kubernetes metrics-server via the metrics.k8s.io API.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 14:34:32 +02:00
Felix Wolf 167fc62b92 feat: Add automated backups for Forgejo (Postgres + git repos)
- CNPG Barman backup to Hetzner S3 (s3://k8s-and-chill-backups/forgejo/cnpg/)
- ScheduledBackup CR: daily at 2 AM, 30d retention, prefer-standby
- Git repo rclone sync to S3 (s3://k8s-and-chill-backups/forgejo/git/) via CronJob at 3 AM
- Requires secrets: forgejo-backup-s3 (S3 creds), hcloud-token (not used but created)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 17:29:03 +02:00
Felix Wolf 25714eeef6 feat: Migrate Forgejo to CNPG PostgreSQL + Hetzner CSI volumes
- Add hcloud-csi prototype (Hetzner Cloud CSI driver)
- Add cloudnative-pg prototype (CNPG operator)
- Add CNPG Cluster CR for Forgejo (2 instances, lean config for 4GB nodes)
- Add 20Gi hcloud-volumes PVC for Forgejo git repos
- Switch Forgejo from SQLite to PostgreSQL (forgejo-cnpg-rw service)
- Switch Forgejo persistence to hcloud-volumes (forgejo-git-storage)
- Fix ClusterRoleBinding subject namespaces for hcloud-csi and CNPG
- Fix CNPG webhook service namespace references

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 16:37:13 +02:00
Felix Wolf a92c5d8dc2 feat: Add VictoriaMetrics monitoring stack
Adds victoria-metrics-single, grafana, kube-state-metrics, and
node-exporter to the cluster. Enables metrics endpoints on traefik,
argocd, and cert-manager for scraping. Grafana available at
grafana.tr1ceracop.de with VictoriaMetrics as default datasource.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 00:20:23 +02:00
Felix Wolf c7bfd4953c feat: Wire ArgoCD to Forgejo for GitOps management
Configure myks with global repoURL pointing to Forgejo, in-cluster
destination, and disabled placeholder cluster Secret. Implement App of
Apps pattern with a root Application that syncs all child apps.

Add argocd-deploy-key-init Job that generates an ed25519 SSH keypair,
registers it as a deploy key via Forgejo API, and creates the ArgoCD
repository secret with insecure host key verification (avoids
chicken-and-egg with ArgoCD managing its own known hosts ConfigMap).

Additional changes:
- Ignore /status field diffs globally (K8s 1.32 compat)
- Add Replace=true sync option on Jobs (immutable resource compat)
- Switch job images from bitnami/kubectl to alpine/k8s
- Update CLAUDE.md with ArgoCD status and no-bitnami rule

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 23:09:50 +02:00
Felix Wolf 14cb67369d feat: Switch Forgejo SSH to hostPort 222
Use hostPort instead of NodePort for SSH access to avoid cross-node
asymmetric routing issues with kube-proxy nftables mode. Pin Forgejo
pod to node 3 (DNS target) and use port 222 to bypass ISP port 22
blocking.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 20:56:38 +02:00
Felix Wolf 6f717a602f feat: Initial setup of GitOps-managed Kubernetes cluster
Configures `myks` for Helm chart rendering with `ytt` overlays to manage cluster applications.
Defines prototypes and environment-specific configurations for core applications including ArgoCD, Traefik, Cert-Manager, and Forgejo.
Adds comprehensive documentation covering cluster setup, GitOps structure, and development environment.
Integrates `direnv` for environment variable management, `gitignore` for file exclusion, and `sops` for secret encryption.
Includes rendered Kubernetes manifests and ArgoCD application resources for initial deployment.
2026-03-30 18:21:05 +02:00