From 0984a2d0efa359cc43620f16b160f180af914638 Mon Sep 17 00:00:00 2001 From: panxiao81 Date: Wed, 9 Sep 2026 18:28:45 +0000 Subject: [PATCH] docs: record live CI bootstrap --- CHANGELOG.md | 3 ++- docs/cicd.md | 15 ++++++++------- docs/homelab-gitops-redesign.md | 32 ++++++++++++++++++++------------ 3 files changed, 30 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5c6886c..bde652b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,9 +28,10 @@ What changed in this homelab, when, and why. Newest first. | cleanup | Removed the retired NapCat tree, the Contour and Kanidm archive trees, and seven generated Terraform plan files before establishing the clean Git baseline; plans may embed complete state and remain globally ignored | | CI | Added a review-first Gitea Actions runner bootstrap: official actions chart 0.1.1, pinned runner 2.3.0, one persistent instance-scoped Kubernetes runner with capacity four, plus an ESO reference to its registration token in OpenBao. The first deployment proved that rootlesskit is blocked by the node's AppArmor unprivileged-userns policy; because the chart requires privileged DinD in either mode, the reviewed fix uses regular DinD instead of weakening the host-wide policy. The runner image intentionally carries neither `uv` nor Terraform: Terraform uses its versioned setup action, while `uv` is pinned and installed from official PyPI because the nested job network reaches PyPI but times out against the GitHub API queried by `setup-uv`. Ansible installs only `ansible-core` in its tool venv and puts declared Galaxy collections in a shared path visible to ansible-lint; installing the `ansible` meta-package had made Galaxy falsely skip that shared installation | | identity | Declared the Samba AD `gitea-admins` group with `panxiao81` as its initial member. Gitea already maps this OIDC group to site administrators; the local `gitea_admin` account remains as break-glass access | +| docs | Reconciled the redesign and CI status with reality: the Gitea remote, instance-scoped runner, OpenBao-projected registration token and green Stage 1 are live; Flux, off-site mirroring and recovery verification remain pending | `Carried forward`: re-verify OpenBao/ESO recovery and remaining Secret inventory; -configure a Git remote and off-site mirror; confirm the running Gitea version; +configure an off-site Git mirror; plan the Gitea upgrade beyond 1.25.5; take an encrypted independent OCI state copy before enabling bucket versioning; reconstruct the missing root to a zero-change plan; bootstrap Gitea Actions and Flux on a low-risk service; then move Tunnel origins to Envoy one hostname at a time. diff --git a/docs/cicd.md b/docs/cicd.md index d008335..73d1a42 100644 --- a/docs/cicd.md +++ b/docs/cicd.md @@ -1,6 +1,7 @@ # CI/CD — what we are building -Status: **design, partly built.** Stage 1 is live. The substrate is undecided. +Status: **partly built.** Stage 1 and the Kubernetes runner are live; Flux and +credentialed stages are not yet installed. Started 2026-07-28. ## Goal @@ -22,9 +23,9 @@ to make drift between this repo and reality visible when it happens. | | | |---|---| -| git | History since 2026-07-28. Four commits, **no remote yet** | -| stage 1 | Live and green — `yamllint`, `ansible-lint`, `terraform fmt`/`validate`. Configs tuned against a real run | -| gitea | 1.25.5, Actions **enabled**, `DEFAULT_ACTIONS_URL=github`. **No runner deployed**, so nothing executes | +| git | `homelab-infra` is hosted on the local Gitea; an independent off-site mirror is still missing | +| stage 1 | Live and green on the Gitea runner — `yamllint`, `ansible-lint`, `terraform fmt`/`validate` | +| gitea | 1.25.5, Actions enabled, `DEFAULT_ACTIONS_URL=github`; one instance-scoped Kubernetes runner is deployed with capacity four | | ansible | 33 roles across `infrastructure/proxmox/`, `infrastructure/samba-ad/`, `infrastructure/openbao/` | | terraform | 4 roots, **local state**, each with **different interactive auth** (`bao login -method=oidc`, `az login`) | | k8s | ~13 Helm releases, all deployed by hand | @@ -171,10 +172,10 @@ the CI system itself. ## Sequencing -1. Externalise secrets — unblocks everything, valuable on its own +1. Externalise secrets — **partly complete**; ESO delivery works, recovery and the remaining inventory are pending 2. Capture `e5renew` and `rustfs` into the repo (`helm get values`) -3. Git remote -4. Pick the substrate; stand it up in an isolated namespace +3. Git remote — **complete locally**; off-site mirror pending +4. Pick the substrate; stand it up in an isolated namespace — **complete** 5. Stage 2, then stage 3 on **one** container-friendly role first 6. Flux on one low-stakes namespace (`http-echo` or `marker`) 7. Drift detection for Terraform and Ansible — scoped machine identities diff --git a/docs/homelab-gitops-redesign.md b/docs/homelab-gitops-redesign.md index 29f84b1..9fb7c2c 100644 --- a/docs/homelab-gitops-redesign.md +++ b/docs/homelab-gitops-redesign.md @@ -1,6 +1,6 @@ # Homelab GitOps and IaC redesign -Status: **design; no infrastructure changes have been applied.** +Status: **implementation in progress; CI bootstrap is live, Flux is not yet installed.** Started 2026-09-09. This is the durable record of the redesign discussion. It separates observations, decisions and open work so an assumption cannot silently @@ -41,7 +41,11 @@ or reconcile later CPU, memory, NIC or boot drift. - Repository history records External Secrets Operator 2.8.0 as deployed. Five `ExternalSecret` resources cover Authelia, Gitea, Cloudflared and SeaweedFS. All five reported `SecretSynced=True` during a live check on 2026-09-09. -- The repository has no configured Git remote yet. +- Gitea Actions now has one instance-scoped runner in namespace `gitea-actions`. + Its Helm release is deployed, its Pod is `2/2 Running`, its identity PVC is + bound, and a sixth `ExternalSecret` delivers the registration token from OpenBao. +- The repository is hosted at `panxiao81/homelab-infra` on the local Gitea + instance. A one-way off-site mirror is still missing. - Terraform roots remain per-service and must not be merged. - The k3s node was `Ready` on 2026-09-09. The Snap-packaged `kubectl` could not start because the user systemd session was degraded; `k3s kubectl` with the @@ -84,10 +88,12 @@ without becoming another deployment controller. ### CI execution Most CI uses Gitea Actions for its GitHub Actions compatibility. A persistent -Gitea Runner StatefulSet runs in Kubernetes with rootless Docker-in-Docker and -capacity four; individual job containers are created dynamically. Rootless DinD -still requires a privileged Pod, so the runner is repository-scoped and accepts -trusted workflows only. +Gitea Runner StatefulSet runs in Kubernetes with Docker-in-Docker and capacity +four; individual job containers are created dynamically. The chart requires a +privileged DinD container in both modes, and rootlesskit is blocked by the node's +AppArmor unprivileged-userns policy, so regular DinD is used instead of weakening +that host-wide policy. The runner is instance-scoped and accepts trusted +repositories and workflows only. Only explicitly labelled jobs needing privilege, nested virtualization, amd64-only software or isolation from k3s use an ephemeral Proxmox VM. IaC owns @@ -196,13 +202,15 @@ offline break-glass path. ESO-generated Secrets are projections, not backups. ## Implementation phases -1. Preserve OCI state and sanitized libvirt/Helm evidence. -2. Re-verify the existing OpenBao/ESO delivery and recovery path, inventory +1. **In progress:** sanitized libvirt/Helm evidence is recorded; the independent + encrypted OCI state backup is still missing. +2. **In progress:** live OpenBao/ESO delivery is verified; verify recovery and inventory remaining manually managed Secrets, then migrate them incrementally. -3. Configure Gitea remote plus a one-way off-site mirror. Confirm whether the - running Gitea supports the 1.27 Terraform State Registry. -4. Manually deploy the reviewed Gitea Runner bootstrap, then bootstrap Flux on - `http-echo` or `marker` without enabling prune until live ownership is audited. +3. **In progress:** the Gitea remote exists; add a one-way off-site mirror and + revisit the Terraform State Registry after upgrading beyond Gitea 1.25.5. +4. **In progress:** the reviewed Gitea Runner is deployed and Stage 1 CI is green; + bootstrap Flux on `http-echo` or `marker` without enabling prune until live + ownership is audited. 5. Move Tunnel origins to Envoy and consolidate split DNS through Blocky. 6. Deploy Backstage read-only with Catalog, Kubernetes, Flux and TechDocs. 7. Reconstruct the OCI root to a zero-change plan and add libvirt drift reports.