Establish clean homelab infrastructure baseline

Reorganize the brownfield repository, remove retired and generated artifacts, harden ignore rules, and record the GitOps/IaC redesign.
2026-09-09 16:47:20 +00:00
commit 88a02ababa
418 changed files with 50579 additions and 0 deletions
@@ -0,0 +1,144 @@
+# Shared PostgreSQL Migration Runbook
+
+## Target
+
+- CloudNativePG cluster: `shared-postgresql`
+- Namespace: `shared-db`
+- Compatibility service: `shared-postgresql.shared-db.svc.cluster.local:5432`
+- CNPG rw service: `shared-postgresql-rw.shared-db.svc.cluster.local:5432`
+- Tailscale service: `shared-postgresql-tailscale.shared-db.svc.cluster.local:5432`
+
+## Apply order
+
+1. Create the CNPG superuser secret
+2. Apply `shared-postgresql/cloudnativepg-cluster.yaml`
+3. Wait for the cluster to become ready
+4. Restore the databases and validate against the CNPG `rw` Service
+5. Remove the legacy Helm release so the old Service can be removed cleanly
+6. Apply `shared-postgresql/shared-postgresql-service.yaml`
+7. Validate app connectivity through the compatibility and Tailscale Services
+
+## Storage preflight
+
+```bash
+kubectl get storageclass localpv-zfs-ceph
+kubectl get pods -n openebs
+```
+
+Make sure the OpenEBS ZFS storage class exists and the OpenEBS components are healthy before starting the cutover.
+
+## Reconcile source passwords
+
+The following passwords are the current source of truth for the migration:
+
+- `SOURCE_POSTGRES_PASSWORD` for the source dump and CNPG superuser secret
+- `OLD_GITEA_PASSWORD` and `OLD_CASDOOR_PASSWORD` for the restored app logins
+
+## Create the CNPG secret
+
+```bash
+kubectl create secret generic shared-postgresql-superuser-secret \
+  -n shared-db \
+  --type=kubernetes.io/basic-auth \
+  --from-literal=username=postgres \
+  --from-literal=password="$SOURCE_POSTGRES_PASSWORD" \
+  --dry-run=client -o yaml | kubectl apply -f -
+```
+
+## Dump from the old instance
+
+Pause writes from dependent apps first.
+
+```bash
+kubectl get pod -n shared-db -l app.kubernetes.io/component=primary,app.kubernetes.io/instance=shared-postgresql,app.kubernetes.io/name=postgresql --show-labels
+SOURCE_POD="$(kubectl get pod -n shared-db -l app.kubernetes.io/component=primary,app.kubernetes.io/instance=shared-postgresql,app.kubernetes.io/name=postgresql -o jsonpath='{.items[0].metadata.name}')"
+kubectl exec -n shared-db "$SOURCE_POD" -- sh -lc "PGPASSWORD='$SOURCE_POSTGRES_PASSWORD' pg_dumpall -h 127.0.0.1 -U postgres --globals-only" > globals.sql
+kubectl exec -n shared-db "$SOURCE_POD" -- sh -lc "PGPASSWORD='$SOURCE_POSTGRES_PASSWORD' pg_dump -h 127.0.0.1 -U postgres -Fc -d e5renew" > e5renew.dump
+kubectl exec -n shared-db "$SOURCE_POD" -- sh -lc "PGPASSWORD='$SOURCE_POSTGRES_PASSWORD' pg_dump -h 127.0.0.1 -U postgres -Fc -d gitea" > gitea.dump
+kubectl exec -n shared-db "$SOURCE_POD" -- sh -lc "PGPASSWORD='$SOURCE_POSTGRES_PASSWORD' pg_dump -h 127.0.0.1 -U postgres -Fc -d casdoor" > casdoor.dump
+```
+
+## Restore into CNPG
+
+`globals.sql` restores the roles and passwords from the source cluster. The init SQL only creates the application databases and grants.
+
+```bash
+cat globals.sql > globals.with-postgres-reset.sql
+printf "ALTER ROLE postgres PASSWORD :'source_postgres_password';\n" >> globals.with-postgres-reset.sql
+PGPASSWORD="$SOURCE_POSTGRES_PASSWORD" psql -v source_postgres_password="$SOURCE_POSTGRES_PASSWORD" -h shared-postgresql-rw.shared-db.svc.cluster.local -U postgres -d postgres -f globals.with-postgres-reset.sql
+PGPASSWORD="$SOURCE_POSTGRES_PASSWORD" psql -h shared-postgresql-rw.shared-db.svc.cluster.local -U postgres -d postgres -f shared-postgresql/shared-postgresql-init.sql
+PGPASSWORD="$SOURCE_POSTGRES_PASSWORD" pg_restore -h shared-postgresql-rw.shared-db.svc.cluster.local -U postgres -d e5renew e5renew.dump
+PGPASSWORD="$SOURCE_POSTGRES_PASSWORD" pg_restore -h shared-postgresql-rw.shared-db.svc.cluster.local -U postgres -d gitea gitea.dump
+PGPASSWORD="$SOURCE_POSTGRES_PASSWORD" pg_restore -h shared-postgresql-rw.shared-db.svc.cluster.local -U postgres -d casdoor casdoor.dump
+```
+
+## Pre-cutover validation
+
+```bash
+PGPASSWORD="$SOURCE_POSTGRES_PASSWORD" psql -h shared-postgresql-rw.shared-db.svc.cluster.local -U postgres -d e5renew -c 'select 1;'
+PGPASSWORD="$SOURCE_POSTGRES_PASSWORD" psql -h shared-postgresql-rw.shared-db.svc.cluster.local -U postgres -d e5renew -c 'create table if not exists migration_check(id int); insert into migration_check values (1); delete from migration_check; drop table migration_check;'
+PGPASSWORD="$OLD_GITEA_PASSWORD" psql -h shared-postgresql-rw.shared-db.svc.cluster.local -U gitea -d gitea -c 'create table if not exists migration_check(id int); insert into migration_check values (1); delete from migration_check; drop table migration_check;'
+PGPASSWORD="$OLD_CASDOOR_PASSWORD" psql -h shared-postgresql-rw.shared-db.svc.cluster.local -U casdoor -d casdoor -c 'create table if not exists migration_check(id int); insert into migration_check values (1); delete from migration_check; drop table migration_check;'
+```
+
+## Post-cutover validation
+
+```bash
+PGPASSWORD="$SOURCE_POSTGRES_PASSWORD" psql -h shared-postgresql.shared-db.svc.cluster.local -U postgres -d e5renew -c 'select 1;'
+PGPASSWORD="$SOURCE_POSTGRES_PASSWORD" psql -h shared-postgresql.shared-db.svc.cluster.local -U postgres -d e5renew -c 'create table if not exists migration_check(id int); insert into migration_check values (1); delete from migration_check; drop table migration_check;'
+PGPASSWORD="$OLD_GITEA_PASSWORD" psql -h shared-postgresql.shared-db.svc.cluster.local -U gitea -d gitea -c 'create table if not exists migration_check(id int); insert into migration_check values (1); delete from migration_check; drop table migration_check;'
+PGPASSWORD="$OLD_CASDOOR_PASSWORD" psql -h shared-postgresql.shared-db.svc.cluster.local -U casdoor -d casdoor -c 'create table if not exists migration_check(id int); insert into migration_check values (1); delete from migration_check; drop table migration_check;'
+PGPASSWORD="$SOURCE_POSTGRES_PASSWORD" psql -h shared-postgresql-tailscale.shared-db.svc.cluster.local -U postgres -d e5renew -c 'select 1;'
+```
+
+## Rollback
+
+- Keep the old Helm-based PostgreSQL deployment until validation passes.
+- If restore fails, delete the CNPG cluster.
+- Delete the CNPG compatibility Service with `kubectl delete -f shared-postgresql/shared-postgresql-service.yaml` so the `shared-postgresql` hostname is free for the legacy release again.
+- Reapply the legacy Helm release.
+- After the old Helm release is back, regenerate the Tailscale Service from the live pod labels and apply it:
+
+```bash
+python - <<'PY' | kubectl apply -f -
+import json
+import subprocess
+import yaml
+
+pod = subprocess.check_output([
+    'kubectl', 'get', 'pod', '-n', 'shared-db',
+    '-l', 'app.kubernetes.io/component=primary,app.kubernetes.io/instance=shared-postgresql,app.kubernetes.io/name=postgresql',
+    '-o', 'json'
+], text=True)
+labels = json.loads(pod)['items'][0]['metadata']['labels']
+selector = {
+    'app.kubernetes.io/component': labels['app.kubernetes.io/component'],
+    'app.kubernetes.io/instance': labels['app.kubernetes.io/instance'],
+    'app.kubernetes.io/name': labels['app.kubernetes.io/name'],
+}
+service = {
+    'apiVersion': 'v1',
+    'kind': 'Service',
+    'metadata': {
+        'name': 'shared-postgresql-tailscale',
+        'namespace': 'shared-db',
+    },
+    'spec': {
+        'type': 'LoadBalancer',
+        'loadBalancerClass': 'tailscale',
+        'ports': [
+            {
+                'name': 'tcp-postgresql',
+                'port': 5432,
+                'protocol': 'TCP',
+                'targetPort': 5432,
+            }
+        ],
+        'selector': selector,
+    },
+}
+print(yaml.safe_dump(service, sort_keys=False))
+PY
+```
+
+- Do not delete the old PVC until the new cluster is verified.