homelab-infra/docs/superpowers/plans/2026-04-18-postgresql-cnpg-zfs-migration.md

PostgreSQL CloudNativePG Migration Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Replace the legacy shared PostgreSQL deployment with a single-instance CloudNativePG cluster on OpenEBS ZFS while preserving the existing shared-postgresql service name for the apps that already depend on it.

Architecture: Keep the old PostgreSQL deployment alive until the new CNPG cluster is ready, then migrate data with a logical dump/restore, swap the service endpoints, and retire the Helm-based deployment only after validation passes. The new cluster stays single-instance because there is only one worker node.

Tech Stack: Kubernetes manifests, CloudNativePG operator, OpenEBS ZFS storage, kubectl, PostgreSQL client tools.

---

File Map

• shared-postgresql/cloudnativepg-cluster.yaml: new authoritative CNPG cluster manifest.
• shared-postgresql/tailscale-loadbalancer.yaml: keep the existing external service, but retarget it to the CNPG primary during cutover.
• shared-postgresql/shared-postgresql-values.yaml: legacy Helm values to delete after the rollback window closes.
• shared-postgresql/shared-postgresql-init.sql: reusable SQL used during migration to recreate app databases and roles.
• shared-postgresql/migration.md: runbook for dump, restore, cutover, and rollback.

Task 1: Add the CNPG cluster manifest

Files:

• Create: shared-postgresql/cloudnativepg-cluster.yaml

• Step 1: Confirm the OpenEBS ZFS storage class name

Run: kubectl get storageclass

Expected: one OpenEBS-backed ZFS class is available and can be copied into shared-postgresql/cloudnativepg-cluster.yaml before apply.

• Step 2: Write the cluster manifest

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: shared-postgresql
  namespace: shared-db
spec:
  instances: 1
  enableSuperuserAccess: true
  storage:
    size: 10Gi
  bootstrap:
    initdb:
      database: shared_data
      owner: sharedadmin

Add the storage.storageClass line using the exact OpenEBS ZFS class name discovered in Step 1.

• Step 3: Verify the manifest is structurally valid

Run: kubectl apply --dry-run=server -f shared-postgresql/cloudnativepg-cluster.yaml

Expected: the API server accepts the Cluster object once the CNPG CRD is installed.

• Step 4: Apply the cluster manifest in the staging namespace

Run: kubectl apply -f shared-postgresql/cloudnativepg-cluster.yaml

Expected: the CNPG cluster is created and starts provisioning a single primary pod.

• Step 5: Wait for the cluster to become ready

Run: kubectl wait -n shared-db --for=condition=Ready cluster/shared-postgresql --timeout=15m

Expected: the cluster reaches a ready state before any data migration begins.

Task 2: Write the migration runbook

Files:

• Create: shared-postgresql/migration.md

• Step 1: Document the source and target endpoints

Source service: shared-postgresql.shared-db.svc.cluster.local:5432
Target rw service: shared-postgresql-rw.shared-db.svc.cluster.local:5432

• Step 2: Document the dump and restore commands

Pause writes from the dependent apps before taking the dump so the logical backup is consistent.

PGPASSWORD="$OLD_SHAREDADMIN_PASSWORD" pg_dump -h shared-postgresql.shared-db.svc.cluster.local -U sharedadmin -Fc -d shared_data -f shared_data.dump
PGPASSWORD="$OLD_GITEA_PASSWORD" pg_dump -h shared-postgresql.shared-db.svc.cluster.local -U gitea -Fc -d gitea -f gitea.dump
PGPASSWORD="$OLD_CASDOOR_PASSWORD" pg_dump -h shared-postgresql.shared-db.svc.cluster.local -U casdoor -Fc -d casdoor -f casdoor.dump

PGPASSWORD="$NEW_SHAREDADMIN_PASSWORD" psql -h shared-postgresql-rw.shared-db.svc.cluster.local -U sharedadmin -d postgres -f shared-postgresql/shared-postgresql-init.sql
PGPASSWORD="$NEW_SHAREDADMIN_PASSWORD" pg_restore -h shared-postgresql-rw.shared-db.svc.cluster.local -U sharedadmin -d shared_data shared_data.dump
PGPASSWORD="$NEW_GITEA_PASSWORD" pg_restore -h shared-postgresql-rw.shared-db.svc.cluster.local -U gitea -d gitea gitea.dump
PGPASSWORD="$NEW_CASDOOR_PASSWORD" pg_restore -h shared-postgresql-rw.shared-db.svc.cluster.local -U casdoor -d casdoor casdoor.dump

• Step 3: Document the rollback decision point

If restore or validation fails, keep the old Helm deployment active and do not delete its Service or PVC.

• Step 4: Review the runbook for exact cutover order

Expected: the document explains that the old service stays live until data restore and app checks pass.

Task 3: Cut over services to CNPG

Files:

• Modify: shared-postgresql/tailscale-loadbalancer.yaml

• Create: shared-postgresql/shared-postgresql-service.yaml

• Step 1: Add the compatibility ClusterIP service

apiVersion: v1
kind: Service
metadata:
  name: shared-postgresql
  namespace: shared-db
spec:
  type: ClusterIP
  selector:
    cnpg.io/cluster: shared-postgresql
    cnpg.io/instanceRole: primary
  ports:
    - name: postgres
      port: 5432
      targetPort: 5432

• Step 2: Update the Tailscale LoadBalancer selector

apiVersion: v1
kind: Service
metadata:
  name: shared-postgresql-tailscale
  namespace: shared-db
spec:
  type: LoadBalancer
  loadBalancerClass: tailscale
  ports:
    - name: tcp-postgresql
      port: 5432
      protocol: TCP
      targetPort: 5432
  selector:
    cnpg.io/cluster: shared-postgresql
    cnpg.io/instanceRole: primary

• Step 3: Apply the compatibility service only after the Helm release is removed

Run: kubectl apply -f shared-postgresql/shared-postgresql-service.yaml -f shared-postgresql/tailscale-loadbalancer.yaml

Expected: the original shared-postgresql hostname resolves to the CNPG primary and Tailscale reaches the same pod.

• Step 4: Validate application connectivity

Run: PGPASSWORD="$NEW_SHAREDADMIN_PASSWORD" psql -h shared-postgresql.shared-db.svc.cluster.local -U sharedadmin -d shared_data -c 'select 1;'

Expected: the compatibility Service resolves and accepts a real PostgreSQL login after the swap.

Task 4: Remove the legacy Helm deployment

Files:

• Delete: shared-postgresql/shared-postgresql-values.yaml

• Step 1: Confirm all dependent apps are using the CNPG-backed Service

Run: kubectl get endpoints -n shared-db shared-postgresql

Expected: the endpoints point at the CNPG pod, not the old Helm chart pod.

• Step 2: Remove the old Helm values file after the rollback window

Run: git rm shared-postgresql/shared-postgresql-values.yaml

Expected: the repository no longer advertises the retired deployment path once cutover is stable.

• Step 3: Keep the bootstrap SQL file as the migration reference

Expected: shared-postgresql/shared-postgresql-init.sql remains in the tree until the migration is fully complete and documented.

• Step 4: Re-run the manifest validation pass

Run: kubectl apply --dry-run=server -f shared-postgresql/

Expected: the remaining manifest set is clean and consistent.