Merge 原子部署解耦后的 dynamic runner 镜像

fix: 原子部署解耦后的 controller 与 runner
Merge 部署修复后的 dynamic runner placement v2
2026-09-25 19:12:07 +00:00 · 2026-09-25 19:11:39 +00:00 · 2026-09-25 18:54:41 +00:00 · 2026-09-25 18:53:23 +00:00 · 2026-09-25 18:37:40 +00:00 · 2026-09-25 18:33:32 +00:00
133 changed files with 3802 additions and 668 deletions
@@ -16,9 +16,6 @@ on:
      - '.ansible-lint'
      - '.gitea/workflows/ansible.yml'
 env:
  ANSIBLE_COLLECTIONS_PATH: /root/.ansible/collections
 jobs:
  lint:
    runs-on: [self-hosted, pod]
@@ -30,6 +27,7 @@ jobs:
          python3 -m pip install --user --break-system-packages \
            --index-url https://pypi.org/simple --quiet uv==0.11.7
          echo "$HOME/.local/bin" >> "$GITHUB_PATH"
          echo "ANSIBLE_COLLECTIONS_PATH=$HOME/.ansible/collections" >> "$GITHUB_ENV"
      - name: Install ansible-lint and collections
        run: |
@@ -50,7 +48,7 @@ jobs:
      - name: ansible-lint
        run: |
          export PATH="$HOME/.local/bin:$PATH"
-          export ANSIBLE_COLLECTIONS_PATH="$PWD/infrastructure/samba-ad/ansible/collections:/root/.ansible/collections"
+          export ANSIBLE_COLLECTIONS_PATH="$PWD/infrastructure/samba-ad/ansible/collections:$HOME/.ansible/collections"
          # 静态检查不应依赖生产 vault 凭据。一次性 checkout 可以去掉加密变量文件；
          # syntax-check 只验证结构，不需要解析变量的运行时值。
          rm -f \
@@ -0,0 +1,22 @@
 name: hydra-login
 on:
  pull_request:
    paths:
      - 'apps/hydra/login-consent/**'
      - '.gitea/workflows/hydra.yml'
  workflow_dispatch:
 jobs:
  verify:
    runs-on: [self-hosted, pod]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
        with:
          go-version-file: apps/hydra/login-consent/go.mod
          cache-dependency-path: apps/hydra/login-consent/go.sum
      - name: Test authentication boundaries
        working-directory: apps/hydra/login-consent
        run: |
          go test -race ./...
          go vet ./...
          CGO_ENABLED=0 go build -trimpath .
@@ -0,0 +1,20 @@
 # Backstage
 Backstage 作为 homelab 的只读开发者门户运行。应用源码、插件、测试和镜像构建归
 `panxiao81/backstage` 仓库管理；本目录只保存 Kubernetes/Flux 部署声明，并通过
 OCI digest 固定镜像。
 ## 外部前置
 - OpenBao `kv/k8s/backstage` 必须包含以下与容器环境变量同名的字段：
  `BACKEND_SECRET`、`AUTH_SESSION_SECRET`、`AUTH_OIDC_CLIENT_ID`、
  `AUTH_OIDC_CLIENT_SECRET`、`POSTGRES_PASSWORD`、`GITEA_TOKEN`。
 - PostgreSQL 需要在共享集群中预先创建由 `backstage` 角色拥有的 `backstage`
  数据库；密码必须与 OpenBao 中的 `POSTGRES_PASSWORD` 一致。
 - Authelia OIDC 客户端回调地址为
  `https://backstage.ad.ddupan.top/api/auth/oidc/handler/frame`。
 - AD DNS 需要将 `backstage.ad.ddupan.top` 指向 Envoy Gateway
  `192.168.10.127`。
 Flux 等待 ExternalSecret 和 Deployment 就绪；任何前置缺失都会使该
 Kustomization 保持 NotReady，而不会回退到明文 Secret。
@@ -0,0 +1,85 @@
 apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: backstage
  namespace: backstage
  labels:
    app.kubernetes.io/name: backstage
    backstage.io/kubernetes-id: homelab-backstage
 spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app.kubernetes.io/name: backstage
  template:
    metadata:
      labels:
        app.kubernetes.io/name: backstage
        backstage.io/kubernetes-id: homelab-backstage
    spec:
      serviceAccountName: backstage
      securityContext:
        fsGroup: 1000
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: backstage
          image: zot.ad.ddupan.top/panxiao81/backstage@sha256:e5a12550726f19a680bc7c40e2cc07cc624318f9279ee121814d293a006ef210
          imagePullPolicy: IfNotPresent
          env:
            - name: BACKSTAGE_BASE_URL
              value: https://backstage.ad.ddupan.top
            - name: POSTGRES_HOST
              value: shared-postgresql-rw.shared-db.svc.cluster.local
            - name: POSTGRES_PORT
              value: "5432"
            - name: POSTGRES_USER
              value: backstage
            - name: POSTGRES_DATABASE
              value: backstage
            - name: GITEA_HOST
              value: git.ddupan.top
          envFrom:
            - secretRef:
                name: backstage
          ports:
            - containerPort: 7007
              name: http
              protocol: TCP
          readinessProbe:
            httpGet:
              path: /.backstage/health/v1/readiness
              port: http
            initialDelaySeconds: 10
            periodSeconds: 10
            timeoutSeconds: 3
          livenessProbe:
            httpGet:
              path: /.backstage/health/v1/liveness
              port: http
            initialDelaySeconds: 30
            periodSeconds: 20
            timeoutSeconds: 3
          resources:
            requests:
              cpu: 100m
              memory: 512Mi
            limits:
              cpu: "1"
              memory: 1Gi
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: [ALL]
            readOnlyRootFilesystem: true
            runAsNonRoot: true
            runAsUser: 1000
          volumeMounts:
            - mountPath: /tmp
              name: tmp
      volumes:
        - emptyDir: {}
          name: tmp
@@ -0,0 +1,16 @@
 apiVersion: external-secrets.io/v1
 kind: ExternalSecret
 metadata:
  name: backstage
  namespace: backstage
 spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: openbao
  target:
    creationPolicy: Owner
    name: backstage
  dataFrom:
    - extract:
        key: k8s/backstage
@@ -0,0 +1,20 @@
 apiVersion: gateway.networking.k8s.io/v1
 kind: HTTPRoute
 metadata:
  name: backstage
  namespace: backstage
 spec:
  parentRefs:
    - name: eg
      namespace: envoy-gateway-system
      sectionName: https
  hostnames:
    - backstage.ad.ddupan.top
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: backstage
          port: 7007
@@ -0,0 +1,11 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
  - namespace.yaml
  - serviceaccount.yaml
  - rbac.yaml
  - external-secret.yaml
  - deployment.yaml
  - service.yaml
  - networkpolicy.yaml
  - httproute.yaml
@@ -0,0 +1,7 @@
 apiVersion: v1
 kind: Namespace
 metadata:
  name: backstage
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: v1.36
@@ -0,0 +1,24 @@
 apiVersion: networking.k8s.io/v1
 kind: NetworkPolicy
 metadata:
  name: backstage-ingress
  namespace: backstage
 spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: backstage
  policyTypes: [Ingress]
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: envoy-gateway-system
      ports:
        - port: 7007
          protocol: TCP
    - from:
        - ipBlock:
            cidr: 192.168.10.127/32
      ports:
        - port: 7007
          protocol: TCP
@@ -0,0 +1,51 @@
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRole
 metadata:
  name: backstage-read-only
 rules:
  - apiGroups: [""]
    resources:
      - configmaps
      - limitranges
      - pods
      - pods/log
      - resourcequotas
      - services
    verbs: [get, list, watch]
  - apiGroups: [apps]
    resources:
      - daemonsets
      - deployments
      - replicasets
      - statefulsets
    verbs: [get, list, watch]
  - apiGroups: [autoscaling]
    resources:
      - horizontalpodautoscalers
    verbs: [get, list, watch]
  - apiGroups: [batch]
    resources:
      - cronjobs
      - jobs
    verbs: [get, list, watch]
  - apiGroups: [networking.k8s.io]
    resources:
      - ingresses
    verbs: [get, list, watch]
  - apiGroups: [metrics.k8s.io]
    resources:
      - pods
    verbs: [get, list]
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRoleBinding
 metadata:
  name: backstage-read-only
 roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: backstage-read-only
 subjects:
  - kind: ServiceAccount
    name: backstage
    namespace: backstage
@@ -0,0 +1,15 @@
 apiVersion: v1
 kind: Service
 metadata:
  name: backstage
  namespace: backstage
  labels:
    backstage.io/kubernetes-id: homelab-backstage
 spec:
  selector:
    app.kubernetes.io/name: backstage
  ports:
    - name: http
      port: 7007
      protocol: TCP
      targetPort: http
@@ -0,0 +1,6 @@
 apiVersion: v1
 kind: ServiceAccount
 metadata:
  name: backstage
  namespace: backstage
 automountServiceAccountToken: true
@@ -44,3 +44,15 @@ API、OIDC、Git/Flux 和 runner 均已验证。第二跳按明确决定跳过
 结构，或者 release notes 指出相关 breaking migration 时，才恢复停机一致备份、分阶段
 suspend、详细日志审计和扩展验收。出现启动失败或 migration error 时也立即升级为完整
 故障流程。
 ## Hydra 人类登录 PoC
 新增 `hydra` OIDC 登录源，旧 `authelia` 入口保留。Hydra 通过通用 OIDC Login/Consent
 适配器转到现有 Authelia 完成人类认证；不是 Gitea 直接验证 LDAP 或 SPIFFE。
 入口为 `https://git.ddupan.top/user/oauth2/hydra`，需 LAN/Tailscale 可达 Hydra 内网域名。
 新 client secret 通过 `ExternalSecret/gitea-hydra-oidc` 从 OpenBao 投射。沿用
 preferred_username、已验证邮箱与 groups；当前仍映射 gitea-admins，不在本轮切换组模型。
 先部署并验证 Hydra discovery 后再接入本配置，避免 Gitea init 因上游不可达而失败。
 实际登录验收与部署状态见 wiki；依赖和回退见 [Hydra README](../hydra/README.md)。
@@ -116,6 +116,13 @@ gitea:
      scopes: openid profile email groups
      groupClaimName: groups
      adminGroup: gitea-admins
    - name: hydra
      provider: openidConnect
      existingSecret: gitea-hydra-oidc
      autoDiscoverUrl: https://hydra.ad.ddupan.top/.well-known/openid-configuration
      scopes: openid profile email groups
      groupClaimName: groups
      adminGroup: gitea-admins
 persistence:
  size: 20Gi
@@ -0,0 +1,22 @@
 apiVersion: external-secrets.io/v1
 kind: ExternalSecret
 metadata:
  name: gitea-hydra-oidc
  namespace: gitea
 spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: openbao
  target:
    name: gitea-hydra-oidc
    creationPolicy: Owner
    template:
      data:
        key: gitea
        secret: "{{ .client_secret }}"
  data:
    - secretKey: client_secret
      remoteRef:
        key: k8s/hydra
        property: gitea_client_secret
@@ -16,3 +16,4 @@ resources:
  - helmrepository.yaml
  - helmrelease.yaml
  - httproute.yaml
  - hydra-external-secret.yaml
@@ -0,0 +1,102 @@
 # Hydra 与 OIDC Login/Consent PoC
 本目录提供独立 Hydra 签发服务，以及一个薄的 **OIDC 上游适配器**。当前上游配置为
 Authelia；适配器不连接 LDAP，也不管理用户目录。Samba AD、密码和 MFA 继续由现有
 Authelia 链路负责。第一轮只接入人类和 Gitea，不实现 agent 动态授权。
 ```text
 Gitea → Hydra → OIDC Login/Consent → Authelia → Samba AD
      ← OIDC ← 经验证的上游身份 ← OIDC callback
 ```
 目标入口：
 - `https://hydra.ad.ddupan.top`：Hydra 公共 OAuth2/OIDC endpoint。
 - `https://hydra-login.ad.ddupan.top`：上游 OIDC 登录及 consent 适配器。
 - `hydra-admin.hydra.svc.cluster.local:4445`：仅集群内管理接口，无 HTTPRoute。
 均为 LAN/Tailscale 入口，复用 Envoy `eg/https` wildcard TLS。没有增加公网 tunnel。
 部署及实际验收状态以 wiki 和对应 PR 为准，文件存在不表示登录已验收。
 ## 首次使用与边界
 在 Gitea 登录页选择 `hydra`，跳转到 Authelia 完成现有人类认证，再返回原有 Gitea
 账号。旧 `authelia` 登录源保留。Gitea 的账号关联和资源权限仍由 Gitea 维护。
 适配器要求验证上游 issuer、audience、签名、过期时间和 nonce，使用 PKCE S256，
 并把单次 state 绑定到 Secure/HttpOnly/SameSite=Lax cookie。短期登录事务只存内存，
 最多 1024 个、10 分钟过期；单副本重启后正在登录的用户需重试，不存人类密码或 token。
 Hydra subject 为上游 `(issuer, sub)` 的 SHA-256 加 `human:` 前缀，与可变邮箱/用户名
 分离。第一轮要求上游返回经过验证的 email 及 preferred_username；这些 claims 必须
 明确配置进 ID token。更换 issuer 会改变本 PoC 的 subject，正式迁移前需要身份绑定设计。
 仅为显式 `ALLOWED_CLIENTS=gitea` 自动 consent，scope 限于 openid/profile/email/groups；
 拒绝额外 access-token audience，不发 refresh token。只按实际请求 scope 释放 claims。
 这不是通用的无人确认授权服务。组当前透传，沿用 Gitea 的 gitea-admins 映射；统一组
 模型和 agent 认证均在后续阶段。不存在对 Authelia 专有协议的调用。
 NetworkPolicy 限制公共端口只接收 Envoy 流量，Hydra admin 只允许适配器访问。
 Hydra 使用正式模式，TLS 由 Envoy 终止；不使用 `--dev`。管理操作使用受控
 `kubectl port-forward`，不要将 admin 接口暴露到 Gateway。
 ## 依赖、秘密与初始化
 依赖共享 CloudNativePG、OpenBao/ESO、Authelia OIDC、Envoy、Samba DNS、zot 镜像仓库。
 Hydra 使用独立 `hydra` database/role，不与其他应用共享数据库角色。
 `kv/k8s/hydra` 保存 dsn、system_secret、upstream_client_secret、upstream_client_digest、
 gitea_client_secret；通过 ExternalSecret 投射，值不写入 Git。Bootstrap 创建角色及数据库
 后才启动 Hydra migration。system_secret 必须持久保存，不得在重启时随机重建。
 Authelia 中新增 confidential client `hydra-login`：
 - redirect URI：`https://hydra-login.ad.ddupan.top/callback`；
 - authorization policy：two_factor；grant：authorization_code；PKCE：S256；
 - token endpoint auth：client_secret_basic；scope：openid/profile/email/groups；
 - claims policy：把 preferred_username、name、email、email_verified、groups 放入 ID token；
 - client secret 的 PBKDF2 digest 存入 Authelia，原值仅供适配器使用。
 Authelia 尚非 Flux 管理。修改 Helm values 时保留所有已有 clients 与 secret 引用，
 通过 `--reuse-values` 和最小 overlay 增加客户端，不能以本目录配置覆盖其完整 values。
 Hydra 中注册 confidential client `gitea`，redirect URI 为
 `https://git.ddupan.top/user/oauth2/hydra/callback`，grant/response 为 authorization_code/code，
 scope 为 openid/profile/email/groups，token endpoint auth 为 client_secret_basic。
 Gitea 启动时读取 OIDC discovery，所以应先确认 Hydra 健康和 discovery 可达，再接入 Gitea。
 ## 构建与检查
 ```bash
 cd apps/hydra/login-consent
 go test -race ./...
 go vet ./...
 CGO_ENABLED=0 go build -trimpath -ldflags='-s -w' -o login-consent .
 docker build -t hydra-login-consent:VERSION .
 ```
 Go module 独立，依赖由 go.sum 锁定；Dockerfile 固定基础镜像 digest。
 使用已授权的短期 SPIFFE zot 凭据发布镜像，部署使用匿名拉取入口与不可变 digest。
 不把 registry 凭据写入源码或 build args。
 ```bash
 kubectl kustomize apps/hydra
 sudo k3s kubectl -n hydra get deployment,pod,externalsecret,httproute
 sudo k3s kubectl -n hydra logs deployment/hydra -c migrate
 sudo k3s kubectl -n hydra logs deployment/hydra-login
 ```
 日志不输出上游 token、授权 code、challenge 或秘密。登录失败先查两端 Pod 状态、
 DNS/discovery 连通性、client redirect URI 和 scope，再由用户重新发起登录。
 不要在故障排查中关闭签名验证、MFA 或 state/nonce 校验。
 ## 恢复与回退
 保留共享 PostgreSQL 中 Hydra 数据及 OpenBao 秘密；数据库持有 clients、会话及签名密钥，
 单独重建 Deployment 不能替代恢复数据库。先恢复依赖，再启动 Hydra 和适配器。
 当前恢复仍依赖 homelab 共享基础设施，不能声称已完成独立灾备。
 第一轮不切换 Authelia 的主入口。撤回 Gitea 的新增 Hydra 登录源即可回到旧入口；
 先撤消费者，再考虑停用 Hydra。不要删除旧 Authelia 登录源、用户或数据库作为回退手段。
 跨服务设计见 [独立 IAM 草案](https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/architecture/independent-iam-draft.md)。
@@ -0,0 +1,95 @@
 apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: hydra
  namespace: hydra
 spec:
  replicas: 1
  selector:
    matchLabels:
      app: hydra
  template:
    metadata:
      labels:
        app: hydra
    spec:
      automountServiceAccountToken: false
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        seccompProfile:
          type: RuntimeDefault
      initContainers:
      - name: migrate
        image: docker.io/oryd/hydra:v26.2.0@sha256:ff67c7fb5f95074fa53374d41151713554960504b340cd3f95b09e65deaea2a9
        args:
        - migrate
        - sql
        - -e
        - --yes
        env:
        - name: DSN
          valueFrom:
            secretKeyRef:
              name: hydra
              key: dsn
        securityContext: &id002
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
          capabilities:
            drop:
            - ALL
        resources: &id001
          requests:
            cpu: 50m
            memory: 64Mi
          limits:
            memory: 256Mi
      containers:
      - name: hydra
        image: docker.io/oryd/hydra:v26.2.0@sha256:ff67c7fb5f95074fa53374d41151713554960504b340cd3f95b09e65deaea2a9
        args:
        - serve
        - all
        - --config
        - /etc/hydra/hydra.yaml
        - --sqa-opt-out
        env:
        - name: DSN
          valueFrom:
            secretKeyRef:
              name: hydra
              key: dsn
        - name: SECRETS_SYSTEM
          valueFrom:
            secretKeyRef:
              name: hydra
              key: system_secret
        ports:
        - name: public
          containerPort: 4444
        - name: admin
          containerPort: 4445
        resources: *id001
        securityContext: *id002
        volumeMounts:
        - name: config
          mountPath: /etc/hydra
          readOnly: true
        readinessProbe:
          httpGet:
            path: /health/ready
            port: admin
          initialDelaySeconds: 5
          periodSeconds: 5
        livenessProbe:
          httpGet:
            path: /health/alive
            port: admin
          initialDelaySeconds: 20
          periodSeconds: 20
      volumes:
      - name: config
        configMap:
          name: hydra-config
@@ -0,0 +1,16 @@
 apiVersion: external-secrets.io/v1
 kind: ExternalSecret
 metadata:
  name: hydra
  namespace: hydra
 spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: openbao
  target:
    name: hydra
    creationPolicy: Owner
  dataFrom:
  - extract:
      key: k8s/hydra
@@ -0,0 +1,33 @@
 apiVersion: gateway.networking.k8s.io/v1
 kind: HTTPRoute
 metadata:
  name: hydra-public
  namespace: hydra
 spec:
  parentRefs:
  - name: eg
    namespace: envoy-gateway-system
    sectionName: https
  hostnames:
  - hydra.ad.ddupan.top
  rules:
  - backendRefs:
    - name: hydra-public
      port: 4444
 ---
 apiVersion: gateway.networking.k8s.io/v1
 kind: HTTPRoute
 metadata:
  name: hydra-login
  namespace: hydra
 spec:
  parentRefs:
  - name: eg
    namespace: envoy-gateway-system
    sectionName: https
  hostnames:
  - hydra-login.ad.ddupan.top
  rules:
  - backendRefs:
    - name: hydra-login
      port: 8080
@@ -0,0 +1,25 @@
 serve:
  public:
    port: 4444
  admin:
    port: 4445
  tls:
    allow_termination_from:
    - 10.42.0.0/16
  cookies:
    same_site_mode: Lax
 urls:
  self:
    issuer: https://hydra.ad.ddupan.top
    public: https://hydra.ad.ddupan.top
  login: https://hydra-login.ad.ddupan.top/login
  consent: https://hydra-login.ad.ddupan.top/consent
 ttl:
  access_token: 15m
  id_token: 15m
  auth_code: 5m
 log:
  level: info
  leak_sensitive_values: false
 oauth2:
  expose_internal_errors: false
@@ -0,0 +1,15 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
 - namespace.yaml
 - external-secret.yaml
 - deployment.yaml
 - login-deployment.yaml
 - services.yaml
 - httproutes.yaml
 - networkpolicy.yaml
 configMapGenerator:
 - name: hydra-config
  namespace: hydra
  files:
  - hydra.yaml
@@ -0,0 +1 @@
 /login-consent
@@ -0,0 +1,4 @@
 FROM gcr.io/distroless/static-debian12:nonroot@sha256:afa5c872c891853ca7fcf1f12c3edb23f7eeef36189728842dd51042ff57f7ab
 COPY login-consent /login-consent
 USER 65532:65532
 ENTRYPOINT ["/login-consent"]
@@ -0,0 +1,13 @@
 module git.ddupan.top/panxiao81/homelab-infra/apps/hydra/login-consent
 go 1.26.0
 require (
 	github.com/coreos/go-oidc/v3 v3.14.1
 	golang.org/x/oauth2 v0.37.0
 )
 require (
 	github.com/go-jose/go-jose/v4 v4.0.5 // indirect
 	golang.org/x/crypto v0.36.0 // indirect
 )
@@ -0,0 +1,18 @@
 github.com/coreos/go-oidc/v3 v3.14.1 h1:9ePWwfdwC4QKRlCXsJGou56adA/owXczOzwKdOumLqk=
 github.com/coreos/go-oidc/v3 v3.14.1/go.mod h1:HaZ3szPaZ0e4r6ebqvsLWlk2Tn+aejfmrfah6hnSYEU=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/go-jose/go-jose/v4 v4.0.5 h1:M6T8+mKZl/+fNNuFHvGIzDz7BTLQPIounk/b9dw3AaE=
 github.com/go-jose/go-jose/v4 v4.0.5/go.mod h1:s3P1lRrkT8igV8D9OjyL4WRyHvjB6a4JSllnOrmmBOA=
 github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
 github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
 github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
 golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34=
 golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc=
 golang.org/x/oauth2 v0.37.0 h1:JUlcxA8oAtauLfiH8FX2/FkAWHAdi0QtGCGc+hofE98=
 golang.org/x/oauth2 v0.37.0/go.mod h1:IxwZNxUULJmpBFf9K/9NTMSIfZZuvuTy1gGxhigP/58=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
@@ -0,0 +1,285 @@
 // Login/Consent adapter for a single trusted upstream and first-party clients.
 package main
 import (
 	"bytes"
 	"context"
 	"crypto/rand"
 	"crypto/sha256"
 	"crypto/subtle"
 	"encoding/base64"
 	"encoding/hex"
 	"encoding/json"
 	"errors"
 	"fmt"
 	"io"
 	"log"
 	"net/http"
 	"net/url"
 	"os"
 	"strings"
 	"sync"
 	"time"
 	"github.com/coreos/go-oidc/v3/oidc"
 	"golang.org/x/oauth2"
 )
 const cookieName = "__Host-hydra-login"
 type pending struct {
 	Challenge, Nonce, Verifier string
 	Expires                    time.Time
 }
 type claims struct {
 	Username      string   `json:"preferred_username"`
 	Email         string   `json:"email"`
 	EmailVerified bool     `json:"email_verified"`
 	Name          string   `json:"name"`
 	Groups        []string `json:"groups"`
 }
 type flowRequest struct {
 	Client struct {
 		ID string `json:"client_id"`
 	} `json:"client"`
 	Subject  string   `json:"subject"`
 	Scopes   []string `json:"requested_scope"`
 	Audience []string `json:"requested_access_token_audience"`
 	Context  claims   `json:"context"`
 }
 type app struct {
 	admin, public string
 	client        *http.Client
 	oauth         oauth2.Config
 	verifier      *oidc.IDTokenVerifier
 	allowed       map[string]bool
 	mu            sync.Mutex
 	pending       map[string]pending
 }
 func required(key string) string {
 	v := os.Getenv(key)
 	if v == "" {
 		log.Fatalf("missing %s", key)
 	}
 	return v
 }
 func random() string {
 	b := make([]byte, 32)
 	if _, err := rand.Read(b); err != nil {
 		panic(err)
 	}
 	return base64.RawURLEncoding.EncodeToString(b)
 }
 func (a *app) api(ctx context.Context, method, path string, in, out any) error {
 	var body io.Reader
 	if in != nil {
 		b, err := json.Marshal(in)
 		if err != nil {
 			return err
 		}
 		body = bytes.NewReader(b)
 	}
 	req, err := http.NewRequestWithContext(ctx, method, a.admin+path, body)
 	if err != nil {
 		return err
 	}
 	req.Header.Set("Content-Type", "application/json")
 	resp, err := a.client.Do(req)
 	if err != nil {
 		return errors.New("Hydra unavailable")
 	}
 	defer resp.Body.Close()
 	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
 		return fmt.Errorf("Hydra status %d", resp.StatusCode)
 	}
 	if out != nil {
 		return json.NewDecoder(io.LimitReader(resp.Body, 1<<20)).Decode(out)
 	}
 	return nil
 }
 func (a *app) request(r *http.Request, kind, challenge string) (flowRequest, error) {
 	var f flowRequest
 	if challenge == "" || len(challenge) > 8192 {
 		return f, errors.New("missing or invalid challenge")
 	}
 	err := a.api(r.Context(), http.MethodGet, "/admin/oauth2/auth/requests/"+kind+"?"+kind+"_challenge="+url.QueryEscape(challenge), nil, &f)
 	if err != nil {
 		return f, err
 	}
 	if !a.allowed[f.Client.ID] {
 		return f, errors.New("client not allowed")
 	}
 	return f, nil
 }
 func (a *app) accept(w http.ResponseWriter, r *http.Request, kind, challenge string, body any) {
 	var result struct {
 		Redirect string `json:"redirect_to"`
 	}
 	if err := a.api(r.Context(), http.MethodPut, "/admin/oauth2/auth/requests/"+kind+"/accept?"+kind+"_challenge="+url.QueryEscape(challenge), body, &result); err != nil {
 		fail(w, 502)
 		return
 	}
 	// Only Hydra's own authorization endpoint can receive a challenge verifier.
 	u, err := url.Parse(result.Redirect)
 	p, _ := url.Parse(a.public)
 	if err != nil || u.Scheme != p.Scheme || u.Host != p.Host || u.User != nil || u.Path != "/oauth2/auth" {
 		fail(w, 502)
 		return
 	}
 	http.Redirect(w, r, result.Redirect, http.StatusSeeOther)
 }
 func fail(w http.ResponseWriter, status int) { http.Error(w, http.StatusText(status), status) }
 func (a *app) login(w http.ResponseWriter, r *http.Request) {
 	challenge := r.URL.Query().Get("login_challenge")
 	if _, err := a.request(r, "login", challenge); err != nil {
 		fail(w, 403)
 		return
 	}
 	state := random()
 	p := pending{challenge, random(), oauth2.GenerateVerifier(), time.Now().Add(10 * time.Minute)}
 	a.mu.Lock()
 	for k, v := range a.pending {
 		if time.Now().After(v.Expires) {
 			delete(a.pending, k)
 		}
 	}
 	if len(a.pending) >= 1024 {
 		a.mu.Unlock()
 		fail(w, 503)
 		return
 	}
 	a.pending[state] = p
 	a.mu.Unlock()
 	http.SetCookie(w, &http.Cookie{Name: cookieName, Value: state, Path: "/", Secure: true, HttpOnly: true, SameSite: http.SameSiteLaxMode, MaxAge: 600})
 	http.Redirect(w, r, a.oauth.AuthCodeURL(state, oidc.Nonce(p.Nonce), oauth2.S256ChallengeOption(p.Verifier)), http.StatusSeeOther)
 }
 func (a *app) take(r *http.Request) (pending, error) {
 	state := r.URL.Query().Get("state")
 	cookie, err := r.Cookie(cookieName)
 	if err != nil || state == "" || subtle.ConstantTimeCompare([]byte(cookie.Value), []byte(state)) != 1 {
 		return pending{}, errors.New("state mismatch")
 	}
 	a.mu.Lock()
 	defer a.mu.Unlock()
 	p, ok := a.pending[state]
 	delete(a.pending, state)
 	if !ok || time.Now().After(p.Expires) {
 		return pending{}, errors.New("expired or used state")
 	}
 	return p, nil
 }
 func (a *app) callback(w http.ResponseWriter, r *http.Request) {
 	p, err := a.take(r)
 	if err != nil {
 		fail(w, 403)
 		return
 	}
 	http.SetCookie(w, &http.Cookie{Name: cookieName, Path: "/", Secure: true, HttpOnly: true, SameSite: http.SameSiteLaxMode, MaxAge: -1})
 	if r.URL.Query().Get("error") != "" || r.URL.Query().Get("code") == "" {
 		fail(w, 403)
 		return
 	}
 	ctx := oidc.ClientContext(r.Context(), a.client)
 	token, err := a.oauth.Exchange(ctx, r.URL.Query().Get("code"), oauth2.VerifierOption(p.Verifier))
 	if err != nil {
 		fail(w, 502)
 		return
 	}
 	raw, ok := token.Extra("id_token").(string)
 	if !ok {
 		fail(w, 502)
 		return
 	}
 	id, err := a.verifier.Verify(ctx, raw)
 	if err != nil || id.Nonce != p.Nonce || id.Subject == "" {
 		fail(w, 403)
 		return
 	}
 	var c claims
 	if id.Claims(&c) != nil || c.Username == "" || c.Email == "" || !c.EmailVerified {
 		fail(w, 403)
 		return
 	}
 	if _, err := a.request(r, "login", p.Challenge); err != nil {
 		fail(w, 403)
 		return
 	}
 	// Stable identity is tied to the verified upstream issuer+subject, never email.
 	sum := sha256.Sum256([]byte(id.Issuer + "\x00" + id.Subject))
 	a.accept(w, r, "login", p.Challenge, map[string]any{"subject": "human:" + hex.EncodeToString(sum[:]), "remember": false, "context": c})
 }
 func consentSession(f flowRequest) (map[string]any, error) {
 	if !strings.HasPrefix(f.Subject, "human:") || f.Context.Username == "" || f.Context.Email == "" || !f.Context.EmailVerified {
 		return nil, errors.New("invalid identity context")
 	}
 	allowed := map[string]bool{"openid": true, "profile": true, "email": true, "groups": true}
 	session := map[string]any{"principal_type": "human"}
 	for _, scope := range f.Scopes {
 		if !allowed[scope] {
 			return nil, errors.New("scope not allowed")
 		}
 		switch scope {
 		case "profile":
 			session["preferred_username"] = f.Context.Username
 			session["name"] = f.Context.Name
 		case "email":
 			session["email"] = f.Context.Email
 			session["email_verified"] = true
 		case "groups":
 			session["groups"] = f.Context.Groups
 		}
 	}
 	if len(f.Audience) > 0 {
 		return nil, errors.New("access token audience not allowed")
 	}
 	return session, nil
 }
 func (a *app) consent(w http.ResponseWriter, r *http.Request) {
 	challenge := r.URL.Query().Get("consent_challenge")
 	f, err := a.request(r, "consent", challenge)
 	if err != nil {
 		fail(w, 403)
 		return
 	}
 	session, err := consentSession(f)
 	if err != nil {
 		fail(w, 403)
 		return
 	}
 	// Explicit policy for pre-approved first-party clients only; no generic auto-consent.
 	a.accept(w, r, "consent", challenge, map[string]any{"grant_scope": f.Scopes, "remember": false, "session": map[string]any{"id_token": session}})
 }
 func (a *app) handler() http.Handler {
 	mux := http.NewServeMux()
 	mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(200) })
 	mux.HandleFunc("GET /login", a.login)
 	mux.HandleFunc("GET /callback", a.callback)
 	mux.HandleFunc("GET /consent", a.consent)
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		w.Header().Set("Cache-Control", "no-store")
 		w.Header().Set("Referrer-Policy", "no-referrer")
 		w.Header().Set("X-Content-Type-Options", "nosniff")
 		w.Header().Set("Content-Security-Policy", "default-src 'none'; frame-ancestors 'none'")
 		mux.ServeHTTP(w, r)
 	})
 }
 func main() {
 	client := &http.Client{Timeout: 15 * time.Second, CheckRedirect: func(r *http.Request, via []*http.Request) error { return http.ErrUseLastResponse }}
 	issuer := required("UPSTREAM_ISSUER")
 	ctx := oidc.ClientContext(context.Background(), client)
 	provider, err := oidc.NewProvider(ctx, issuer)
 	if err != nil {
 		log.Fatal("upstream discovery failed")
 	}
 	clientID := required("UPSTREAM_CLIENT_ID")
 	a := &app{admin: required("HYDRA_ADMIN_URL"), public: required("HYDRA_PUBLIC_URL"), client: client, allowed: map[string]bool{}, pending: map[string]pending{},
 		oauth:    oauth2.Config{ClientID: clientID, ClientSecret: required("UPSTREAM_CLIENT_SECRET"), RedirectURL: required("CALLBACK_URL"), Endpoint: provider.Endpoint(), Scopes: []string{"openid", "profile", "email", "groups"}},
 		verifier: provider.Verifier(&oidc.Config{ClientID: clientID})}
 	for _, id := range strings.Split(required("ALLOWED_CLIENTS"), ",") {
 		a.allowed[id] = true
 	}
 	s := http.Server{Addr: ":8080", Handler: a.handler(), ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 20 * time.Second, WriteTimeout: 45 * time.Second, IdleTimeout: 60 * time.Second, MaxHeaderBytes: 16384}
 	log.Print("login/consent adapter listening on :8080")
 	log.Fatal(s.ListenAndServe())
 }
@@ -0,0 +1,107 @@
 package main
 import (
 	"encoding/json"
 	"net/http"
 	"net/http/httptest"
 	"net/url"
 	"strings"
 	"testing"
 	"time"
 	"golang.org/x/oauth2"
 )
 func TestStateBoundToCookieSingleUseAndExpiry(t *testing.T) {
 	a := &app{pending: map[string]pending{"valid": {Challenge: "challenge", Expires: time.Now().Add(time.Minute)}, "expired": {Expires: time.Now().Add(-time.Minute)}}}
 	request := func(state, cookie string) *http.Request {
 		r := httptest.NewRequest("GET", "https://login.example/callback?state="+state, nil)
 		if cookie != "" {
 			r.AddCookie(&http.Cookie{Name: cookieName, Value: cookie})
 		}
 		return r
 	}
 	for _, r := range []*http.Request{request("valid", ""), request("valid", "other"), request("expired", "expired")} {
 		if _, err := a.take(r); err == nil {
 			t.Fatal("invalid state accepted")
 		}
 	}
 	if p, err := a.take(request("valid", "valid")); err != nil || p.Challenge != "challenge" {
 		t.Fatal("valid state rejected")
 	}
 	if _, err := a.take(request("valid", "valid")); err == nil {
 		t.Fatal("replayed state accepted")
 	}
 }
 func TestConsentRejectsPrivilegeExpansionAndFiltersClaims(t *testing.T) {
 	f := flowRequest{Subject: "human:known", Scopes: []string{"openid", "email"}, Context: claims{Username: "alice", Email: "[email protected]", EmailVerified: true, Groups: []string{"operators"}}}
 	s, err := consentSession(f)
 	if err != nil {
 		t.Fatal(err)
 	}
 	if _, ok := s["groups"]; ok {
 		t.Fatal("groups leaked without scope")
 	}
 	if _, ok := s["preferred_username"]; ok {
 		t.Fatal("profile leaked without scope")
 	}
 	for _, scope := range []string{"admin", "offline_access", "unknown"} {
 		bad := f
 		bad.Scopes = append([]string{"openid"}, scope)
 		if _, err := consentSession(bad); err == nil {
 			t.Fatalf("accepted %s", scope)
 		}
 	}
 	f.Audience = []string{"other-service"}
 	if _, err := consentSession(f); err == nil {
 		t.Fatal("unexpected audience accepted")
 	}
 	f.Audience = nil
 	f.Context.EmailVerified = false
 	if _, err := consentSession(f); err == nil {
 		t.Fatal("unverified email accepted")
 	}
 }
 func TestLoginValidatesClientAndUsesPKCEAndNonce(t *testing.T) {
 	admin := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		json.NewEncoder(w).Encode(map[string]any{"client": map[string]string{"client_id": r.URL.Query().Get("login_challenge")}})
 	}))
 	defer admin.Close()
 	a := &app{admin: admin.URL, client: admin.Client(), allowed: map[string]bool{"gitea": true}, pending: map[string]pending{}, oauth: oauth2.Config{ClientID: "hydra-login", RedirectURL: "https://login.example/callback", Endpoint: oauth2.Endpoint{AuthURL: "https://upstream.example/authorize"}}}
 	w := httptest.NewRecorder()
 	a.handler().ServeHTTP(w, httptest.NewRequest("GET", "https://login.example/login?login_challenge=rogue", nil))
 	if w.Code != 403 {
 		t.Fatal("unknown client accepted")
 	}
 	w = httptest.NewRecorder()
 	a.handler().ServeHTTP(w, httptest.NewRequest("GET", "https://login.example/login?login_challenge=gitea", nil))
 	if w.Code != 303 {
 		t.Fatalf("status %d", w.Code)
 	}
 	u, _ := url.Parse(w.Header().Get("Location"))
 	q := u.Query()
 	if q.Get("code_challenge_method") != "S256" || q.Get("code_challenge") == "" || q.Get("nonce") == "" || q.Get("state") == "" {
 		t.Fatal("missing protocol binding")
 	}
 	cookies := w.Result().Cookies()
 	if len(cookies) != 1 || !cookies[0].Secure || !cookies[0].HttpOnly || cookies[0].SameSite != http.SameSiteLaxMode || cookies[0].Value != q.Get("state") {
 		t.Fatal("unsafe cookie")
 	}
 	if w.Header().Get("Cache-Control") != "no-store" {
 		t.Fatal("missing cache protection")
 	}
 }
 func TestHydraRedirectCannotLeaveTrustedOrigin(t *testing.T) {
 	for _, target := range []string{"https://evil.example/oauth2/auth", "https://[email protected]/oauth2/auth", "https://hydra.example/other"} {
 		admin := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 			json.NewEncoder(w).Encode(map[string]string{"redirect_to": target})
 		}))
 		a := &app{admin: admin.URL, public: "https://hydra.example", client: admin.Client()}
 		w := httptest.NewRecorder()
 		a.accept(w, httptest.NewRequest("GET", "https://login.example/login", nil), "login", "challenge", map[string]string{"subject": "human:test"})
 		if w.Code != 502 || strings.Contains(w.Header().Get("Location"), "evil") {
 			t.Fatal("untrusted redirect accepted")
 		}
 		admin.Close()
 	}
 }
@@ -0,0 +1,69 @@
 apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: hydra-login
  namespace: hydra
 spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: hydra-login
  template:
    metadata:
      labels:
        app: hydra-login
    spec:
      automountServiceAccountToken: false
      securityContext:
        runAsNonRoot: true
        runAsUser: 65532
        runAsGroup: 65532
        seccompProfile:
          type: RuntimeDefault
      containers:
      - name: login-consent
        image: zot.ad.ddupan.top/iam/oidc-login-consent@sha256:fede9b9e93c457c4b7a8a6022d9df86ff5d5900d3f6b6c4f3439851a0ae1e944
        env:
        - name: HYDRA_ADMIN_URL
          value: http://hydra-admin.hydra.svc.cluster.local:4445
        - name: HYDRA_PUBLIC_URL
          value: https://hydra.ad.ddupan.top
        - name: UPSTREAM_ISSUER
          value: https://auth.ddupan.top
        - name: UPSTREAM_CLIENT_ID
          value: hydra-login
        - name: CALLBACK_URL
          value: https://hydra-login.ad.ddupan.top/callback
        - name: ALLOWED_CLIENTS
          value: gitea
        - name: UPSTREAM_CLIENT_SECRET
          valueFrom:
            secretKeyRef:
              name: hydra
              key: upstream_client_secret
        ports:
        - name: http
          containerPort: 8080
        resources:
          requests:
            cpu: 50m
            memory: 64Mi
          limits:
            memory: 256Mi
        securityContext:
          allowPrivilegeEscalation: false
          readOnlyRootFilesystem: true
          capabilities:
            drop:
            - ALL
        readinessProbe:
          httpGet:
            path: /healthz
            port: http
        livenessProbe:
          httpGet:
            path: /healthz
            port: http
          initialDelaySeconds: 10
@@ -0,0 +1,6 @@
 apiVersion: v1
 kind: Namespace
 metadata:
  name: hydra
  labels:
    pod-security.kubernetes.io/enforce: restricted
@@ -0,0 +1,46 @@
 apiVersion: networking.k8s.io/v1
 kind: NetworkPolicy
 metadata:
  name: hydra
  namespace: hydra
 spec:
  podSelector:
    matchLabels:
      app: hydra
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: envoy-gateway-system
    ports:
    - port: 4444
      protocol: TCP
  - from:
    - podSelector:
        matchLabels:
          app: hydra-login
    ports:
    - port: 4445
      protocol: TCP
 ---
 apiVersion: networking.k8s.io/v1
 kind: NetworkPolicy
 metadata:
  name: hydra-login
  namespace: hydra
 spec:
  podSelector:
    matchLabels:
      app: hydra-login
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: envoy-gateway-system
    ports:
    - port: 8080
      protocol: TCP
@@ -0,0 +1,35 @@
 apiVersion: v1
 kind: Service
 metadata:
  name: hydra-public
  namespace: hydra
 spec:
  selector:
    app: hydra
  ports:
  - port: 4444
    targetPort: 4444
 ---
 apiVersion: v1
 kind: Service
 metadata:
  name: hydra-admin
  namespace: hydra
 spec:
  selector:
    app: hydra
  ports:
  - port: 4445
    targetPort: 4445
 ---
 apiVersion: v1
 kind: Service
 metadata:
  name: hydra-login
  namespace: hydra
 spec:
  selector:
    app: hydra-login
  ports:
  - port: 8080
    targetPort: 8080
@@ -0,0 +1,157 @@
 # Nexus Repository POC
 本目录声明一个 Nexus Repository Community Edition POC，用来验证一次性 CI runner 通过
 网络服务复用 Ansible Galaxy、Go Modules 与 OCI/BuildKit 缓存。Nexus 固定为 `3.96.1`，
 镜像固定到官方 multi-arch index digest；LAN 入口为
 `https://nexus.ad.ddupan.top`。
 ## POC 边界
 - 单副本 Deployment，`Recreate` 更新，避免一个 RWO 卷被两个 Pod 同时挂载。
 - `/nexus-data` 使用 `localpv-zfs-ceph` 上的 50 GiB RWO PVC。
 - 资源预算为 250m/2 GiB request、2 CPU/4 GiB limit；JVM heap 上限 2 GiB。
 - 使用容器默认的 embedded H2。它只用于 POC；正式接管 OCI 制品前必须迁移到外部
  PostgreSQL，并验证备份恢复。
 - 入口只在 LAN wildcard Gateway 上发布，不创建公网 DNS 或 Cloudflare route。
 - 不套 Authelia forward-auth；它会破坏 Go、Ansible 与 OCI 非浏览器客户端。
 - 现有 zot 保持不变。Nexus 完成 OCI、BuildKit cache 和恢复验收前不得迁移或删除 zot。
 Terraform provider 创建 `ansible-public`、`go-public`、最小匿名权限与 OCI Bearer Token
 Realm。Nexus 3.94 才加入的原生 OCI repository 已有 REST API，但当前锁定的 community
 provider 尚未暴露 OCI resource；`terraform/reconcile-oci.sh` 因此根据 3.96.1 实例 Swagger
 固定的 JSON schema，幂等调和 `oci-hosted`、`oci-proxy` 与 `oci-public`。不得绕过该入口在
 UI 中创建无人管理的长期 repository。
 ## 部署
 Flux 从 `clusters/homelab/apps/nexus.yaml` 协调本目录，并依赖 Envoy Gateway 与 OpenEBS。
 合并前只渲染配置，不直接 apply：
 ```bash
 kubectl kustomize apps/nexus
 ```
 合并并由 Flux 部署后检查：
 ```bash
 kubectl -n flux-system get kustomization nexus
 kubectl -n nexus get pod,pvc,service,httproute
 kubectl -n nexus logs deployment/nexus --tail=100
 ```
 启动可能需要数分钟，startup probe 允许最多十分钟。不要因初次启动较慢反复删除 Pod；
 先确认 PVC 已 Bound、Pod 没有 OOM，以及日志仍在推进。
 DNS 期望状态已加入 `infrastructure/dns/records.yml`，需从 Samba AD Ansible root 以
 `--check --diff` 核对后再按其 README 应用 DNS tag。没有 DNS 时可先用 port-forward
 验证应用，但不能据此宣称 Gateway 路径已通过。
 ## 首次初始化与 Terraform
 初始管理员密码生成在 PVC 的 `/nexus-data/admin.password`。只在交互式终端中读取并立即
 完成首次密码轮换；不得把密码复制进 shell tracing、工单、Git 或命令参数。随后将
 Terraform 管理账号的凭据存入 OpenBao，由 CI 通过 Terraform input variable 注入以下
 环境变量：
 ```text
 TF_VAR_nexus_url=https://nexus.ad.ddupan.top
 TF_VAR_nexus_username=admin
 TF_VAR_nexus_password=<OpenBao kv/infra/nexus 的 admin_password 字段>
 ```
 `terraform/` 使用 `sonatype-nexus-community/sonatyperepo` 1.17.0，当前声明：
 - `ansible-galaxy-proxy` → `https://galaxy.ansible.com`
 - `ansible-public` group
 - `go-proxy` → `https://proxy.golang.org`
 - `go-public` group
 provider credential 不写入 HCL 或 tfvars。正式 apply 前还必须为这个独立 Terraform root
 配置远端 backend；本地 state 只允许用于可丢弃的 POC，不提交。验证命令：
 ```bash
 terraform -chdir=apps/nexus/terraform init -backend=false
 terraform -chdir=apps/nexus/terraform validate
 ```
 先以 `--check` 查看 OCI repository 漂移，再明确 apply；脚本只从上述环境变量取得凭据，
 用临时 `0600` netrc 调用 REST API，退出时删除：
 ```bash
 apps/nexus/terraform/reconcile-oci.sh --check
 apps/nexus/terraform/reconcile-oci.sh --apply
 ```
 Terraform 同时把内置 `anonymous` 用户从默认的全仓库 `nx-anonymous` 角色收窄到
 `ansible-public`、其返回制品 URL 使用的 `ansible-galaxy-proxy`、`go-public`，以及
 `oci-public`/`oci-proxy` 的 `browse/read` 权限。`oci-hosted` 不向匿名用户开放。首次接管
 已有实例时先执行
 `terraform import sonatyperepo_user.anonymous anonymous,default`，再 apply；不要先启用默认的
 全仓库匿名读取。
 2026-09-20 的 POC 现场验收已确认：Flux 与 Pod Ready、PVC Bound、HTTPRoute 通过 HTTPS
 返回 Nexus 状态 200，Samba DNS 已幂等收敛。全新客户端目录通过匿名入口下载
 `community.general:11.2.0` 时冷缓存为 8.49 秒、热缓存为 1.89 秒，两次 tarball SHA-256
 一致；`golang.org/x/[email protected]` 为 2.92 秒与 1.51 秒。
 ## 客户端验收
 先验证冷缓存，再原样重复命令验证热缓存；记录 Nexus 请求、上游流量和耗时，不只观察
 命令成功。Ansible 配置的 URL 必须以 `/` 结尾：
 ```ini
 [galaxy]
 server_list = nexus
 [galaxy_server.nexus]
 url = https://nexus.ad.ddupan.top/repository/ansible-public/
 ```
 ```bash
 ansible-galaxy collection install -r collections/requirements.yml \
  -p .ansible/collections
 ```
 Go POC 使用：
 ```bash
 GOPROXY=https://nexus.ad.ddupan.top/repository/go-public/ go mod download
 ```
 私有 `git.ddupan.top/*` module 的 `GOPRIVATE`、凭据与是否允许 `direct` fallback 在实际
 workflow 中单独决定；不要让私有 module path 意外发往公共 proxy。
 OCI 使用 path-based routing：匿名公共拉取地址形如
 `nexus.ad.ddupan.top/oci-public/library/alpine:3.22`，认证写入地址形如
 `nexus.ad.ddupan.top/oci-hosted/<namespace>/<image>:<tag>`。2026-09-20 现场验收结果：
 - `oci-public` 匿名代理拉取 Alpine 冷缓存 4.75 秒、热缓存 0.80 秒，digest 一致；
 - `oci-hosted` 认证 push/pull 成功，匿名 pull 返回 401；
 - amd64/arm64 OCI image index push 成功，两个平台 manifest 可见；
 - Helm chart push/pull digest 与本地 tarball SHA-256 一致；
 - Cosign 3.1.3 使用一次性密钥签名并验证成功，OCI 1.1 referrers API 返回一个
  `application/vnd.dev.sigstore.bundle.v0.3+json` artifact；
 - BuildKit `registry` cache 以 `mode=max` 导出成功，销毁首个 builder 后由新 builder 导入，
  两个 `RUN` step 均明确命中 `CACHED`。
 本机安装的测试客户端包括 `/usr/local/bin/cosign` 3.1.3；安装时核对官方 Linux amd64
 binary SHA-256 `4629c757b7618056f8ddd7e2625ae9fdd94c0372a65049520bc7d9df9efc7f71`。
 上述结果仍不代表备份恢复、外部 PostgreSQL 或正式 publisher service account 已完成；
 这些项目通过前不得迁移或删除 zot。
 ## 数据与恢复
 POC 的数据库、配置、blob、初始管理员状态都位于 `nexus-data` PVC。删除 Deployment
 不会删除 PVC；删除 PVC 会永久删除整个 POC。当前没有独立备份，不能将它用于唯一副本的
 正式制品。
 恢复验证至少包括：停止写入、取得一致备份、在独立 PVC/实例恢复、登录、列出 repository、
 拉取已缓存的 Ansible/Go 制品，并核对 OCI digest/referrers。正式化时再把数据库迁移至
 外部 PostgreSQL，并分别定义数据库与 blob 的备份、恢复顺序和 RPO。
 参考：
 - [Nexus OCI repositories](https://help.sonatype.com/en/oci-repositories.html)
 - [Nexus Ansible repositories](https://help.sonatype.com/en/ansible-repositories.html)
 - [Nexus Go repositories](https://help.sonatype.com/en/go-repositories.html)
 - [官方容器镜像](https://hub.docker.com/r/sonatype/nexus3)
@@ -0,0 +1,82 @@
 apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: nexus
  namespace: nexus
  labels:
    app.kubernetes.io/name: nexus
 spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app.kubernetes.io/name: nexus
  template:
    metadata:
      labels:
        app.kubernetes.io/name: nexus
    spec:
      automountServiceAccountToken: false
      securityContext:
        fsGroup: 200
        fsGroupChangePolicy: OnRootMismatch
        runAsGroup: 200
        runAsNonRoot: true
        runAsUser: 200
        seccompProfile:
          type: RuntimeDefault
      terminationGracePeriodSeconds: 120
      containers:
        - name: nexus
          image: docker.io/sonatype/nexus3:3.96.1@sha256:56142f13432cf072e017aebb2025f201e42ae36ff40bb82618c702504c61f7dd
          imagePullPolicy: IfNotPresent
          env:
            - name: INSTALL4J_ADD_VM_PARAMS
              value: >-
                -Xms1024m -Xmx2048m -XX:MaxDirectMemorySize=1024m
                -Djava.util.prefs.userRoot=/nexus-data/javaprefs
          ports:
            - name: http
              containerPort: 8081
              protocol: TCP
          resources:
            requests:
              cpu: 250m
              memory: 2Gi
            limits:
              cpu: "2"
              memory: 4Gi
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
          startupProbe:
            httpGet:
              path: /service/rest/v1/status
              port: http
            failureThreshold: 60
            periodSeconds: 10
            timeoutSeconds: 5
          readinessProbe:
            httpGet:
              path: /service/rest/v1/status
              port: http
            failureThreshold: 6
            periodSeconds: 10
            timeoutSeconds: 5
          livenessProbe:
            httpGet:
              path: /service/rest/v1/status
              port: http
            failureThreshold: 6
            periodSeconds: 30
            timeoutSeconds: 5
          volumeMounts:
            - name: data
              mountPath: /nexus-data
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: nexus-data
@@ -0,0 +1,16 @@
 apiVersion: gateway.networking.k8s.io/v1
 kind: HTTPRoute
 metadata:
  name: nexus
  namespace: nexus
 spec:
  parentRefs:
    - name: eg
      namespace: envoy-gateway-system
      sectionName: https
  hostnames:
    - nexus.ad.ddupan.top
  rules:
    - backendRefs:
        - name: nexus
          port: 8081
@@ -0,0 +1,9 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 resources:
  - namespace.yaml
  - pvc.yaml
  - deployment.yaml
  - service.yaml
  - httproute.yaml
  - networkpolicy.yaml
@@ -0,0 +1,8 @@
 apiVersion: v1
 kind: Namespace
 metadata:
  name: nexus
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted
@@ -0,0 +1,23 @@
 apiVersion: networking.k8s.io/v1
 kind: NetworkPolicy
 metadata:
  name: nexus-ingress
  namespace: nexus
 spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: nexus
  policyTypes:
    - Ingress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: envoy-gateway-system
          podSelector:
            matchLabels:
              gateway.envoyproxy.io/owning-gateway-name: eg
              gateway.envoyproxy.io/owning-gateway-namespace: envoy-gateway-system
      ports:
        - protocol: TCP
          port: 8081
@@ -0,0 +1,12 @@
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
  name: nexus-data
  namespace: nexus
 spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi
  storageClassName: localpv-zfs-ceph
@@ -0,0 +1,14 @@
 apiVersion: v1
 kind: Service
 metadata:
  name: nexus
  namespace: nexus
 spec:
  type: ClusterIP
  selector:
    app.kubernetes.io/name: nexus
  ports:
    - name: http
      port: 8081
      protocol: TCP
      targetPort: http
@@ -0,0 +1,6 @@
 .terraform/
 *.tfstate
 *.tfstate.*
 *.tfplan
 crash.log
 crash.*.log
@@ -0,0 +1,24 @@
 # This file is maintained automatically by "terraform init".
 # Manual edits may be lost in future updates.
 provider "registry.terraform.io/sonatype-nexus-community/sonatyperepo" {
  version     = "1.17.0"
  constraints = "1.17.0"
  hashes = [
    "h1:uKhvhhhI7B+HBsh0zq/ybqKt+EnOGyI6rjcRCtj79ZA=",
    "zh:0dde99e7b343fa01f8eefc378171fb8621bedb20f59157d6cc8e3d46c738105f",
    "zh:3315929df254a3a6ac27c8c846c2006f7d2a91fadc014351bc4d617f948e5bf9",
    "zh:36be5a455af3ce4e187de26753de63e78c1ee9a32dba0135c6cf96a6c1fff25f",
    "zh:3f73f7ff57b8c339a7c7ac37653e2dc0b2dd9dcc3f3a538788e7e3ac838337b2",
    "zh:40286ecca4c22ab7ae90618ac6d2743f5055199dac81cf5204a4a397c784d439",
    "zh:4d24e5c0195fb3155b1967583ee64cfeda402d7cc7f3c73369438f6c69f4245b",
    "zh:828a9d7aceaac36af7f9c07af43ec8d20a89148780645d170ffb1c68b2da792d",
    "zh:a5ab04de3fe626ec57c832618c6f990abd6610f81e132621651e0b180b970cff",
    "zh:a959fa6090a8c0f53739879184e7346423494aee598003df0d1ab4a22b2eee91",
    "zh:bdda26c2f03f918bbe59e75abea44868fafda019c3a543725331195df126350b",
    "zh:d8048e149ee97ba62971e6a79355d59887bc6d10fcf72cc2feff3d0a2582670c",
    "zh:dd36f9988af4e1ca5b1ca7b7bb6f658df9a220dfcda7fec7392fedfe9064f652",
    "zh:dda2688d46c7e539fe97e8fe9d3ec81fb364170e018d9c6a681364c8955d4e9d",
    "zh:e6b519afe2dea1c0434f766eb6bc9ba78cc5b6ef2c311c2ca3c65cb24744f31f",
  ]
 }
@@ -0,0 +1,17 @@
 {
  "name": "oci-hosted",
  "online": true,
  "storage": {
    "blobStoreName": "default",
    "strictContentTypeValidation": true,
    "writePolicy": "ALLOW",
    "latestPolicy": false
  },
  "oci": {
    "v1Enabled": false,
    "forceBasicAuth": false,
    "pathEnabled": true
  },
  "component": { "proprietaryComponents": false },
  "cosign": { "enforcement": "NONE" }
 }
@@ -0,0 +1,27 @@
 {
  "name": "oci-proxy",
  "online": true,
  "storage": {
    "blobStoreName": "default",
    "strictContentTypeValidation": true
  },
  "oci": {
    "v1Enabled": false,
    "forceBasicAuth": false,
    "pathEnabled": true
  },
  "ociProxy": {
    "indexType": "HUB",
    "cacheForeignLayers": false,
    "foreignLayerUrlWhitelist": []
  },
  "proxy": {
    "remoteUrl": "https://registry-1.docker.io",
    "contentMaxAge": 1440,
    "metadataMaxAge": 60,
    "preserveEncodedCharacters": false
  },
  "negativeCache": { "enabled": true, "timeToLive": 60 },
  "httpClient": { "blocked": false, "autoBlock": true },
  "cosign": { "enforcement": "NONE" }
 }
@@ -0,0 +1,15 @@
 {
  "name": "oci-public",
  "online": true,
  "storage": {
    "blobStoreName": "default",
    "strictContentTypeValidation": true
  },
  "group": { "memberNames": ["oci-proxy"] },
  "oci": {
    "v1Enabled": false,
    "forceBasicAuth": false,
    "pathEnabled": true
  },
  "cosign": { "enforcement": "NONE" }
 }
@@ -0,0 +1,80 @@
 #!/usr/bin/env bash
 set -euo pipefail
 mode="${1:---check}"
 case "$mode" in
  --check | --apply) ;;
  *) echo "usage: $0 [--check|--apply]" >&2; exit 2 ;;
 esac
 : "${TF_VAR_nexus_url:?set TF_VAR_nexus_url}"
 : "${TF_VAR_nexus_username:?set TF_VAR_nexus_username}"
 : "${TF_VAR_nexus_password:?set TF_VAR_nexus_password}"
 script_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
 auth_file="$(mktemp /tmp/nexus-oci-auth.XXXXXX)"
 trap 'rm -f -- "$auth_file"' EXIT
 chmod 0600 "$auth_file"
 printf 'machine %s\nlogin %s\npassword %s\n' \
  "${TF_VAR_nexus_url#*://}" "$TF_VAR_nexus_username" \
  "$TF_VAR_nexus_password" >"$auth_file"
 drift=0
 for entry in \
  "hosted:$script_dir/oci/oci-hosted.json" \
  "proxy:$script_dir/oci/oci-proxy.json" \
  "group:$script_dir/oci/oci-public.json"; do
  repository_type="${entry%%:*}"
  desired_file="${entry#*:}"
  repository_name="$(jq -er '.name' "$desired_file")"
  endpoint="$TF_VAR_nexus_url/service/rest/v1/repositories/oci/$repository_type"
  current_file="$(mktemp /tmp/nexus-oci-current.XXXXXX)"
  status="$(curl --silent --show-error --netrc-file "$auth_file" \
    --output "$current_file" --write-out '%{http_code}' \
    "$endpoint/$repository_name")"
  if [[ "$status" == 404 ]]; then
    drift=1
    if [[ "$mode" == --apply ]]; then
      curl --fail --silent --show-error --netrc-file "$auth_file" \
        --header 'Content-Type: application/json' \
        --data-binary "@$desired_file" "$endpoint"
      echo "created $repository_name"
    else
      echo "missing $repository_name" >&2
    fi
  elif [[ "$status" == 200 ]]; then
    if jq -e --slurpfile desired "$desired_file" '
      def subset($actual; $wanted):
        if ($wanted | type) == "object" then
          all($wanted | keys[];
            ($actual[.] != null) and subset($actual[.]; $wanted[.]))
        else
          $actual == $wanted
        end;
      subset(.; $desired[0])
    ' "$current_file" >/dev/null; then
      echo "in sync $repository_name"
    else
      drift=1
      if [[ "$mode" == --apply ]]; then
        curl --fail --silent --show-error --netrc-file "$auth_file" \
          --request PUT --header 'Content-Type: application/json' \
          --data-binary "@$desired_file" "$endpoint/$repository_name"
        echo "updated $repository_name"
      else
        echo "drifted $repository_name" >&2
      fi
    fi
  else
    cat "$current_file" >&2
    echo "unexpected HTTP $status for $repository_name" >&2
    exit 1
  fi
  rm -f -- "$current_file"
 done
 if [[ "$mode" == --check && "$drift" -ne 0 ]]; then
  exit 1
 fi
@@ -0,0 +1,64 @@
 locals {
  proxy_http_client = {
    auto_block = true
    blocked    = false
  }
  proxy_negative_cache = {
    enabled      = true
    time_to_live = 60
  }
  repository_storage = {
    blob_store_name                = "default"
    strict_content_type_validation = true
  }
 }
 resource "sonatyperepo_repository_ansiblegalaxy_proxy" "galaxy" {
  name   = "ansible-galaxy-proxy"
  online = true
  http_client    = local.proxy_http_client
  negative_cache = local.proxy_negative_cache
  proxy = {
    remote_url       = "https://galaxy.ansible.com"
    content_max_age  = 1440
    metadata_max_age = 60
  }
  storage = local.repository_storage
 }
 resource "sonatyperepo_repository_ansiblegalaxy_group" "public" {
  name   = "ansible-public"
  online = true
  group = {
    member_names = [sonatyperepo_repository_ansiblegalaxy_proxy.galaxy.name]
  }
  storage = local.repository_storage
 }
 resource "sonatyperepo_repository_go_proxy" "public" {
  name   = "go-proxy"
  online = true
  http_client    = local.proxy_http_client
  negative_cache = local.proxy_negative_cache
  proxy = {
    remote_url       = "https://proxy.golang.org"
    content_max_age  = 1440
    metadata_max_age = 60
  }
  storage = local.repository_storage
 }
 resource "sonatyperepo_repository_go_group" "public" {
  name   = "go-public"
  online = true
  group = {
    member_names = [sonatyperepo_repository_go_proxy.public.name]
  }
  storage = local.repository_storage
 }
@@ -0,0 +1,75 @@
 resource "sonatyperepo_privilege_repository_view" "anonymous_ansible" {
  name        = "ci-anonymous-ansible-read"
  description = "Anonymous read access to the Ansible Galaxy group"
  actions     = ["BROWSE", "READ"]
  format      = "ansiblegalaxy"
  repository  = sonatyperepo_repository_ansiblegalaxy_group.public.name
 }
 resource "sonatyperepo_privilege_repository_view" "anonymous_ansible_proxy" {
  name        = "ci-anonymous-ansible-proxy-read"
  description = "Anonymous artifact read access to the Ansible Galaxy proxy"
  actions     = ["BROWSE", "READ"]
  format      = "ansiblegalaxy"
  repository  = sonatyperepo_repository_ansiblegalaxy_proxy.galaxy.name
 }
 resource "sonatyperepo_privilege_repository_view" "anonymous_go" {
  name        = "ci-anonymous-go-read"
  description = "Anonymous read access to the Go module group"
  actions     = ["BROWSE", "READ"]
  format      = "go"
  repository  = sonatyperepo_repository_go_group.public.name
 }
 resource "sonatyperepo_privilege_repository_view" "anonymous_oci_public" {
  name        = "ci-anonymous-oci-public-read"
  description = "Anonymous read access to the public OCI group"
  actions     = ["BROWSE", "READ"]
  format      = "oci"
  repository  = "oci-public"
 }
 resource "sonatyperepo_privilege_repository_view" "anonymous_oci_proxy" {
  name        = "ci-anonymous-oci-proxy-read"
  description = "Anonymous read access to the OCI proxy member"
  actions     = ["BROWSE", "READ"]
  format      = "oci"
  repository  = "oci-proxy"
 }
 resource "sonatyperepo_role" "anonymous_ci" {
  id          = "ci-anonymous-read"
  name        = "CI anonymous read"
  description = "Read-only access to public CI dependency proxy groups"
  privileges = [
    sonatyperepo_privilege_repository_view.anonymous_ansible.name,
    sonatyperepo_privilege_repository_view.anonymous_ansible_proxy.name,
    sonatyperepo_privilege_repository_view.anonymous_go.name,
    sonatyperepo_privilege_repository_view.anonymous_oci_public.name,
    sonatyperepo_privilege_repository_view.anonymous_oci_proxy.name,
  ]
  roles = []
 }
 resource "sonatyperepo_user" "anonymous" {
  user_id       = "anonymous"
  first_name    = "Anonymous"
  last_name     = "User"
  email_address = "[email protected]"
  status        = "active"
  roles         = [sonatyperepo_role.anonymous_ci.id]
 }
 resource "sonatyperepo_system_anonymous_access" "ci" {
  enabled    = true
  user_id    = sonatyperepo_user.anonymous.user_id
  realm_name = "NexusAuthorizingRealm"
 }
 resource "sonatyperepo_security_realms" "active" {
  active = [
    "NexusAuthenticatingRealm",
    "OciBearerToken",
  ]
 }
@@ -0,0 +1,33 @@
 terraform {
  required_version = ">= 1.11.0"
  required_providers {
    sonatyperepo = {
      source  = "sonatype-nexus-community/sonatyperepo"
      version = "1.17.0"
    }
  }
 }
 provider "sonatyperepo" {
  url      = var.nexus_url
  username = var.nexus_username
  password = var.nexus_password
 }
 variable "nexus_url" {
  description = "Nexus Repository base URL"
  type        = string
 }
 variable "nexus_username" {
  description = "Nexus Terraform management username"
  type        = string
  sensitive   = true
 }
 variable "nexus_password" {
  description = "Nexus Terraform management password"
  type        = string
  sensitive   = true
 }
@@ -70,11 +70,11 @@ configFiles:
        },
        "accessControl": {
          "repositories": {
-            "panxiao81/gitea-dynamic-runner-controller": {
+            "panxiao81/backstage": {
              "policies": [
                {
                  "users": [
-                    "spiffe://ddupan.top/ci/panxiao81/gitea-dynamic-runner/publish-images",
+                    "spiffe://ddupan.top/ci/panxiao81/backstage/image",
                    "spiffe://ddupan.top/dev/panxiao81"
                  ],
                  "actions": [
@@ -88,18 +88,42 @@ configFiles:
                "read"
              ]
            },
-            "panxiao81/gitea-dynamic-runner-runner": {
+            "panxiao81/gitea-dynamic-runner-controller": {
              "policies": [
                {
                  "users": [
-                    "spiffe://ddupan.top/ci/panxiao81/gitea-dynamic-runner/publish-images",
+                    "spiffe://ddupan.top/ci/panxiao81/gitea-dynamic-runner/publish-images"
                    "spiffe://ddupan.top/dev/panxiao81"
                  ],
                  "actions": [
                    "read",
                    "create",
                    "update"
                  ]
                },
                {
                  "users": ["spiffe://ddupan.top/dev/panxiao81"],
                  "actions": ["read", "create", "update", "delete"]
                }
              ],
              "defaultPolicy": [
                "read"
              ]
            },
            "panxiao81/gitea-dynamic-runner-runner": {
              "policies": [
                {
                  "users": [
                    "spiffe://ddupan.top/ci/panxiao81/gitea-dynamic-runner/publish-images"
                  ],
                  "actions": [
                    "read",
                    "create",
                    "update"
                  ]
                },
                {
                  "users": ["spiffe://ddupan.top/dev/panxiao81"],
                  "actions": ["read", "create", "update", "delete"]
                }
              ],
              "defaultPolicy": [
@@ -53,4 +53,6 @@ sudo k3s kubectl -n flux-system get gitrepositories,kustomizations
 - SPIRE 已按 hardened chart 内部 fork `0.30.2-ddupan.1`（基于上游 `0.30.2`，SPIRE
  `1.15.3`）声明，使用共享
  PostgreSQL 与独立 signing-key PVC；首次上线和 OpenBao JWT-SVID PoC 尚待合并后验证；
 - Nexus Repository CE POC 已加入 GitOps 声明，计划验证 Ansible Galaxy、Go Modules 与
  OCI/BuildKit 缓存；尚未部署或完成现场验收，现有 zot 保持不变；
 - root Kustomization 与所有 brownfield 子 Kustomization 继续保持 `prune: false`。
@@ -0,0 +1,22 @@
 apiVersion: kustomize.toolkit.fluxcd.io/v1
 kind: Kustomization
 metadata:
  name: backstage
  namespace: flux-system
 spec:
  dependsOn:
    - name: envoy-gateway
    - name: external-secrets
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: backstage
      namespace: backstage
  interval: 10m
  path: ./apps/backstage
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
  timeout: 5m
  wait: true
@@ -6,11 +6,12 @@ metadata:
 spec:
  dependsOn:
    - name: external-secrets
    - name: nats
    - name: spire
  interval: 10m
  path: ./platform/dynamic-runner
-  prune: false
+  # The runner backends are replaceable. Prune is required when a retired
  # worker is removed from the component; otherwise it keeps consuming work.
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
@@ -1,14 +1,17 @@
 apiVersion: kustomize.toolkit.fluxcd.io/v1
 kind: Kustomization
 metadata:
-  name: gitea-actions
+  name: hydra
  namespace: flux-system
 spec:
  dependsOn:
  - name: envoy-gateway
  - name: external-secrets
  interval: 10m
-  path: ./platform/gitea-runner
+  path: ./apps/hydra
  prune: false
  sourceRef:
    kind: GitRepository
    name: flux-system
-  timeout: 3m
+  timeout: 5m
-  wait: false
+  wait: true
@@ -0,0 +1,22 @@
 apiVersion: kustomize.toolkit.fluxcd.io/v1
 kind: Kustomization
 metadata:
  name: nexus
  namespace: flux-system
 spec:
  dependsOn:
    - name: envoy-gateway
    - name: openebs
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: nexus
      namespace: nexus
  interval: 10m
  path: ./apps/nexus
  prune: false
  sourceRef:
    kind: GitRepository
    name: flux-system
  timeout: 15m
  wait: true
@@ -7,7 +7,7 @@ resources:
  - apps/envoy-gateway.yaml
  - apps/external-secrets.yaml
  - apps/gitea.yaml
-  - apps/gitea-actions.yaml
+  - apps/backstage.yaml
  - apps/http-echo.yaml
  - apps/openebs.yaml
  - apps/nats.yaml
@@ -15,3 +15,5 @@ resources:
  - apps/spire.yaml
  - apps/observability.yaml
  - apps/zot.yaml
  - apps/nexus.yaml
  - apps/hydra.yaml
@@ -15,7 +15,8 @@ Root bootstrap 已完成。后续按依赖顺序分别引入：
 1. 监控 CRD、kube-state-metrics 以及 kubelet/cAdvisor 抓取配置；
 2. SPIRE Agent、SPIFFE CSI Driver 与 workload registration；
 3. Kata Containers、`block-plain` RuntimeClass；
-4. OpenSandbox operator/server 及 `ci-pod`、`ci-vm` Pools。
+4. 独立 External Secrets Operator 与 sandbox 专用 OpenBao auth backend；
 5. OpenSandbox controller/server；CI Pool 与 runner 调度器随后独立接入。
 每一阶段单独合并并等待对应 Flux Kustomization Ready，不在 bootstrap 时一次性部署。
 第一阶段监控拆为 `monitoring-operator` 与依赖它的 `monitoring`，防止 VM CR 在
@@ -42,15 +43,25 @@ attestation。Server 使用 external bundle publisher 持续维护 sandbox
 显式关闭 Server 与 OIDC Provider，只部署 Agent DaemonSet 和 SPIFFE CSI Driver；因此
 不会产生第二个 trust root。
-`spire-smoke` namespace、ServiceAccount 和 `sandbox-spire-smoke` ClusterSPIFFEID 是
+`spire-smoke` namespace、ServiceAccount 和 `sandbox-spire-smoke` ClusterSPIFFEID 只用于
-普通 Pod 与后续 Kata guest 的回归夹具，稳定身份为
+普通 Pod 的 CSI 回归夹具，稳定身份为 `spiffe://ddupan.top/sandbox/smoke`。Kata guest
-`spiffe://ddupan.top/sandbox/smoke`。测试 Pod 临时创建并在验收后删除，身份声明保留。
+不能复用 node Agent 暴露的 Unix socket；virtio-fs 只能呈现 socket 路径，不能把连接
 跨过 VM 边界。Kata workload 必须使用 guest 内 Agent，具体约束见
 `platform/sandbox-kata/README.md`。测试 Pod 临时创建并在验收后删除，普通 Pod 的身份
 声明保留。
 Kata 阶段使用官方 4.1.0 `kata-deploy` chart 的短生命周期 `job` 模式，逐节点安装并
 重启 K3s。只启用 `kata-clh-runtime-rs`，不创建默认 `kata` 别名；该 handler 的
 `emptyDir` 固定使用 `block-plain`，为 Docker/BuildKit overlay2 与 kind 提供 guest
 内块设备文件系统。详细限制与上线验收见 `platform/sandbox-kata/README.md`。
 Sandbox 的 ESO 通过独立 `auth/kubernetes-sandbox` 向 OpenBao 证明 ServiceAccount
 身份，只能读取共享的 `kv/k8s/opensandbox-api`。它不保存 reviewer JWT 或长期 Bao token；相关
 Terraform 与 Flux 边界见 `platform/sandbox-external-secrets/README.md`。
 OpenSandbox 阶段固定官方源码 commit 与 umbrella chart `0.2.2`，只部署 controller、
 ClusterIP server 和 CRD。API key 由 ESO 从 OpenBao 投影，明文不进入 Git。
 ## 监控边界
 这里只管理 sandbox LXC 内的 Kubernetes 监控，不负责 PVE 宿主监控。LXC 与宿主共享
@@ -2,13 +2,14 @@
 apiVersion: kustomize.toolkit.fluxcd.io/v1
 kind: Kustomization
 metadata:
-  name: opensandbox-pools
+  name: ci-runners
  namespace: flux-system
 spec:
  dependsOn:
    - name: opensandbox
    - name: spire-agents
  interval: 10m
-  path: ./platform/sandbox-opensandbox-pools
+  path: ./platform/sandbox-ci-runners
  prune: true
  sourceRef:
    kind: GitRepository
@@ -0,0 +1,17 @@
 ---
 apiVersion: kustomize.toolkit.fluxcd.io/v1
 kind: Kustomization
 metadata:
  name: external-secrets-operator
  namespace: flux-system
 spec:
  dependsOn:
    - name: monitoring-operator
  interval: 10m
  path: ./platform/sandbox-external-secrets/operator
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
  timeout: 10m
  wait: true
@@ -0,0 +1,17 @@
 ---
 apiVersion: kustomize.toolkit.fluxcd.io/v1
 kind: Kustomization
 metadata:
  name: external-secrets
  namespace: flux-system
 spec:
  dependsOn:
    - name: external-secrets-operator
  interval: 10m
  path: ./platform/sandbox-external-secrets/config
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
  timeout: 10m
  wait: true
@@ -6,13 +6,14 @@ metadata:
  namespace: flux-system
 spec:
  dependsOn:
    - name: external-secrets
    - name: kata
-    - name: spire-agents
+    - name: monitoring-operator
  interval: 10m
  path: ./platform/sandbox-opensandbox
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system
-  timeout: 20m
+  timeout: 15m
  wait: true
@@ -7,5 +7,7 @@ resources:
  - apps/spire-bootstrap.yaml
  - apps/spire-agents.yaml
  - apps/kata.yaml
  - apps/external-secrets-operator.yaml
  - apps/external-secrets.yaml
  - apps/opensandbox.yaml
-  - apps/opensandbox-pools.yaml
+  - apps/ci-runners.yaml
@@ -18,7 +18,10 @@ homelab_dns:
      - { zone: ad.ddupan.top, name: metrics-write, type: A, values: [192.168.10.127] }
      - { zone: ad.ddupan.top, name: netbox, type: A, values: [192.168.10.127] }
      - { zone: ad.ddupan.top, name: nats, type: A, values: [192.168.10.127] }
      - { zone: ad.ddupan.top, name: nexus, type: A, values: [192.168.10.127] }
      - { zone: ad.ddupan.top, name: s3, type: A, values: [192.168.10.127] }
      - { zone: ad.ddupan.top, name: hydra, type: A, values: [192.168.10.127] }
      - { zone: ad.ddupan.top, name: hydra-login, type: A, values: [192.168.10.127] }
      - { zone: ad.ddupan.top, name: spire-oidc, type: A, values: [192.168.10.127] }
      - { zone: ad.ddupan.top, name: spire-server, type: A, values: [192.168.10.127] }
      - { zone: ad.ddupan.top, name: zot, type: A, values: [192.168.10.127] }
@@ -51,7 +51,7 @@ few things Terraform must not own.**
 | secrets engine mounts (`kv`, `pki`, `ssh-client-signer`) | the daemon, Raft, TLS files, systemd |
 | PKI role, issuing/CRL URLs, cluster paths, **ACME** | `bao operator init` / unseal (manual, PGP-wrapped) |
 | SSH signing role (`ai-agent`) | **PKI root CA + SSH CA signing key** |
-| OIDC auth *mount* and *role* | OIDC **client secret** (`auth/oidc/config`) |
+| OIDC/Kubernetes auth mount、config 与 role | OIDC **client secret** (`auth/oidc/config`) |
 | all policies | snapshot token + script + systemd timer |
 |  | host-level CA trust distribution (`openbao_ssh_ca_trust`) |
@@ -92,6 +92,13 @@ then `VAULT_ADDR`/`VAULT_TOKEN`), mirroring how `smtp-relay/terraform` uses
 native `openbao/openbao` provider is published only to the OpenTofu registry and
 cannot be resolved by the HashiCorp `terraform` CLI.
 Sandbox 集群使用独立的 `auth/kubernetes-sandbox`。其 API 地址、公开 Kubernetes CA、
 ESO role 与只读 `kv/k8s/opensandbox-api` policy 全部由 Terraform 管理；CA 位于
 `terraform/certs/sandbox-kubernetes-ca.crt`。集群重建并轮换 CA 后，先更新该文件并
 apply，再让 Flux 恢复 ESO reconciliation。该 backend 不保存 reviewer JWT，而是使用
 ESO 的短期登录 JWT 执行 TokenReview。该组资源已于 2026-09-18 apply，随后复验 plan
 为 zero-diff。
 ## DNS
 `bao.ad.ddupan.top` is an **internal-only** name — not a public Cloudflare record and
@@ -59,3 +59,33 @@ resource "vault_kubernetes_auth_backend_role" "external_secrets" {
  # in a long TTL and every extra hour is a longer-lived credential in memory.
  token_ttl = 3600
 }
 # A Kubernetes auth mount can validate identities from only the API server it is
 # configured against. The sandbox cluster therefore cannot reuse auth/kubernetes,
 # whose TokenReview endpoint belongs to homelab.
 resource "vault_auth_backend" "sandbox_kubernetes" {
  type = "kubernetes"
  path = "kubernetes-sandbox"
 }
 resource "vault_kubernetes_auth_backend_config" "sandbox" {
  backend              = vault_auth_backend.sandbox_kubernetes.path
  kubernetes_host      = "https://10.60.0.13:6443"
  kubernetes_ca_cert   = file("${path.module}/certs/sandbox-kubernetes-ca.crt")
  disable_local_ca_jwt = true
  # Deliberately omit token_reviewer_jwt. OpenBao uses the login JWT for
  # TokenReview; the sandbox external-secrets ServiceAccount is bound only to
  # system:auth-delegator and all issued JWTs remain short-lived.
 }
 resource "vault_kubernetes_auth_backend_role" "sandbox_external_secrets" {
  backend   = vault_auth_backend.sandbox_kubernetes.path
  role_name = "external-secrets"
  bound_service_account_names      = ["external-secrets"]
  bound_service_account_namespaces = ["external-secrets"]
  token_policies = [vault_policy.sandbox_external_secrets.name]
  token_ttl      = 3600
 }
@@ -0,0 +1,10 @@
 -----BEGIN CERTIFICATE-----
 MIIBdjCCAR2gAwIBAgIBADAKBggqhkjOPQQDAjAjMSEwHwYDVQQDDBhrM3Mtc2Vy
 dmVyLWNhQDE3ODk2NTI4NTcwHhcNMjYwOTE3MTI0NzM3WhcNMzYwOTE0MTI0NzM3
 WjAjMSEwHwYDVQQDDBhrM3Mtc2VydmVyLWNhQDE3ODk2NTI4NTcwWTATBgcqhkjO
 PQIBBggqhkjOPQMBBwNCAAR4SbqzTXZnlZdUPz7viN6+dYbB1Maw44Qiepn9r5XG
 sOzkYkN8t1aG3Ugo8TqQ3xJaKkM89n1Rluj0vbOhiNajo0IwQDAOBgNVHQ8BAf8E
 BAMCAqQwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUyLSGoKAKAJuiniuRdBLG
 XYaDQC8wCgYIKoZIzj0EAwIDRwAwRAIgFkVzyUZexk/ynnxBEOg+3foJv3WKqAei
 hTSRjO1gL0UCIFbBKR7BMrJJAgW3DJFeeBM+b+tTg93jNx55qZACbFOL
 -----END CERTIFICATE-----
@@ -35,3 +35,10 @@ resource "vault_policy" "external_secrets" {
  name   = "external-secrets"
  policy = file("${path.module}/policies/external-secrets.hcl")
 }
 # The sandbox cluster has its own Kubernetes auth backend and a deliberately
 # narrower KV view than the homelab ESO instance.
 resource "vault_policy" "sandbox_external_secrets" {
  name   = "sandbox-external-secrets"
  policy = file("${path.module}/policies/sandbox-external-secrets.hcl")
 }
@@ -0,0 +1,9 @@
 # Read only the shared OpenSandbox control-plane API key. The same Bao object is
 # consumed by the server in sandbox and, later, by the scheduler in homelab.
 path "kv/data/k8s/opensandbox-api" {
  capabilities = ["read"]
 }
 path "kv/metadata/k8s/opensandbox-api" {
  capabilities = ["read"]
 }
@@ -83,6 +83,11 @@ vyos_postgresql_primary_address: "10.60.0.11"
 vyos_sandbox_k3s_api_address: "10.60.0.13"
 vyos_sandbox_k3s_api_interface: eth1
 vyos_sandbox_k3s_api_port: 6443
 vyos_opensandbox_api_port: 8080
 vyos_opensandbox_node_port: 30080
 # Lifecycle create is synchronous and a cold Kata image pull can exceed the
 # HAProxy 50-second default. Keep this below OpenSandbox's overall timeout.
 vyos_opensandbox_api_timeout: 600
 vyos_sandbox_k3s_servers:
  - name: sandbox1
    address: "10.60.0.11"
@@ -15,6 +15,20 @@ set interfaces ethernet {{ i.iface }} address {{ i.address }}
 set interfaces ethernet {{ i.iface }} description '{{ i.description }}'
 {% endfor %}
 {# OpenSandbox stays on a NodePort; VyOS owns the stable routed frontend. #}
 set load-balancing haproxy service opensandbox-api listen-address {{ vyos_sandbox_k3s_api_address }}
 set load-balancing haproxy service opensandbox-api port {{ vyos_opensandbox_api_port }}
 set load-balancing haproxy service opensandbox-api mode 'tcp'
 set load-balancing haproxy service opensandbox-api backend 'opensandbox-api'
 set load-balancing haproxy service opensandbox-api timeout client {{ vyos_opensandbox_api_timeout }}
 set load-balancing haproxy backend opensandbox-api mode 'tcp'
 set load-balancing haproxy backend opensandbox-api timeout server {{ vyos_opensandbox_api_timeout }}
 {% for server in vyos_sandbox_k3s_servers %}
 set load-balancing haproxy backend opensandbox-api server {{ server.name }} address {{ server.address }}
 set load-balancing haproxy backend opensandbox-api server {{ server.name }} port {{ vyos_opensandbox_node_port }}
 set load-balancing haproxy backend opensandbox-api server {{ server.name }} check
 {% endfor %}
 {# --- default route out; OSPF carries the rest --- #}
 set protocols static route 0.0.0.0/0 next-hop {{ vyos_lan_gateway }}
@@ -18,7 +18,8 @@ Flux 管理以下 Kubernetes 资源：
 - Kata Containers 和 CI 专用的 `block-plain` RuntimeClass；
 - SPIRE Agent、SPIFFE CSI Driver 与 workload identity 声明；
 - vmagent、kube-state-metrics、kubelet/cAdvisor scrape 配置和告警；
- OpenSandbox operator/server、`ci-pod` 与 `ci-vm` Pools。
+- sandbox External Secrets Operator、OpenSandbox controller/server；CI Pool 与 runner
  调度器由 runner 项目接入。
 同一个对象只能有一个 owner。Ansible 不直接部署上述集群内 workload；Flux 不管理
 LXC、K3s datastore 或 K3s 本身。
@@ -92,6 +93,13 @@ ansible-playbook site.yml
 ansible-playbook k3s.yml
 ```
 只 reconcile Flux controllers 与 root sync manifest（不触碰 LXC、PostgreSQL 或
 K3s lifecycle）：
 ```bash
 ansible-playbook flux.yml
 ```
 K3s 外部 datastore URI 由运行时 `SANDBOX_K3S_DB_PASSWORD` 生成，密码在 URI 中
 进行 URL 编码，最终仅持久化于节点 root 可读的 `/etc/rancher/k3s/config.yaml`
 （mode `0600`）。首节点生成的 K3s join token 仅在同一次 Ansible run 内传给第二节点；
@@ -106,7 +114,9 @@ ansible-playbook verify.yml
 当前已经声明 LXC 生命周期、最小 OS baseline、PostgreSQL 和 K3s，包括系统级
 homelab CA trust。Flux `v2.9.5` controllers 与 root sync 也由 Ansible 通过 K3s
 server manifests 管理；root 使用 homelab CA 访问公开 Gitea 仓库，不保存 Git token。
-集群内 workload 由 `clusters/sandbox/` 分阶段纳入 Flux。
+集群内 workload 由 `clusters/sandbox/` 分阶段纳入 Flux。root Kustomization 的健康检查
 timeout 为 40 分钟，用于覆盖 Kata 等首次安装时会逐节点重启 K3s 的子
 Kustomization；各子项仍保留自己的更短 timeout，故障会在对应子项先行暴露。
 ## SPIRE 跨集群 bootstrap
@@ -0,0 +1,6 @@
 ---
 - name: Reconcile Flux controllers and root sync
  hosts: sandbox1
  gather_facts: false
  roles:
    - sandbox_flux
@@ -35,5 +35,5 @@ spec:
  sourceRef:
    kind: GitRepository
    name: flux-system
-  timeout: 3m
+  timeout: 40m
  wait: true
@@ -1,51 +1,109 @@
-# Gitea dynamic runner controller
+# Gitea dynamic runner
-此目录只管理 homelab 中的 controller 部署。controller、worker、Cloud Hypervisor
+本目录部署单副本 Go controller，在同一进程运行 RunnerService scheduler、原生
-launcher 和 guest runner 的源码与发布位于独立仓库
+Kubernetes worker、OpenSandbox worker 和 SPIFFE mTLS facade：
 `panxiao81/gitea-dynamic-runner`。
 当前 bootstrap controller 接收 Gitea `workflow_job` webhook，将 `[self-hosted, pod]` 和
 `[self-hosted, vm]` 的 queued job 分别发布到 NATS。Pod worker 在本集群创建一次性
 privileged host runner；Docker、BuildKit 和 kind 由 workflow 自行 setup。内部
 endpoint：
 ```text
-http://dynamic-runner-controller.dynamic-runner.svc.cluster.local:8787/webhook
+Gitea RunnerService -> scheduler -> JetStream ci.runner.container.kubernetes
                                      |
                                      v
                              homelab Kubernetes Pod
                                      |
                         SPIFFE mTLS RunnerService facade
                                      |
                                      v
                                     Gitea
 ```
-OpenBao 路径：
+`container+kubernetes` placement 不经过 OpenSandbox；`vm+opensandbox` placement
 访问 VyOS 暴露的
 OpenSandbox Lifecycle API；本目录不修改 sandbox 平台侧 ESO、Bao Terraform 或
 OpenSandbox chart 所有权边界。
- `kv/k8s/nats.ci_producer_password`：已有 NATS producer 密码。
+## Canary 安全边界
 - `kv/k8s/nats.ci_worker_password`：已有 NATS worker 密码。
 - `kv/k8s/dynamic-runner.webhook_secret`：Gitea webhook HMAC secret。
 - `kv/k8s/gitea-runner.token`：现有 instance runner registration token。
-首期 controller 与 runner 镜像由 laptop 本机构建后导入 k3s containerd，作为 CI
+- Deployment 为单副本，滚动策略固定 `maxSurge: 1`、`maxUnavailable: 0`，保证新
-发布链路建立前的 bootstrap。部署使用 `imagePullPolicy: Never`。正式发布 workflow
+  facade Ready 后才终止旧实例。scheduler 必须通过 Kubernetes Lease 保持单 leader，
-获得专用 SPIFFE ID 后，必须将 image 改为 zot digest 并移除本地导入步骤。
+  不能依赖 Recreate 避免重复领取。
 - scheduler 使用一个 runner registration，并按总容量启动并发 `FetchTask` goroutine；
  `POD_CAPACITY=4` 与 `VM_CAPACITY=1` 分别限制两个 durable consumer 和 backend
  admission pool。池满时 assignment 保持 JetStream pending，任一 backend 不占用
  另一方的执行槽位，也不会创建超出容量的 workload。
 - rollout 重叠期间只有持有 `Lease/dynamic-runner-scheduler` 的 controller 执行
  `FetchTask`；所有 Ready 实例都可通过 backend metadata 恢复 claim 并服务 facade。
 - executor 镜像使用 digest；Pod 以 UID 2000 运行，SPIRE `ClusterStaticEntry` 同时绑定
  具体 Pod UID 与 `unix:uid:2000`。
 - facade 通过仅内网可路由的 `192.168.10.127:30443` NodePort 提供给 sandbox executor；
  双方使用 Workload API X509-SVID mTLS，并按 SPIFFE ID 而不是 IP/DNS 名验证服务端。
  该入口不经过公网或 Cloudflare Tunnel。
 - controller 的 SPIFFE ID 固定为
  `spiffe://ddupan.top/ns/dynamic-runner/sa/dynamic-runner-controller`。
 - NATS 保留既有最小权限分离：`ci-producer` 仅 publish，`ci-worker` 仅 pull/ACK。
-## 身份绑定
+facade claim registry 在启动时从 Pod/OpenSandbox metadata 恢复；scheduler leadership
 由 Kubernetes Lease 持久化协调。Deployment 仍保持 `replicas: 1`，滚动更新期间允许
 一个额外 Pod 提供 facade 连续性。
-queued webhook 只负责创建没有业务身份的 Pod。runner 实际领取任务后，Gitea 的
+## Secret 边界
 `in_progress` webhook 会携带实际 `runner_name`；controller 将 binding 消息发布到
 NATS，Pod worker 再给对应 Pod 添加：
-```text
+`ExternalSecret/dynamic-runner` 从既有 `ClusterSecretStore/openbao` 读取：
-ci.ddupan.top/identity-bound=true
+
-ci.ddupan.top/spiffe-path=<owner>/<repository>/<percent-encoded-job-name>
+- `kv/k8s/nats`：producer/worker 密码；
 - `kv/k8s/opensandbox-api:api_key`：保留给后续 VM worker；
 - `kv/k8s/gitea-runner:token`：保留的 runner registration token；
 - `kv/k8s/dynamic-runner`：scheduler UUID/token、facade HMAC key 和回滚所需 webhook secret。
 scheduler credential 由官方 Gitea Runner v3.5.0 一次注册生成；它只挂载到 controller，
 不会进入 executor。facade capability key 至少 32 字节，controller 为每个 assignment
 确定性生成独立 capability。不要打印 Secret、创建静态 Bao token或把 credential 写入
 Git。OpenBao 写入使用本机 SPIFFE JWT 换取的短期 `local-development` token。
 ## Workflow 身份与依赖配置
 workflow 可复用 [`panxiao81/ci-actions`](https://git.ddupan.top/panxiao81/ci-actions)
 中的 `spiffe-openbao-login@v1` 和 `setup-nexus@v1`。这不改变 runner 的权限边界：
 runner 仅提供 Node.js 20、`spire-agent` 与 Workload API socket，workflow 负责声明 Bao
 role、audience 和具体用途，目标服务 policy 决定是否授权。短期 Bao token 会进入
 Actions job 临时文件，因此这些 Action 只允许在本目录管理的一次性 Pod/VM executor
 中使用，不能迁移到共享或持久 runner。
 匿名读取 Nexus public repository 只需 `setup-nexus@v1`，不应为了依赖下载额外申请
 Bao 凭据；需要发布制品时再为对应 repository 建立独立 service account 与最小权限
 policy。
 ## 首次验收
 合并后先观察 Flux 与 controller，不要立即开启 VM：
 ```bash
 flux reconcile kustomization dynamic-runner --with-source
 kubectl -n dynamic-runner wait externalsecret/dynamic-runner \
  --for=condition=Ready --timeout=2m
 kubectl -n dynamic-runner rollout status deploy/dynamic-runner-controller --timeout=5m
 kubectl -n dynamic-runner logs deploy/dynamic-runner-controller -f
 ```
-`ClusterSPIFFEID/gitea-dynamic-runner` 只匹配已经绑定的 Pod，并签发
+确认 scheduler 只领取一条 `[self-hosted,container,kubernetes]` task，然后验证：
 `spiffe://ddupan.top/ci/<owner>/<repository>/<job-name>`。runner 的 job-start hook 在
 SVID 可用之前不会放行第一步，因此不能根据 queued 事件错配身份。
-每个 runner Pod 使用 `gitea-dynamic-runner` ServiceAccount。该 ServiceAccount 没有
+1. assignment message 进入并离开 durable `container.kubernetes` consumer；
-Kubernetes API 权限；只有 `dynamic-runner-pod-worker` ServiceAccount 能在本 namespace
+2. `gitea-task-<task-id>` Pod 创建，取得实际 Pod UID；
-create/get/patch/delete Pod。
+3. 同名 `ClusterStaticEntry` 的 parent ID 包含该 UID，SPIFFE ID 使用 repository/job key；
 4. 官方 runner v3.5.0 经 facade claim 精确 task，Gitea 实时收到日志和终态；
 5. Pod consumer 达到 capacity 时 VM consumer 仍可独立接受任务。
-长期实现将由兼容 Gitea RunnerService 的 scheduler 直接领取 task，再交给 Pod/VM
+Pod 与 VM 都在 Gitea 接受终态后先把 terminal marker 写入各自 backend metadata，再由
-executor；届时删除 webhook、临时 runner 注册和 identity binding 消息。跟踪见
+lifecycle reconciler 删除执行器。首次 VM 测试仍须观察 BatchSandbox、Pod 与
-`panxiao81/gitea-dynamic-runner` issue #7。
+ClusterStaticEntry 全部消失；完整自动清理通过前不得提高 `POD_CAPACITY` 或
 `VM_CAPACITY`。
-Gitea webhook 只订阅 `workflow_job`，content type 使用 JSON，secret 与 Bao 中值
+VM backend 已通过 `vm-dev` canary 完成 Docker、kind、SPIFFE 和完整生命周期测试。
-一致。不要启用 `send_everything`，否则 controller 会收到无关仓库事件。
+生产规范标签为 `[self-hosted, vm, opensandbox]`，旧 `[self-hosted, vm]` 仍严格映射到
 同一 placement。初始保持 `VM_CAPACITY=1`；扩容前先观察实际任务的
 资源水位、等待时间以及 OpenSandbox 是否存在 terminal sandbox 残留。
 ## 回滚
 若 controller 在领取 task 前失败，回滚到前一 commit 的 Python controller/Pod worker
 manifests。若已经创建 `gitea-task-*` Pod，先保留现场并核对 Gitea task 状态，不能直接
 重启 scheduler 造成重复执行。scheduler registration 和 capability key 保留在 Bao，
 回滚不需要删除或打印它们。
@@ -1,17 +1,16 @@
 apiVersion: spire.spiffe.io/v1alpha1
 kind: ClusterSPIFFEID
 metadata:
-  name: gitea-dynamic-runner
+  name: dynamic-runner-controller
 spec:
  className: spire-mgmt-spire
-  spiffeIDTemplate: 'spiffe://{{ .TrustDomain }}/ci/{{ index .PodMeta.Annotations "ci.ddupan.top/spiffe-path" }}'
+  spiffeIDTemplate: spiffe://ddupan.top/ns/dynamic-runner/sa/dynamic-runner-controller
  namespaceSelector:
    matchLabels:
      kubernetes.io/metadata.name: dynamic-runner
  podSelector:
    matchLabels:
-      app.kubernetes.io/name: gitea-dynamic-runner
+      app.kubernetes.io/name: dynamic-runner-controller
      ci.ddupan.top/identity-bound: "true"
  workloadSelectorTemplates:
    - k8s:ns:dynamic-runner
-    - k8s:sa:gitea-dynamic-runner
+    - k8s:sa:dynamic-runner-controller
@@ -5,6 +5,11 @@ metadata:
  namespace: dynamic-runner
 spec:
  replicas: 1
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app.kubernetes.io/name: dynamic-runner-controller
@@ -14,17 +19,10 @@ spec:
        app.kubernetes.io/name: dynamic-runner-controller
    spec:
      serviceAccountName: dynamic-runner-controller
      automountServiceAccountToken: false
      initContainers:
        - name: fetch-internal-ca
          image: curlimages/curl:8.16.0@sha256:463eaf6072688fe96ac64fa623fe73e1dbe25d8ad6c34404a669ad3ce1f104b6
-          args:
+          args: [--fail, --silent, --show-error, --output, /trust/ca.pem, https://bao.ad.ddupan.top:8200/v1/pki/ca/pem]
            - --fail
            - --silent
            - --show-error
            - --output
            - /trust/ca.pem
            - https://bao.ad.ddupan.top:8200/v1/pki/ca/pem
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
@@ -40,40 +38,89 @@ spec:
              mountPath: /trust
      containers:
        - name: controller
-          # Bootstrap import on laptop. Replace with a zot digest after the
+          image: zot.ad.ddupan.top/panxiao81/gitea-dynamic-runner-controller@sha256:15735667e8990974a30d9a5e6a1030d63f791ed6d80fa2b0dc9e39959ffcf334
-          # repository's image publishing workflow has a dedicated identity.
+          imagePullPolicy: IfNotPresent
-          image: gitea-dynamic-runner-controller:0.3.0-bootstrap
+          args: [controller]
          imagePullPolicy: Never
          env:
            - name: COMPONENTS
              value: scheduler,kubernetes-worker,opensandbox-worker
            - name: GITEA_INSTANCE_URL
              value: https://git.ddupan.top
            - name: GITEA_RUNNER_UUID_FILE
              value: /run/dynamic-runner-secrets/scheduler-uuid
            - name: GITEA_RUNNER_TOKEN_FILE
              value: /run/dynamic-runner-secrets/scheduler-token
            - name: NATS_URL
              value: tls://nats.ad.ddupan.top:4222
            - name: NATS_CA_FILE
              value: /run/trust/ca.pem
-            - name: NATS_PASSWORD_FILE
+            - name: NATS_PRODUCER_PASSWORD_FILE
              value: /run/dynamic-runner-secrets/nats-password
-            - name: WEBHOOK_SECRET_FILE
+            - name: NATS_WORKER_PASSWORD_FILE
-              value: /run/dynamic-runner-secrets/webhook-secret
+              value: /run/dynamic-runner-secrets/nats-worker-password
            - name: RUNNER_FACADE_CAPABILITY_KEY_FILE
              value: /run/dynamic-runner-secrets/facade-capability-key
            - name: RUNNER_FACADE_LISTEN
              value: :8443
            - name: RUNNER_FACADE_URL
              value: https://192.168.10.127:30443
            - name: RUNNER_FACADE_SPIFFE_ID
              value: spiffe://ddupan.top/ns/dynamic-runner/sa/dynamic-runner-controller
            - name: SPIFFE_ENDPOINT_SOCKET
              value: unix:///run/spire/agent-sockets/spire-agent.sock
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: POD_EXECUTOR_IMAGE
              value: zot.ad.ddupan.top/panxiao81/gitea-dynamic-runner-runner@sha256:83e96c3959af663a75a7229cb04cbbb97a642d4ef3704ac5cdceb1d502a72ed3
            - name: POD_SERVICE_ACCOUNT
              value: gitea-dynamic-runner
            - name: POD_EXECUTOR_UID
              value: "2000"
            - name: POD_CAPACITY
              value: "4"
            - name: OPENSANDBOX_API
              value: http://10.60.0.13:8080
            - name: OPENSANDBOX_API_KEY_FILE
              value: /run/dynamic-runner-secrets/opensandbox-api-key
            - name: OPENSANDBOX_POOL
              value: ci-vm
            - name: VM_CAPACITY
              value: "1"
            - name: VM_RUNNER_LABEL
              value: vm
            - name: VM_TIMEOUT_SECONDS
              value: "14400"
            - name: SPIRE_CLUSTER
              value: homelab
            - name: SPIRE_CLASS
              value: spire-mgmt-spire
            - name: SPIRE_AGENT_ID
              value: spiffe://ddupan.top/spire/agent/k8s_psat/homelab/cd2d0233-c4ea-4031-8327-e7e359e766dd
            - name: SSL_CERT_FILE
              value: /run/trust/ca.pem
          ports:
-            - name: http
+            - name: facade
-              containerPort: 8787
+              containerPort: 8443
          readinessProbe:
-            httpGet:
+            httpGet: null
-              path: /healthz
+            tcpSocket:
-              port: http
+              port: facade
            periodSeconds: 5
          livenessProbe:
-            httpGet:
+            httpGet: null
-              path: /healthz
+            tcpSocket:
-              port: http
+              port: facade
-            initialDelaySeconds: 10
+            initialDelaySeconds: 15
            periodSeconds: 10
          resources:
            requests:
-              cpu: 25m
+              cpu: 50m
-              memory: 32Mi
+              memory: 64Mi
            limits:
-              cpu: 250m
+              cpu: 500m
-              memory: 128Mi
+              memory: 256Mi
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
@@ -91,6 +138,9 @@ spec:
            - name: trust
              mountPath: /run/trust
              readOnly: true
            - name: spire-agent-socket
              mountPath: /run/spire/agent-sockets
              readOnly: true
      securityContext:
        fsGroup: 65532
        fsGroupChangePolicy: OnRootMismatch
@@ -104,3 +154,7 @@ spec:
        - name: trust
          emptyDir:
            sizeLimit: 1Mi
        - name: spire-agent-socket
          csi:
            driver: csi.spiffe.io
            readOnly: true
@@ -20,6 +20,10 @@ spec:
      remoteRef:
        key: k8s/nats
        property: ci_worker_password
    - secretKey: opensandbox-api-key
      remoteRef:
        key: k8s/opensandbox-api
        property: api_key
    - secretKey: webhook-secret
      remoteRef:
        key: k8s/dynamic-runner
@@ -28,3 +32,15 @@ spec:
      remoteRef:
        key: k8s/gitea-runner
        property: token
    - secretKey: scheduler-uuid
      remoteRef:
        key: k8s/dynamic-runner
        property: scheduler_uuid
    - secretKey: scheduler-token
      remoteRef:
        key: k8s/dynamic-runner
        property: scheduler_token
    - secretKey: facade-capability-key
      remoteRef:
        key: k8s/dynamic-runner
        property: facade_capability_key
@@ -6,5 +6,4 @@ resources:
  - rbac.yaml
  - clusterspiffeid.yaml
  - deployment.yaml
  - pod-worker-deployment.yaml
  - service.yaml
@@ -1,100 +0,0 @@
 apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: dynamic-runner-pod-worker
  namespace: dynamic-runner
 spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: dynamic-runner-pod-worker
  template:
    metadata:
      labels:
        app.kubernetes.io/name: dynamic-runner-pod-worker
    spec:
      serviceAccountName: dynamic-runner-pod-worker
      initContainers:
        - name: fetch-internal-ca
          image: curlimages/curl:8.16.0@sha256:463eaf6072688fe96ac64fa623fe73e1dbe25d8ad6c34404a669ad3ce1f104b6
          args:
            - --fail
            - --silent
            - --show-error
            - --output
            - /trust/ca.pem
            - https://bao.ad.ddupan.top:8200/v1/pki/ca/pem
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: [ALL]
            readOnlyRootFilesystem: true
            runAsNonRoot: true
            runAsUser: 101
            runAsGroup: 102
            seccompProfile:
              type: RuntimeDefault
          volumeMounts:
            - name: trust
              mountPath: /trust
      containers:
        - name: pod-worker
          image: gitea-dynamic-runner-controller:0.3.0-bootstrap
          imagePullPolicy: Never
          command: [/venv/bin/gitea-dynamic-runner-pod-worker]
          env:
            - name: NATS_URL
              value: tls://nats.ad.ddupan.top:4222
            - name: NATS_CA_FILE
              value: /run/trust/ca.pem
            - name: NATS_PASSWORD_FILE
              value: /run/dynamic-runner-secrets/nats-worker-password
            - name: RUNNER_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: RUNNER_IMAGE
              value: zot.ad.ddupan.top/panxiao81/gitea-dynamic-runner-runner@sha256:4c61f6315453d68a827ee9542f1345ed86576eaaf62325eb803c8fe3f06ddf2a
            - name: RUNNER_SERVICE_ACCOUNT
              value: gitea-dynamic-runner
            - name: RUNNER_TOKEN_SECRET
              value: dynamic-runner
            - name: RUNNER_CAPACITY
              value: "4"
          resources:
            requests:
              cpu: 25m
              memory: 32Mi
            limits:
              cpu: 250m
              memory: 128Mi
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: [ALL]
            readOnlyRootFilesystem: true
            runAsNonRoot: true
            runAsUser: 65532
            runAsGroup: 65532
            seccompProfile:
              type: RuntimeDefault
          volumeMounts:
            - name: secret
              mountPath: /run/dynamic-runner-secrets
              readOnly: true
            - name: trust
              mountPath: /run/trust
              readOnly: true
      securityContext:
        fsGroup: 65532
        fsGroupChangePolicy: OnRootMismatch
        seccompProfile:
          type: RuntimeDefault
      volumes:
        - name: secret
          secret:
            secretName: dynamic-runner
            defaultMode: 0400
        - name: trust
          emptyDir:
            sizeLimit: 1Mi
@@ -6,12 +6,6 @@ metadata:
 ---
 apiVersion: v1
 kind: ServiceAccount
 metadata:
  name: dynamic-runner-pod-worker
  namespace: dynamic-runner
 ---
 apiVersion: v1
 kind: ServiceAccount
 metadata:
  name: gitea-dynamic-runner
  namespace: dynamic-runner
@@ -19,23 +13,48 @@ metadata:
 apiVersion: rbac.authorization.k8s.io/v1
 kind: Role
 metadata:
-  name: dynamic-runner-pod-worker
+  name: dynamic-runner-controller
  namespace: dynamic-runner
 rules:
  - apiGroups: [""]
    resources: [pods]
-    verbs: [create, get, patch, delete]
+    verbs: [create, get, list, watch, patch, delete]
  - apiGroups: [coordination.k8s.io]
    resources: [leases]
    verbs: [create, get, list, watch, update, patch]
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: RoleBinding
 metadata:
-  name: dynamic-runner-pod-worker
+  name: dynamic-runner-controller
  namespace: dynamic-runner
 roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
-  name: dynamic-runner-pod-worker
+  name: dynamic-runner-controller
 subjects:
  - kind: ServiceAccount
-    name: dynamic-runner-pod-worker
+    name: dynamic-runner-controller
    namespace: dynamic-runner
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRole
 metadata:
  name: dynamic-runner-spiffe-entries
 rules:
  - apiGroups: [spire.spiffe.io]
    resources: [clusterstaticentries]
    verbs: [create, get, delete]
 ---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRoleBinding
 metadata:
  name: dynamic-runner-spiffe-entries
 roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: dynamic-runner-spiffe-entries
 subjects:
  - kind: ServiceAccount
    name: dynamic-runner-controller
    namespace: dynamic-runner
@@ -4,9 +4,11 @@ metadata:
  name: dynamic-runner-controller
  namespace: dynamic-runner
 spec:
  type: NodePort
  selector:
    app.kubernetes.io/name: dynamic-runner-controller
  ports:
-    - name: http
+    - name: facade
-      port: 8787
+      port: 8443
-      targetPort: http
+      targetPort: facade
      nodePort: 30443
@@ -1,114 +0,0 @@
 # Gitea Actions runner
 This is the bootstrap runner for Gitea Actions. One persistent runner Pod accepts
 up to four jobs; each job runs in a dynamically created container inside a
 Docker-in-Docker daemon. The official chart runs DinD privileged. Rootless DinD
 would still be privileged and is blocked by the node's AppArmor user-namespace
 policy, so this deployment uses regular DinD instead of weakening that host-wide
 policy. Only trusted workflows may target this runner.
 DinD 同时使用 `--mtu=1450` 和
 `--default-network-opt=bridge=com.docker.network.driver.mtu=1450`，与 k3s Pod 的
 `eth0` 一致。前者只覆盖 Docker 默认 bridge；act 为每个 job 创建 user-defined
 bridge，必须由后者设置默认 MTU。不要在未验证节点 Pod MTU 的情况下删除或修改这
 两个参数：MTU 1500 的 job 容器虽然能够解析 GitHub、甚至建立 TCP 连接，但较大的
 TLS 数据包会在嵌套网络路径中丢失，表现为 `github.com` / `api.github.com` 超时或
 `setup-go` 每次请求卡满 6 分钟后重试。Pod 网络和默认 Docker bridge 正常不代表
 Actions job bridge 正常。
 The runner is registered at instance scope so it is available to every repository
 on this Gitea instance. Repository permissions and protected-branch review are
 therefore the security boundary; do not enable Actions for untrusted repositories.
 The runner registration token is authoritative in OpenBao at
 `kv/k8s/gitea-runner`. External Secrets Operator projects its `token` property to
 the `gitea-runner-token` Secret. Never put the token in this directory or a Helm
 command line.
 ## SPIRE 与 OCI 发布
 runner Pod 使用专用 ServiceAccount `gitea-actions`，并由精确匹配 namespace、
 ServiceAccount 隐含的 Pod、以及 chart labels 的 `ClusterSPIFFEID` 获得：
 ```text
 spiffe://ddupan.top/ci/gitea-actions
 ```
 SPIFFE CSI socket 同时只读挂载到 runner 和 DinD。act 的 volume allowlist 只允许
 `/run/spire/agent-sockets`；workflow 仍必须在 job container 中显式请求该 bind
 mount。原因是 bind mount 由 DinD 内的 dockerd 解析，只挂 runner 容器无法让 job
 访问 Workload API。
 该身份不是通用 registry 管理员。zot 只对明确列出的 CI 镜像仓库授予
 `read/create/update`，不授予 delete 或其他仓库写入。workflow 应获取
 `aud=zot` 的短期 JWT-SVID，并经 stdin 传给 registry client，不得把 JWT、X.509
 SVID 或 Docker auth 写入 workspace/artifact。
 ## Flux 接管状态
 该 release 最初通过下述 review-first 流程手动 bootstrap。下一个 GitOps 阶段将
 使用 Flux `HelmRelease` 接管它，并首先固定现有 chart `0.1.1`，不在接管 PR 中升级。
 迁移前审计发现：Helm 保存的 user-supplied values 和 release manifest 仍描述失败的
 rootless DinD 尝试，但 live StatefulSet 与本目录 `values.yaml` 都已经使用 regular
 DinD。首次 reconcile 的验收条件是修正 Helm 存储状态，同时 live Pod spec、PVC
 identity、runner capacity 和在线状态保持不变。接管稳定后再用独立 PR 升级 chart。
 接管分两阶段：第一阶段提交 `suspend: true` 的 HelmRelease、HelmRepository 和由
 `values.yaml` 生成的 ConfigMap。Flux 只登记这些对象，不执行 Helm action。合并后
 检查 HelmRepository Ready，并用固定 chart 重复比较期望清单与 live StatefulSet；
 第二阶段解除 suspend。第一阶段已经确认 source Ready、完整 chart render 与 live
 资源零差异，且登记过程中现有 runner 没有 rollout。失败重试使用
 `RetryOnFailure`，不会用 stored rootless release 做 rollback。
 ## 历史 review-first bootstrap
 这是 Flux 安装前执行过的一次性手动部署流程，保留用于恢复和审计：
 1. Merge the reviewed PR.
 2. As a Gitea site administrator, create an instance-scoped runner registration
   token under **Site Administration → Actions → Runners**.
 3. Store it as the `token` property at `kv/k8s/gitea-runner` without exposing it
   in shell history:
   ```bash
   read -rsp 'Runner token: ' runner_token
   printf '%s' "$runner_token" | bao kv put kv/k8s/gitea-runner token=-
   unset runner_token
   ```
 4. From the updated `main`, create the namespace and ExternalSecret, then wait
   for `SecretSynced=True`:
   ```bash
   KUBECONFIG="$HOME/.kube/config" k3s kubectl apply \
     -f platform/gitea-runner/namespace.yaml
   KUBECONFIG="$HOME/.kube/config" k3s kubectl apply \
     -f platform/gitea-runner/external-secret.yaml
   KUBECONFIG="$HOME/.kube/config" k3s kubectl wait \
     --namespace gitea-actions \
     --for=condition=Ready externalsecret/gitea-runner-token \
     --timeout=60s
   ```
 5. Install chart `actions` version `0.1.1` from
   `https://dl.gitea.com/charts/` with this `values.yaml`:
   ```bash
   helm repo add gitea-charts https://dl.gitea.com/charts/
   helm repo update gitea-charts
   helm upgrade --install gitea-actions gitea-charts/actions \
     --namespace gitea-actions \
     --version 0.1.1 \
     --values platform/gitea-runner/values.yaml \
     --wait --timeout 10m
   ```
 6. Confirm the runner is online, then re-run the queued lint workflow.
 Do not deploy from an unmerged feature branch. Do not use `--set` for the token.
 The 1 GiB PVC preserves `.runner` identity. Docker image layers are ephemeral;
 the Pod has a 20 GiB ephemeral-storage limit. Terraform apply jobs must use a
 workflow concurrency group because runner capacity does not serialize access to
 a shared state.
@@ -1,17 +0,0 @@
 apiVersion: spire.spiffe.io/v1alpha1
 kind: ClusterSPIFFEID
 metadata:
  name: gitea-actions
 spec:
  className: spire-mgmt-spire
  spiffeIDTemplate: spiffe://{{ .TrustDomain }}/ci/gitea-actions
  namespaceSelector:
    matchLabels:
      kubernetes.io/metadata.name: gitea-actions
  podSelector:
    matchLabels:
      app.kubernetes.io/instance: gitea-actions
      app.kubernetes.io/name: actions-runner
  workloadSelectorTemplates:
    - k8s:ns:gitea-actions
    - k8s:sa:gitea-actions
@@ -1,8 +0,0 @@
 apiVersion: source.toolkit.fluxcd.io/v1
 kind: HelmRepository
 metadata:
  name: gitea-charts
  namespace: gitea-actions
 spec:
  interval: 1h
  url: https://dl.gitea.com/charts/
@@ -1,18 +0,0 @@
 apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 generatorOptions:
  disableNameSuffixHash: true
  labels:
    reconcile.fluxcd.io/watch: Enabled
 configMapGenerator:
  - name: gitea-actions-values
    namespace: gitea-actions
    files:
      - values.yaml=values.yaml
 resources:
  - namespace.yaml
  - serviceaccount.yaml
  - clusterspiffeid.yaml
  - external-secret.yaml
  - helmrepository.yaml
  - helmrelease.yaml
@@ -1,6 +0,0 @@
 apiVersion: v1
 kind: ServiceAccount
 metadata:
  name: gitea-actions
  namespace: gitea-actions
 automountServiceAccountToken: false
@@ -1,81 +0,0 @@
 enabled: true
 giteaRootURL: http://gitea-http.gitea.svc.cluster.local:3000
 existingSecret: gitea-runner-token
 existingSecretKey: token
 statefulset:
  replicas: 1
  timezone: Etc/UTC
  serviceAccountName: gitea-actions
  extraVolumes:
    - name: spiffe-workload-api
      csi:
        driver: csi.spiffe.io
        readOnly: true
  securityContext:
    fsGroup: 1000
  # Chart 0.1.1 applies this block to both runner and DinD containers.
  resources:
    requests:
      cpu: 250m
      memory: 512Mi
      ephemeral-storage: 2Gi
    limits:
      cpu: "4"
      memory: 6Gi
      ephemeral-storage: 20Gi
  persistence:
    size: 1Gi
  runner:
    registry: docker.io
    repository: gitea/runner
    tag: 2.3.0
    pullPolicy: IfNotPresent
    extraVolumeMounts:
      - name: spiffe-workload-api
        mountPath: /run/spire/agent-sockets
        readOnly: true
    config: |
      log:
        level: info
      runner:
        file: .runner
        capacity: 4
        timeout: 3h
        shutdown_timeout: 3h
        labels:
          - self-hosted:docker://docker.gitea.com/runner-images:ubuntu-latest
      cache:
        enabled: false
      container:
        require_docker: true
        docker_timeout: 300s
        # Workflows must still request this exact bind mount explicitly. The
        # allowlist prevents arbitrary host paths from reaching job containers.
        valid_volumes:
          - /run/spire/agent-sockets
  dind:
    # The node enforces AppArmor's unprivileged-userns restriction, which blocks
    # rootlesskit even though this chart must run DinD privileged either way.
    rootless: false
    registry: docker.io
    repository: docker
    tag: 29.7.1-dind
    pullPolicy: IfNotPresent
    # Bind mounts are resolved by dockerd, so the CSI socket must exist in the
    # DinD container as well as in the runner container.
    extraVolumeMounts:
      - name: spiffe-workload-api
        mountPath: /run/spire/agent-sockets
        readOnly: true
    # k3s uses a 1450-byte pod MTU. Without matching it here, nested Actions
    # networks advertise 1500 and GitHub TLS packets disappear on the outer
    # overlay path while direct pod traffic remains healthy.
    extraArgs:
      - --mtu=1450
      # --mtu only changes Docker's default bridge. act creates a user-defined
      # bridge per job, so give every new bridge the same explicit default.
      - --default-network-opt=bridge=com.docker.network.driver.mtu=1450
@@ -35,10 +35,11 @@ ci_worker_password
 ## CI stream 约定
-controller 首次启动时幂等创建 `CI_RUNNER` stream：`ci.runner.>`、
+controller 使用 `CI_RUNNER` stream：`ci.runner.>`、
 `WorkQueuePolicy`、file storage、24h/10000 条/256 MiB 上限。每类 runner 使用独立
-subject 和 durable pull consumer；`ci.runner.<backend>.binding` 传递 runner 实际
+placement subject 和 durable pull consumer；assignment v2 subject 为
-领取任务后的身份绑定。同类型的多个 worker 共享 durable consumer。ACK 后消息立即
+`ci.runner.<workload-class>.<driver>`，当前为 `ci.runner.container.kubernetes` 与
 `ci.runner.vm.opensandbox`。同 placement 的多个 worker 共享 durable consumer。ACK 后消息立即
 删除，不保存 CI 历史。
 ## 验证
@@ -32,6 +32,14 @@ datasources:
        isDefault: true
        jsonData:
          prometheusType: Prometheus
      - name: Alertmanager
        uid: alertmanager
        type: alertmanager
        access: proxy
        url: http://vmalertmanager-main.monitoring.svc:9093
        jsonData:
          implementation: prometheus
          handleGrafanaManagedAlerts: false
      - name: VictoriaLogs
        type: victoriametrics-logs-datasource
        access: proxy
@@ -0,0 +1,18 @@
 apiVersion: external-secrets.io/v1
 kind: ExternalSecret
 metadata:
  name: alertmanager-telegram
  namespace: monitoring
 spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: openbao
  target:
    name: alertmanager-telegram
    creationPolicy: Owner
  data:
    - secretKey: telegram_bot_token
      remoteRef:
        key: k8s/alertmanager
        property: telegram_bot_token
@@ -0,0 +1,60 @@
 apiVersion: helm.toolkit.fluxcd.io/v2
 kind: HelmRelease
 metadata:
  name: kube-state-metrics
  namespace: monitoring
 spec:
  interval: 30m
  timeout: 5m
  dependsOn:
  - name: vm-operator
  - name: prometheus-operator-crds
  chart:
    spec:
      chart: kube-state-metrics
      version: 8.5.0
      sourceRef:
        kind: HelmRepository
        name: prometheus-community
        namespace: monitoring
      interval: 1h
  driftDetection:
    mode: enabled
  install:
    remediation:
      retries: 3
  upgrade:
    remediation:
      retries: 3
  values:
    fullnameOverride: kube-state-metrics
    collectors:
    - cronjobs
    - daemonsets
    - deployments
    - jobs
    - namespaces
    - nodes
    - persistentvolumeclaims
    - persistentvolumes
    - pods
    - statefulsets
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 500m
        memory: 384Mi
    prometheus:
      monitor:
        enabled: true
        jobLabel: app.kubernetes.io/name
        http:
          interval: 30s
          honorLabels: true
        metrics:
          interval: 30s
          honorLabels: true
    selfMonitor:
      enabled: true
@@ -6,6 +6,7 @@ resources:
  - vmagent.yaml
  - vmalert.yaml
  - vmalertmanager.yaml
  - alertmanager-external-secret.yaml
  - reload-rbac.yaml
  - rules/vm-health.yaml
  - rules/vmagent.yaml
@@ -16,3 +17,7 @@ resources:
  - scrapes/kubelet.yaml
  - exporters/node-exporter.yaml
  - exporters/process-exporter.yaml
  - exporters/kube-state-metrics-helmrelease.yaml
  - rules/kubernetes-health.yaml
  - rules/host-health.yaml
  - rules/monitoring-delivery.yaml
@@ -0,0 +1,66 @@
 apiVersion: monitoring.coreos.com/v1
 kind: PrometheusRule
 metadata:
  name: host-health
  namespace: monitoring
  labels:
    app.kubernetes.io/part-of: victoria-metrics
 spec:
  groups:
  - name: host-health
    interval: 30s
    rules:
    - alert: HostMemoryLow
      expr: (node_memory_MemAvailable_bytes{job="node-exporter"} / node_memory_MemTotal_bytes{job="node-exporter"})
        < 0.10
      for: 10m
      labels:
        severity: warning
      annotations:
        summary: 主机 {{ $labels.instance }} 可用内存不足 10%
        description: 检查主机内存、Swap、进程 PSS；仅使用 node-exporter job，避免重复采集统计。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: HostFilesystemSpaceLow
      expr: (node_filesystem_avail_bytes{job="node-exporter",fstype!~"tmpfs|devtmpfs|overlay|squashfs|nsfs|fuse.*",mountpoint!~"/run(/.*)?|/var/lib/kubelet/.*|/var/lib/docker/.*"}
        / node_filesystem_size_bytes{job="node-exporter",fstype!~"tmpfs|devtmpfs|overlay|squashfs|nsfs|fuse.*",mountpoint!~"/run(/.*)?|/var/lib/kubelet/.*|/var/lib/docker/.*"}
        < 0.10) and on(instance,device,mountpoint) (node_filesystem_readonly{job="node-exporter",fstype!~"tmpfs|devtmpfs|overlay|squashfs|nsfs|fuse.*",mountpoint!~"/run(/.*)?|/var/lib/kubelet/.*|/var/lib/docker/.*"}
        == 0)
      for: 15m
      labels:
        severity: warning
      annotations:
        summary: 主机 {{ $labels.instance }} {{ $labels.mountpoint }} 可用空间不足 10%
        description: 检查文件系统和 ZFS dataset 容量，清理前确认数据归属。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: HostFilesystemInodesLow
      expr: (node_filesystem_files_free{job="node-exporter",fstype!~"tmpfs|devtmpfs|overlay|squashfs|nsfs|fuse.*",mountpoint!~"/run(/.*)?|/var/lib/kubelet/.*|/var/lib/docker/.*"}
        / node_filesystem_files{job="node-exporter",fstype!~"tmpfs|devtmpfs|overlay|squashfs|nsfs|fuse.*",mountpoint!~"/run(/.*)?|/var/lib/kubelet/.*|/var/lib/docker/.*"}
        < 0.10) and on(instance,device,mountpoint) (node_filesystem_files{job="node-exporter",fstype!~"tmpfs|devtmpfs|overlay|squashfs|nsfs|fuse.*",mountpoint!~"/run(/.*)?|/var/lib/kubelet/.*|/var/lib/docker/.*"}
        > 0) and on(instance,device,mountpoint) (node_filesystem_readonly{job="node-exporter",fstype!~"tmpfs|devtmpfs|overlay|squashfs|nsfs|fuse.*",mountpoint!~"/run(/.*)?|/var/lib/kubelet/.*|/var/lib/docker/.*"}
        == 0)
      for: 15m
      labels:
        severity: warning
      annotations:
        summary: 主机 {{ $labels.instance }} {{ $labels.mountpoint }} inode 不足 10%
        description: 检查小文件数量，确认 filesystem 是否耗尽 inode。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: HostFilesystemReadOnly
      expr: node_filesystem_readonly{job="node-exporter",fstype!~"tmpfs|devtmpfs|overlay|squashfs|nsfs|fuse.*",mountpoint!~"/run(/.*)?|/var/lib/kubelet/.*|/var/lib/docker/.*"}
        == 1
      for: 5m
      labels:
        severity: critical
      annotations:
        summary: 主机 {{ $labels.instance }} {{ $labels.mountpoint }} 变为只读
        description: 排查磁盘和文件系统错误；不要直接强制重新挂载。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: HostZpoolUnhealthy
      expr: node_zfs_zpool_state{job="node-exporter",state!="online"} == 1
      for: 5m
      labels:
        severity: critical
      annotations:
        summary: 主机 {{ $labels.instance }} ZFS 池 {{ $labels.zpool }} 状态 {{ $labels.state }}
        description: 检查 zpool status、磁盘和冗余；零值状态不触发。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
@@ -0,0 +1,121 @@
 apiVersion: monitoring.coreos.com/v1
 kind: PrometheusRule
 metadata:
  name: kubernetes-health
  namespace: monitoring
  labels:
    app.kubernetes.io/part-of: victoria-metrics
 spec:
  groups:
  - name: kubernetes-health
    interval: 30s
    rules:
    - alert: KubeNodeNotReady
      expr: max by (node) (kube_node_status_condition{job="kube-state-metrics",condition="Ready",status="true"})
        == 0
      for: 5m
      labels:
        severity: critical
      annotations:
        summary: 节点 {{ $labels.node }} 未就绪
        description: 检查节点、kubelet 和网络；维护时按 node 精确静默。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: KubeNodePressure
      expr: max by (node, condition) (kube_node_status_condition{job="kube-state-metrics",condition=~"MemoryPressure|DiskPressure|PIDPressure",status="true"})
        == 1
      for: 5m
      labels:
        severity: warning
      annotations:
        summary: 节点 {{ $labels.node }} 出现 {{ $labels.condition }}
        description: 检查内存、磁盘/inode 或 PID 资源压力。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: KubePodCrashLooping
      expr: max by (namespace,pod,container) (kube_pod_container_status_waiting_reason{job="kube-state-metrics",reason="CrashLoopBackOff"})
        == 1
      for: 10m
      labels:
        severity: warning
      annotations:
        summary: 容器 {{ $labels.namespace }}/{{ $labels.pod }}/{{ $labels.container }} 持续崩溃
        description: 检查容器退出原因和日志；持续 10 分钟后通知，避免短暂重启噪声。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: KubePodFrequentRestarts
      expr: sum by (namespace,pod,container) (increase(kube_pod_container_status_restarts_total{job="kube-state-metrics"}[15m]))
        > 3
      for: 5m
      labels:
        severity: warning
      annotations:
        summary: 容器 {{ $labels.namespace }}/{{ $labels.pod }}/{{ $labels.container }} 频繁重启
        description: 15 分钟内重启超过 3 次，检查退出原因和资源限制。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: KubePodOOMKilled
      expr: (max by (namespace,pod,container) (kube_pod_container_status_last_terminated_reason{job="kube-state-metrics",reason="OOMKilled"})
        == 1) and on(namespace,pod,container) (sum by (namespace,pod,container) (increase(kube_pod_container_status_restarts_total{job="kube-state-metrics"}[10m]))
        > 0)
      for: 1m
      labels:
        severity: warning
      annotations:
        summary: 容器 {{ $labels.namespace }}/{{ $labels.pod }}/{{ $labels.container }} 发生 OOM 重启
        description: 仅在近期有重启且最近终止原因为 OOMKilled 时触发，检查内存峰值和限制。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: KubePodPending
      expr: max by (namespace,pod) (kube_pod_status_phase{job="kube-state-metrics",phase="Pending"}) == 1
      for: 15m
      labels:
        severity: warning
      annotations:
        summary: Pod {{ $labels.namespace }}/{{ $labels.pod }} 长时间 Pending
        description: 检查调度事件、资源、卷和镜像拉取。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: KubeDeploymentUnavailable
      expr: (max by(namespace,deployment) (kube_deployment_spec_replicas{job="kube-state-metrics"}) - max by(namespace,deployment)
        (kube_deployment_status_replicas_available{job="kube-state-metrics"})) > 0
      for: 10m
      labels:
        severity: warning
      annotations:
        summary: Deployment {{ $labels.namespace }}/{{ $labels.deployment }} 副本不足
        description: 可用副本持续 10 分钟低于声明值；缩容至零不会触发。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: KubeStatefulSetUnavailable
      expr: (max by(namespace,statefulset) (kube_statefulset_replicas{job="kube-state-metrics"}) - max by(namespace,statefulset)
        (kube_statefulset_status_replicas_ready{job="kube-state-metrics"})) > 0
      for: 10m
      labels:
        severity: warning
      annotations:
        summary: StatefulSet {{ $labels.namespace }}/{{ $labels.statefulset }} 副本不足
        description: 检查 Pod 就绪、存储和依赖；已知维护按工作负载精确静默。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: KubeDaemonSetUnavailable
      expr: (max by(namespace,daemonset) (kube_daemonset_status_desired_number_scheduled{job="kube-state-metrics"})
        - max by(namespace,daemonset) (kube_daemonset_status_number_ready{job="kube-state-metrics"})) > 0
      for: 10m
      labels:
        severity: warning
      annotations:
        summary: DaemonSet {{ $labels.namespace }}/{{ $labels.daemonset }} 副本不足
        description: 检查节点和 Pod 就绪；不按整个 namespace 屏蔽。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: KubePVCPending
      expr: max by(namespace,persistentvolumeclaim) (kube_persistentvolumeclaim_status_phase{job="kube-state-metrics",phase="Pending"})
        == 1
      for: 15m
      labels:
        severity: warning
      annotations:
        summary: PVC {{ $labels.namespace }}/{{ $labels.persistentvolumeclaim }} 长时间 Pending
        description: 检查 StorageClass、调度拓扑、容量与 provisioner。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: KubeJobFailed
      expr: max by(namespace,job_name) (kube_job_failed{job="kube-state-metrics",condition="true"}) == 1
      for: 5m
      labels:
        severity: warning
      annotations:
        summary: Job {{ $labels.namespace }}/{{ $labels.job_name }} 已失败
        description: 检查失败 Job 的 Pod 和任务日志；成功重试中的失败 Pod 计数不作为失败 Job。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
@@ -0,0 +1,58 @@
 apiVersion: monitoring.coreos.com/v1
 kind: PrometheusRule
 metadata:
  name: monitoring-delivery
  namespace: monitoring
  labels:
    app.kubernetes.io/part-of: victoria-metrics
 spec:
  groups:
  - name: monitoring-delivery
    interval: 30s
    rules:
    - alert: KubeStateMetricsUnavailable
      expr: up{job="kube-state-metrics"} == 0 or absent(up{job="kube-state-metrics"})
      for: 5m
      labels:
        severity: critical
      annotations:
        summary: kube-state-metrics 采集不可用
        description: 检查 HelmRelease、ServiceMonitor 转换、targets 与 exporter；目标消失也会触发。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: NodeExporterUnavailable
      expr: up{job="node-exporter"} == 0 or absent(up{job="node-exporter"})
      for: 5m
      labels:
        severity: critical
      annotations:
        summary: node-exporter 采集不可用
        description: 检查节点与 exporter，主机健康规则依赖此采集。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: NatsMetricsUnavailable
      expr: up{job="nats/nats"} == 0 or absent(up{job="nats/nats"})
      for: 5m
      labels:
        severity: warning
      annotations:
        summary: NATS 指标采集不可用
        description: 依次检查 PodMonitor、转换后的 VMPodScrape、:7777 exporter 及 targets。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: AlertmanagerTelegramDeliveryFailed
      expr: sum by(job,instance,integration) (increase(alertmanager_notifications_failed_total{job="vmalertmanager-main",integration="telegram"}[5m]))
        > 0
      for: 1m
      labels:
        severity: critical
      annotations:
        summary: Alertmanager 向 Telegram 发送失败
        description: 检查网络、bot token 和频道权限。若 Telegram 完全不可达，本告警也无法经同一渠道送达；需在 Alertmanager/Grafana 排查，集群外心跳另行建设。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
    - alert: AlertmanagerConfigurationReloadFailed
      expr: alertmanager_config_last_reload_successful{job="vmalertmanager-main"} == 0
      for: 5m
      labels:
        severity: critical
      annotations:
        summary: Alertmanager 配置加载失败
        description: 检查 operator 和 Alertmanager 日志、配置格式及 Secret 引用。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
@@ -1,6 +1,6 @@
-# Pull metrics from Docker hosts running ../../docker-hosts/compose.yaml.
+# 仅抓取已存在的宿主机 node-exporter。旧 :8080 目标实际返回 nginx 404，
-# vmagent scrapes each host's node-exporter (:9100) and cAdvisor (:8080) over the LAN.
+# 独立 Docker cAdvisor 尚未部署；部署并确认端口后再添加其目标。
-# Add one target block per host; keep the `host` label in sync with its HOST_LABEL.
+# Kubernetes 容器指标由 kubelet.yaml 的 /metrics/cadvisor 独立采集。
 apiVersion: operator.victoriametrics.com/v1beta1
 kind: VMStaticScrape
 metadata:
@@ -17,13 +17,6 @@ spec:
      labels:
        job: node-exporter
        host: docker-01
-    - targets:
+    # --- 新增主机前确认 exporter 已部署且 /metrics 返回成功 ---
        - "192.168.10.127:8080"
      labels:
        job: cadvisor
        host: docker-01
    # --- add more hosts below, mirroring the two blocks above ---
    # - targets: ["192.168.10.x:9100"]
    #   labels: { job: node-exporter, host: docker-02 }
    # - targets: ["192.168.10.x:8080"]
    #   labels: { job: cadvisor, host: docker-02 }
@@ -13,6 +13,9 @@ spec:
  honorLabels: true
  honorTimestamps: false
  interval: 30s
  # k3s 此端点包含 apiserver/etcd 指标，响应已超过默认 16 MiB。
  # 仅放宽 kubelet 任务，其他采集目标保留默认限制。
  max_scrape_size: "32MiB"
  tlsConfig:
    insecureSkipVerify: true
    caFile: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
@@ -0,0 +1,20 @@
 #!/usr/bin/env bash
 # 依赖 Python 3 + PyYAML，以及 PATH 中的 promtool（验证版本 3.5.0）。
 set -euo pipefail
 test_dir="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
 check_dir="$(mktemp -d)"
 trap 'rm -rf "$check_dir"' EXIT
 python3 - "$test_dir/../rules" "$check_dir/rules.yaml" <<'PY'
 import pathlib
 import sys
 import yaml
 groups = []
 for name in ("kubernetes-health", "host-health", "monitoring-delivery"):
    path = pathlib.Path(sys.argv[1]) / f"{name}.yaml"
    groups.extend(yaml.safe_load(path.read_text())["spec"]["groups"])
 pathlib.Path(sys.argv[2]).write_text(yaml.safe_dump({"groups": groups}, allow_unicode=True))
 PY
 cp "$test_dir/foundation.test.yaml" "$check_dir/foundation.test.yaml"
 promtool check rules "$check_dir/rules.yaml"
 promtool test rules "$check_dir/foundation.test.yaml"
@@ -0,0 +1,137 @@
 rule_files:
 - rules.yaml
 evaluation_interval: 1m
 tests:
 - name: 健康 ZFS 的零值故障状态不报警
  interval: 1m
  input_series:
  - series: node_zfs_zpool_state{job="node-exporter",instance="laptop",zpool="data",state="degraded"}
    values: '0x20'
  - series: node_zfs_zpool_state{job="node-exporter",instance="laptop",zpool="data",state="online"}
    values: 1x20
  alert_rule_test:
  - eval_time: 10m
    alertname: HostZpoolUnhealthy
    exp_alerts: &id001 []
 - name: ZFS 降级持续五分钟触发
  interval: 1m
  input_series:
  - series: node_zfs_zpool_state{job="node-exporter",instance="laptop",zpool="data",state="degraded"}
    values: 1x20
  alert_rule_test:
  - eval_time: 4m
    alertname: HostZpoolUnhealthy
    exp_alerts: *id001
  - eval_time: 6m
    alertname: HostZpoolUnhealthy
    exp_alerts:
    - exp_labels:
        job: node-exporter
        instance: laptop
        zpool: data
        state: degraded
        severity: critical
      exp_annotations:
        summary: 主机 laptop ZFS 池 data 状态 degraded
        description: 检查 zpool status、磁盘和冗余；零值状态不触发。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
 - name: 忽略旧 docker-hosts 重复主机指标
  interval: 1m
  input_series:
  - series: node_memory_MemAvailable_bytes{job="node-exporter",instance="laptop"}
    values: 50x20
  - series: node_memory_MemTotal_bytes{job="node-exporter",instance="laptop"}
    values: 100x20
  - series: node_memory_MemAvailable_bytes{job="docker-hosts",instance="laptop"}
    values: 1x20
  - series: node_memory_MemTotal_bytes{job="docker-hosts",instance="laptop"}
    values: 100x20
  alert_rule_test:
  - eval_time: 15m
    alertname: HostMemoryLow
    exp_alerts: *id001
 - name: 临时文件系统耗尽不触发持久磁盘告警
  interval: 1m
  input_series:
  - series: node_filesystem_avail_bytes{job="node-exporter",instance="laptop",device="tmpfs",mountpoint="/run",fstype="tmpfs"}
    values: 1x20
  - series: node_filesystem_size_bytes{job="node-exporter",instance="laptop",device="tmpfs",mountpoint="/run",fstype="tmpfs"}
    values: 100x20
  - series: node_filesystem_readonly{job="node-exporter",instance="laptop",device="tmpfs",mountpoint="/run",fstype="tmpfs"}
    values: '0x20'
  alert_rule_test:
  - eval_time: 20m
    alertname: HostFilesystemSpaceLow
    exp_alerts: *id001
 - name: Deployment 允许短暂滚动且缩零不报错
  interval: 1m
  input_series:
  - series: kube_deployment_spec_replicas{job="kube-state-metrics",namespace="app",deployment="web"}
    values: 2x20
  - series: kube_deployment_status_replicas_available{job="kube-state-metrics",namespace="app",deployment="web"}
    values: 1x20
  - series: kube_deployment_spec_replicas{job="kube-state-metrics",namespace="app",deployment="scaled-down"}
    values: '0x20'
  - series: kube_deployment_status_replicas_available{job="kube-state-metrics",namespace="app",deployment="scaled-down"}
    values: '0x20'
  alert_rule_test:
  - eval_time: 5m
    alertname: KubeDeploymentUnavailable
    exp_alerts: *id001
  - eval_time: 11m
    alertname: KubeDeploymentUnavailable
    exp_alerts:
    - exp_labels:
        namespace: app
        deployment: web
        severity: warning
      exp_annotations:
        summary: Deployment app/web 副本不足
        description: 可用副本持续 10 分钟低于声明值；缩容至零不会触发。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
 - name: NATS 发现集合完全消失也报警
  interval: 1m
  input_series: []
  alert_rule_test:
  - eval_time: 6m
    alertname: NatsMetricsUnavailable
    exp_alerts:
    - exp_labels:
        job: nats/nats
        severity: warning
      exp_annotations:
        summary: NATS 指标采集不可用
        description: 依次检查 PodMonitor、转换后的 VMPodScrape、:7777 exporter 及 targets。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
 - name: 历史 OOM 没有新重启不反复报警
  interval: 1m
  input_series:
  - series: kube_pod_container_status_last_terminated_reason{job="kube-state-metrics",namespace="app",pod="web",container="web",reason="OOMKilled"}
    values: 1x20
  - series: kube_pod_container_status_restarts_total{job="kube-state-metrics",namespace="app",pod="web",container="web"}
    values: 5x20
  alert_rule_test:
  - eval_time: 10m
    alertname: KubePodOOMKilled
    exp_alerts: *id001
 - name: Telegram 失败跨 reason 汇总为一条
  interval: 1m
  input_series:
  - series: alertmanager_notifications_failed_total{job="vmalertmanager-main",instance="am",integration="telegram",reason="clientError"}
    values: 0+1x20
  - series: alertmanager_notifications_failed_total{job="vmalertmanager-main",instance="am",integration="telegram",reason="serverError"}
    values: 0+1x20
  alert_rule_test:
  - eval_time: 6m
    alertname: AlertmanagerTelegramDeliveryFailed
    exp_alerts:
    - exp_labels:
        job: vmalertmanager-main
        instance: am
        integration: telegram
        severity: critical
      exp_annotations:
        summary: Alertmanager 向 Telegram 发送失败
        description: 检查网络、bot token 和频道权限。若 Telegram 完全不可达，本告警也无法经同一渠道送达；需在 Alertmanager/Grafana
          排查，集群外心跳另行建设。
        runbook_url: https://git.ddupan.top/panxiao81/homelab-wiki/src/branch/main/guides/monitoring-foundation.md
@@ -10,6 +10,11 @@ spec:
  replicaCount: 1
  selectAllByDefault: true
  evaluationInterval: 30s
  # 告警 Source 复用已认证的 Grafana 入口，携带表达式和触发时间。
  # 整个 Explore JSON 一次性 URL 编码，避免表达式中的引号、&、+ 损坏链接。
  extraArgs:
    external.url: https://grafana.ad.ddupan.top
    external.alert.source: 'explore?left={{ printf "{\"datasource\":\"VictoriaMetrics\",\"queries\":[{\"expr\":%s,\"refId\":\"A\"}],\"range\":{\"from\":\"%d\",\"to\":\"now\"}}" (.Expr | jsonEscape) .ActiveAt.UnixMilli | queryEscape }}'
  datasource:
    url: http://vmsingle-main.monitoring.svc:8428
  remoteWrite:
--- a/Show More
+++ b/Show More