归档：Kata / microVM runner 实验（2026-09-17 工作区快照）

从旧工作区 chore/recover-old-workspace 清理时保存，内容与 2026-09-17 stash@{0} 快照中的版本一致；未合并、未在 main 上使用，仅作参考，不开 PR。被 gitignore 的 tfstate 与凭据文件不在此分支，仍留在本地工作区。 Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-01 17:30:48 +00:00
parent 69d3476a6b
commit 55bb5be8b6
39 changed files with 1891 additions and 0 deletions
@@ -0,0 +1,180 @@
+# kata-lxc-lab
+
+在 `pve2` 上验证 privileged LXC 内运行 k3s、Kata/QEMU，并直接使用 PVE host 的
+`/dev/kvm`、`/dev/vhost-net` 与 `/dev/vhost-vsock`。这是可销毁 PoC，不承载持久数据。
+
+Terraform 复用原 VM PoC 的 ignored credential 文件，不复制 token：
+
+```bash
+cd infrastructure/kata-lxc-lab/terraform
+export SSL_CERT_FILE=/tmp/kata-lab-ddupan-ca.pem
+terraform init
+terraform plan \
+  -var-file=../../kata-lab/terraform/credentials.auto.tfvars
+terraform apply \
+  -var-file=../../kata-lab/terraform/credentials.auto.tfvars
+```
+
+当前 PoC 使用 `pve2`、VMID 147、Ubuntu 24.04 LXC template 和 16 GiB
+`local-lvm` rootfs。容器为 privileged，并开启 nesting、keyctl、FUSE、mknod；
+Kata 所需的 KVM、vhost 与 `/dev/net/tun` 设备显式透传；另透传 `/dev/kmsg`，因为
+kubelet 启动时会打开该设备。
+PVE 9 的 `force_rw_sys=1` feature 用于开放 kubelet 所需的少量 `/proc/sys` 写操作；
+provider 0.111.1 尚未暴露该 feature，因此同样由 `pct` 设置。
+
+k3s/Kata worker 还使用以下 privileged LXC 原生配置；PVE 9 使用 cgroup v2，因此设备
+规则采用 `lxc.cgroup2.devices.allow`：
+
+```text
+lxc.apparmor.profile: unconfined
+lxc.cgroup2.devices.allow: a
+lxc.cap.drop:
+lxc.mount.auto: proc:rw sys:rw
+```
+
+这些设置让容器内 kubelet、containerd 和 Kata shim 能管理 mount、cgroup、设备与
+共享宿主 sysctl。因此外层 LXC 不是安全边界；不可信 CI 的安全边界是内层 Kata VM。
+`/dev/kmsg` 当前直接透传，没有使用 `/dev/console` symlink fallback。
+
+PVE 将 privileged LXC 的创建限制为 `root@pam`（API token 即使拥有 `PVEAdmin` 也
+缺少 root-only 的 `Sys.Modify`）。因此实际创建入口使用 root SSH 上的 Ansible/pct：
+
+```bash
+cd infrastructure/kata-lxc-lab/ansible
+ansible-playbook create.yml
+ansible-playbook bootstrap.yml
+ansible-playbook kata.yml
+ansible-playbook verify.yml
+ansible-playbook verify-fuse.yml
+```
+
+Terraform 文件保留用于记录 provider 权限边界与声明式目标；当前 token 执行 apply 会
+在创建前返回 403，不会产生资源或 state。
+
+## 验证结果
+
+当前管理地址为 `192.168.10.128`：2026-09-14 主 LAN DHCP 池调整后，从 `.11`
+续租迁移到 `.128`，并同步 Ansible inventory。它仍是动态租约，不是固定地址。
+
+2026-09-14，pve2 VMID 147 / `192.168.10.11` 验证通过：
+
+- LXC 内 k3s `v1.36.4+k3s1` node Ready；
+- Kata Containers 4.1.0 `qemu-runtime-rs` 安装成功；
+- Kata guest kernel `6.18.35`，外层共享的 PVE kernel `7.0.2-6-pve`；
+- Kata Pod 内的 Docker 27.5.1 daemon 成功运行 `busybox:1.37` 嵌套容器；
+- `/dev/net/tun` 是 Kata 创建 TAP/link 的必要设备，未透传时 runtime-rs 在
+  `create link` 返回 `ENOENT`；
+- `fuse-overlayfs` smoke test 成功：Docker 报告
+  `storage-driver=fuse-overlayfs`，并成功运行嵌套容器。
+
+正式方案保持 Kata 默认 `emptydir_mode = "shared-fs"`，不向 LXC 暴露 host
+`/dev/loop-control` 或 `/dev/loopN`。dockerd 镜像需要包含 `fuse-overlayfs`；启动前在
+Kata guest 内创建临时设备节点 `mknod /dev/fuse c 10 229`，随后强制传入
+`--storage-driver=fuse-overlayfs`。该 `/dev/fuse` 属于内层 guest kernel，不是 PVE
+host 设备透传，并随 Kata Pod 一起销毁。
+
+删除三个 smoke Pod 后，LXC cgroup 统计约 505 MB anonymous memory、1.34 GB file
+cache；后者主要来自刚拉取的 Kata/Docker images，可由宿主回收。也就是说常驻 worker
+的不可回收 working set 约 500 MB，8 GiB 配置是 cgroup 上限而非预分配。
+
+## Gitea ephemeral runner PoC
+
+`runner/` 定义一次性 Gitea runner Pod。整个 Pod 使用 `runtimeClassName: kata`；
+runner 通过 guest 内同 Pod 的 dockerd 创建 job container，不连接 LXC/PVE host 的
+Docker socket。dockerd 使用 zot 中预先发布的 `dind-fuse:29.7.1`，其
+`/var/lib/docker` 是随 Kata Pod 删除的 `emptyDir`。
+
+runner 以 `GITEA_RUNNER_EPHEMERAL=1` 注册，只接收一个 `runs-on: kata-poc` job；接单
+后 Gitea 撤销其 runner credential，job 完成后进程与 Kata VM 一起退出。当前 PoC
+手动创建一个 runner 等待 `.gitea/workflows/kata-runner-smoke.yml`，尚未部署
+`workflow_job` webhook controller，因此不会保留常驻空闲 runner。
+
+注册 token 只在运行时从现有 OpenBao/ESO 消费副本投递到 PoC 集群的临时 Secret，
+不写入此目录。两个 Pod image 使用 `imagePullPolicy: Never`，部署前从可信工作站将
+已验证镜像导入 PoC k3s containerd；这是当前 zot 拉取仍要求 SPIFFE 身份时的
+bootstrap 边界，不是正式镜像分发方案。
+
+正式接入 zot 前需要把这个 cluster 注册到 SPIRE，并为 runner Pod 创建专用
+`ClusterSPIFFEID`。届时 runner 与 dockerd sidecar 在同一路径只读挂载 CSI Workload
+API socket，runner 的 `container.options` 再把该路径 bind-mount 到 job container。
+zot 只对最终 SPIFFE ID 的 `panxiao81/dind-fuse` 仓库授予
+`read/create/update`；禁止授予 namespace 或整个 trust domain 写权限。
+
+2026-09-14 已完成真实 Gitea job 验收：Actions run `242` / job `445` 使用
+`alpine:3.22`，10 秒成功；runner container 退出码为 `0`，原生 dockerd sidecar 随后
+由 kubelet 终止，Pod 最终为 `Succeeded`，没有保留空闲 Kata VM。guest 内核为
+`6.18.35`，dockerd `29.7.1` 使用 `fuse-overlayfs`。
+
+首次尝试暴露了两个部署陷阱：`COPY` 默认保留源文件 mode，入口脚本原为 `0664`，
+必须在 Dockerfile 使用 `COPY --chmod=0755`；只配置 `ephemeral-storage: 20Gi` limit
+会形成同值 request，使 12 GiB PoC 节点无法调度。当前 manifest 显式 request 1 GiB、
+limit 8 GiB。完整 `ubuntu-latest` job image 也不适合这个小 rootfs，基础生命周期验收
+改用 Alpine；正式 worker 必须扩容本地 image/cache 空间或使用更精简的 job image。
+
+## Cloud Hypervisor 内存记账验证
+
+2026-09-17 在 pve2 的一次性 VMID 148 中安装 k3s v1.36.4+k3s1 与 Kata 4.1.0，
+只启用 `clh-runtime-rs`，验证 Cloud Hypervisor 在 privileged LXC 内不会重现早期
+microVM PoC 的 private memfd 双重记账问题。测试完成后已删除 VMID 148。
+
+Cloud Hypervisor `/api/v1/vm.info` 报告：
+
+```json
+{"size":3254779904,"shared":true,"hugepages":false,"prefault":false,"thp":true}
+```
+
+Kata guest 在 memory-backed `emptyDir` 中写入 1 GiB 后：
+
+- VMM `Pss_Shmem` 从约 247 MiB 增至约 1269 MiB；
+- VMM `Pss_Anon` 写入前后均约 1.2 MiB；
+- 外层 LXC `shmem` 增加约 1 GiB，`anon` 基本不变；
+- LXC `memory.events` 的 `max`、`oom` 与 `oom_kill` 均为 0；
+- 删除 Pod 后 VMM 退出，LXC `shmem` 回落到约 1.6 MiB。
+
+因此 Kata 的 Cloud Hypervisor handler 已使用 shared memfd，等价于独立 launcher 的
+`--memory shared=on` 修复；后续 OpenSandbox/Kata 方案无需为 guest RAM 预留近似双倍
+的 LXC cgroup 内存。部署验收仍应检查 `vm.info.config.memory.shared=true`，防止上游
+配置或 runtime handler 变化造成回归。
+
+测试期间还发现 pve2 `local-lvm` thin pool 已达到 100%；临时 rootfs 必须放到
+`pve-rg-hdd`，不能根据容器内部 `df` 的剩余空间判断 thin pool 是否可写。
+
+## Block-backed emptyDir 与 CI 构建验证
+
+2026-09-17 又在 pve2 的一次性 VMID 148 中验证 Kata 4.1.0
+`clh-runtime-rs` 的以下 drop-in：
+
+```toml
+[runtime]
+emptydir_mode = "block-plain"
+```
+
+结果如下：
+
+- 普通 Kubernetes `emptyDir` 被创建为稀疏 `disk.img`，由 Cloud Hypervisor 直接作为
+  block device 热插拔；guest 内 `/var/lib/docker` 是 `/dev/vdb` 上的 ext4。
+- Docker 29.1.5 成功强制使用原生 `overlay2`，不再需要 `fuse-overlayfs`；整个过程
+  没有使用 PVE host 或 LXC 的 loop device。
+- `emptyDir.sizeLimit: 8Gi` 没有决定虚拟磁盘容量。由于 kubelet 所在文件系统约
+  12 GiB，guest 看到的 ext4 约 11.7 GiB；这与 Kata 已知的 block-backed emptyDir
+  限制一致。
+- 包含 256 MiB payload、2000 个小文件和一个额外复制层的冷 BuildKit 构建耗时
+  46 秒；同一 LXC、同一 DRBD HDD 存储上的 runc 对照为 34 秒，Kata 约慢 35%。
+- 在 overlay2 容器层内写入并 fsync 512 MiB：Kata 为 12 秒（40.5 MB/s），runc 为
+  10 秒（50.3 MB/s）；创建一万个小文件两者均约 2 秒。
+- kind v0.27.0 / Kubernetes v1.32.2 首次启动失败是因为 Kata guest 内的 dockerd
+  容器没有 `/dev/kmsg`，不是 block storage 或 cgroup 故障。在 privileged dockerd
+  启动前执行 `mknod /dev/kmsg c 1 11` 后，kind 创建成功：总计 122 秒，进入等待阶段
+  后 23 秒 control-plane Ready，随后可正常删除。
+- 测试时稀疏 backing file 逻辑大小约 12 GiB，构建和 kind 后实际占用约 1.8 GiB。
+  删除 Pod 后 backing file、Cloud Hypervisor 进程和 kind 容器全部消失；host/LXC
+  均无 loop device 残留，LXC 没有 OOM 事件。
+
+因此 `block-plain` 能解决 virtio-fs 不能作为 overlayfs upperdir 的问题，并满足
+BuildKit 与 kind 的功能要求。正式 CI RuntimeClass 必须使用独立的 block-backed
+配置，并在 dockerd bootstrap 中创建 guest `/dev/kmsg`。它仍比同机 runc 构建慢约
+20–35%，且 `emptyDir.sizeLimit` 不能可靠限制虚拟磁盘容量。PoC 曾据此建议为 kubelet
+目录提供独立文件系统；正式 sandbox 集群最终没有采用该设计：独立 volume 增加了
+DRBD 与 LXC 挂载生命周期复杂度，却没有隔离 containerd 等其他节点数据。正式节点
+改用单一、容量明确的 rootfs，并以磁盘占用监控、Pod 删除后的 backing-file 回收检查
+和实际构建基准作为上线门槛。
@@ -0,0 +1,10 @@
+[defaults]
+inventory = inventory/hosts.yml
+host_key_checking = True
+interpreter_python = auto_silent
+retry_files_enabled = False
+local_tmp = /tmp/ansible-kata-lxc-local
+remote_tmp = /tmp/ansible-kata-lxc-remote
+
+[ssh_connection]
+pipelining = True
@@ -0,0 +1,58 @@
+---
+- name: Bootstrap k3s in the Kata LXC worker
+  hosts: pve2
+  gather_facts: false
+  vars:
+    kata_lxc_id: 147
+  tasks:
+    - name: Check base packages in LXC
+      ansible.builtin.command:
+        cmd: >-
+          pct exec {{ kata_lxc_id }} -- dpkg-query -W
+          ca-certificates curl jq openssh-server
+      register: kata_lxc_packages
+      changed_when: false
+      failed_when: false
+
+    - name: Install base packages and SSH server in LXC
+      ansible.builtin.command:
+        cmd: >-
+          pct exec {{ kata_lxc_id }} -- bash -lc
+          'apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y
+          ca-certificates curl jq openssh-server'
+      when: kata_lxc_packages.rc != 0
+
+    - name: Install k3s in LXC
+      ansible.builtin.shell:
+        cmd: |
+          set -euo pipefail
+          pct exec {{ kata_lxc_id }} -- bash -lc '
+            if ! command -v k3s >/dev/null; then
+              curl -sfL https://get.k3s.io -o /tmp/install-k3s.sh
+              chmod 0755 /tmp/install-k3s.sh
+              INSTALL_K3S_EXEC="server --disable=traefik --write-kubeconfig-mode=0644" /tmp/install-k3s.sh
+            fi
+          '
+        executable: /bin/bash
+      register: kata_lxc_k3s_install
+      changed_when: "'No change detected' not in kata_lxc_k3s_install.stdout"
+
+    - name: Wait for k3s node readiness
+      ansible.builtin.command:
+        cmd: >-
+          pct exec {{ kata_lxc_id }} -- /usr/local/bin/k3s kubectl wait
+          --for=condition=Ready node/kata-lxc-lab --timeout=180s
+      changed_when: false
+
+    - name: Check LXC virtualization and devices
+      ansible.builtin.command:
+        cmd: >-
+          pct exec {{ kata_lxc_id }} -- bash -lc
+          'systemd-detect-virt; ls -l /dev/kvm /dev/vhost-net /dev/vhost-vsock;
+          /usr/local/bin/k3s kubectl get node -o wide'
+      register: kata_lxc_ready
+      changed_when: false
+
+    - name: Report k3s readiness
+      ansible.builtin.debug:
+        var: kata_lxc_ready.stdout_lines
@@ -0,0 +1,137 @@
+---
+- name: Create the disposable Kata LXC worker
+  hosts: pve2
+  gather_facts: false
+  vars:
+    kata_lxc_id: 147
+    kata_lxc_template: laptop:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst
+  tasks:
+    - name: Check whether the LXC already exists
+      ansible.builtin.stat:
+        path: /etc/pve/lxc/{{ kata_lxc_id }}.conf
+      register: kata_lxc_config
+
+    - name: Create privileged LXC with Kata host devices
+      ansible.builtin.command:
+        argv:
+          - pct
+          - create
+          - "{{ kata_lxc_id }}"
+          - "{{ kata_lxc_template }}"
+          - --hostname
+          - kata-lxc-lab
+          - --ostype
+          - ubuntu
+          - --rootfs
+          - local-lvm:12
+          - --cores
+          - "4"
+          - --memory
+          - "8192"
+          - --swap
+          - "0"
+          - --net0
+          - name=eth0,bridge=vmbr0,ip=dhcp
+          - --features
+          - nesting=1,keyctl=1,fuse=1,mknod=1,force_rw_sys=1
+          - --unprivileged
+          - "0"
+          - --onboot
+          - "0"
+          - --ssh-public-keys
+          - /root/.ssh/authorized_keys
+          - --dev0
+          - path=/dev/kvm,mode=0660
+          - --dev1
+          - path=/dev/vhost-net,mode=0660
+          - --dev2
+          - path=/dev/vhost-vsock,mode=0660
+          - --dev3
+          - path=/dev/kmsg,mode=0660
+          - --dev4
+          - path=/dev/net/tun,mode=0666
+          - --tags
+          - disposable;kata;lxc;ansible
+      when: not kata_lxc_config.stat.exists
+
+    - name: Read LXC status
+      ansible.builtin.command:
+        cmd: pct status {{ kata_lxc_id }}
+      register: kata_lxc_status
+      changed_when: false
+
+    - name: Read current LXC configuration
+      ansible.builtin.command:
+        cmd: pct config {{ kata_lxc_id }}
+      register: kata_lxc_current_config
+      changed_when: false
+
+    - name: Ensure kubelet kernel log device is present
+      ansible.builtin.command:
+        cmd: pct set {{ kata_lxc_id }} --dev3 path=/dev/kmsg,mode=0660
+      register: kata_lxc_kmsg
+      when: "'/dev/kmsg' not in kata_lxc_current_config.stdout"
+
+    - name: Enable writable sysctls required by kubelet
+      ansible.builtin.command:
+        cmd: >-
+          pct set {{ kata_lxc_id }} --features
+          nesting=1,keyctl=1,fuse=1,mknod=1,force_rw_sys=1
+      register: kata_lxc_rw_sys
+      when: "'force_rw_sys=1' not in kata_lxc_current_config.stdout"
+
+    - name: Ensure TUN device required by Kata networking is present
+      ansible.builtin.command:
+        cmd: pct set {{ kata_lxc_id }} --dev4 path=/dev/net/tun,mode=0666
+      register: kata_lxc_tun
+      when: "'/dev/net/tun' not in kata_lxc_current_config.stdout"
+
+    - name: Configure privileged nested-runtime LXC directives
+      ansible.builtin.lineinfile:
+        path: /etc/pve/lxc/{{ kata_lxc_id }}.conf
+        regexp: "^{{ item.key | regex_escape }}:"
+        line: "{{ item.key }}: {{ item.value }}"
+      loop:
+        - key: lxc.apparmor.profile
+          value: unconfined
+        - key: lxc.cgroup2.devices.allow
+          value: a
+        - key: lxc.cap.drop
+          value: ""
+        - key: lxc.mount.auto
+          value: proc:rw sys:rw
+      register: kata_lxc_raw_config
+
+    - name: Restart LXC after device configuration change
+      ansible.builtin.command:
+        cmd: pct reboot {{ kata_lxc_id }}
+      when: >-
+        kata_lxc_kmsg.changed or kata_lxc_rw_sys.changed or kata_lxc_tun.changed or
+        kata_lxc_raw_config.changed
+
+    - name: Start LXC
+      ansible.builtin.command:
+        cmd: pct start {{ kata_lxc_id }}
+      when: "'status: stopped' in kata_lxc_status.stdout"
+
+    - name: Wait for systemd in LXC
+      ansible.builtin.command:
+        cmd: pct exec {{ kata_lxc_id }} -- systemctl is-system-running --wait
+      register: kata_lxc_systemd
+      retries: 30
+      delay: 2
+      until: kata_lxc_systemd.rc in [0, 1]
+      changed_when: false
+      failed_when: kata_lxc_systemd.rc not in [0, 1]
+
+    - name: Show LXC address and Kata devices
+      ansible.builtin.command:
+        cmd: >-
+          pct exec {{ kata_lxc_id }} -- sh -c
+          'hostname -I; ls -l /dev/kvm /dev/vhost-net /dev/vhost-vsock /dev/net/tun'
+      register: kata_lxc_facts
+      changed_when: false
+
+    - name: Report LXC address and devices
+      ansible.builtin.debug:
+        var: kata_lxc_facts.stdout_lines
@@ -0,0 +1,9 @@
+---
+all:
+  hosts:
+    pve2:
+      ansible_host: 192.168.10.7
+      ansible_user: root
+    kata-lab:
+      ansible_host: 192.168.10.128
+      ansible_user: root
@@ -0,0 +1,3 @@
+---
+# Reuse the proven Kata 4.1.0 installation playbook against the LXC inventory.
+- import_playbook: ../../kata-lab/ansible/kata.yml
@@ -0,0 +1,73 @@
+---
+- name: Verify fuse-overlayfs for Docker inside Kata
+  hosts: kata-lab
+  gather_facts: false
+  tasks:
+    - name: Remove an earlier fuse-overlayfs smoke Pod
+      ansible.builtin.command:
+        cmd: k3s kubectl delete pod kata-fuse-smoke --ignore-not-found --wait=true
+      changed_when: false
+
+    - name: Create Kata Docker Pod using fuse-overlayfs
+      ansible.builtin.shell:
+        cmd: |
+          set -o pipefail
+          k3s kubectl apply -f - <<'EOF'
+          apiVersion: v1
+          kind: Pod
+          metadata:
+            name: kata-fuse-smoke
+          spec:
+            runtimeClassName: kata
+            restartPolicy: Never
+            containers:
+              - name: dockerd
+                image: docker.io/library/docker:27-dind
+                securityContext:
+                  privileged: true
+                env:
+                  - name: DOCKER_TLS_CERTDIR
+                    value: ""
+                command:
+                  - sh
+                  - -c
+                  - |
+                    apk add --no-cache fuse-overlayfs
+                    test -e /dev/fuse || mknod /dev/fuse c 10 229
+                    chmod 0666 /dev/fuse
+                    exec dockerd-entrypoint.sh --storage-driver=fuse-overlayfs
+              - name: client
+                image: docker.io/library/docker:27-cli
+                env:
+                  - name: DOCKER_HOST
+                    value: tcp://127.0.0.1:2375
+                command:
+                  - sh
+                  - -c
+                  - |
+                    until docker info >/dev/null 2>&1; do sleep 1; done
+                    docker info --format 'storage-driver={{ "{{" }}.Driver{{ "}}" }}'
+                    docker run --rm docker.io/library/busybox:1.37 echo fuse-nested-docker-ok
+          EOF
+        executable: /bin/bash
+      changed_when: true
+
+    - name: Wait for the Docker client to finish
+      ansible.builtin.command:
+        cmd: >-
+          k3s kubectl wait --for=jsonpath='{.status.containerStatuses[?(@.name=="client")].state.terminated.exitCode}'=0
+          pod/kata-fuse-smoke --timeout=10m
+      changed_when: false
+
+    - name: Read Docker client result
+      ansible.builtin.command:
+        cmd: k3s kubectl logs kata-fuse-smoke -c client
+      register: kata_fuse_smoke_log
+      changed_when: false
+
+    - name: Require fuse-overlayfs and nested Docker
+      ansible.builtin.assert:
+        that:
+          - "'storage-driver=fuse-overlayfs' in kata_fuse_smoke_log.stdout"
+          - "'fuse-nested-docker-ok' in kata_fuse_smoke_log.stdout"
+        success_msg: Docker used fuse-overlayfs and completed a nested container inside Kata
@@ -0,0 +1,3 @@
+---
+# Reuse the guest-kernel and Docker sidecar smoke tests against the LXC worker.
+- import_playbook: ../../kata-lab/ansible/verify.yml
@@ -0,0 +1,35 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: gitea-actions
+  labels:
+    pod-security.kubernetes.io/enforce: privileged
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: kata-ephemeral-runner-config
+  namespace: gitea-actions
+data:
+  config.yaml: |
+    log:
+      level: info
+    runner:
+      file: /runner-state/.runner
+      capacity: 1
+      envs:
+        DOCKER_HOST: tcp://host.docker.internal:2375
+      timeout: 3h
+      shutdown_timeout: 5m
+      labels:
+        - kata-poc:docker://docker.io/library/alpine:3.22
+    cache:
+      enabled: false
+    container:
+      network: ""
+      options: --add-host=host.docker.internal:host-gateway
+      docker_host: tcp://127.0.0.1:2375
+      require_docker: true
+      docker_timeout: 5m
+      force_pull: true
+      bind_workdir: false
@@ -0,0 +1,111 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  generateName: kata-ephemeral-runner-
+  namespace: gitea-actions
+  labels:
+    app.kubernetes.io/name: kata-ephemeral-runner
+spec:
+  runtimeClassName: kata
+  restartPolicy: Never
+  terminationGracePeriodSeconds: 330
+  automountServiceAccountToken: false
+  initContainers:
+    # Native sidecar: startupProbe gates the runner, and kubelet terminates this
+    # container after the ephemeral runner exits so the Kata sandbox can end.
+    - name: docker
+      image: zot.ad.ddupan.top/panxiao81/dind-fuse:29.7.1
+      imagePullPolicy: Never
+      restartPolicy: Always
+      env:
+        - name: DOCKER_TLS_CERTDIR
+          value: ""
+        - name: DIND_LOOPBACK_SIZE
+          value: 6G
+        - name: DIND_LOOPBACK_FILE
+          value: /loopback/docker.img
+      args:
+        - --host=tcp://0.0.0.0:2375
+        - --tls=false
+        - --mtu=1450
+        - --default-network-opt=bridge=com.docker.network.driver.mtu=1450
+      securityContext:
+        privileged: true
+      resources:
+        requests:
+          cpu: 250m
+          memory: 512Mi
+          ephemeral-storage: 1Gi
+        limits:
+          cpu: "4"
+          memory: 6Gi
+          ephemeral-storage: 8Gi
+      volumeMounts:
+        - name: docker-data
+          mountPath: /var/lib/docker
+        - name: loopback-data
+          mountPath: /loopback
+      startupProbe:
+        tcpSocket:
+          port: 2375
+        periodSeconds: 2
+        failureThreshold: 150
+      readinessProbe:
+        tcpSocket:
+          port: 2375
+        periodSeconds: 5
+  containers:
+    - name: runner
+      image: docker.io/gitea/runner:2.3.0
+      imagePullPolicy: Never
+      env:
+        - name: CONFIG_FILE
+          value: /config/config.yaml
+        - name: DOCKER_HOST
+          value: tcp://127.0.0.1:2375
+        - name: GITEA_INSTANCE_URL
+          value: https://git.ddupan.top
+        - name: GITEA_RUNNER_EPHEMERAL
+          value: "1"
+        - name: GITEA_RUNNER_NAME
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.name
+        - name: GITEA_RUNNER_LABELS
+          value: kata-poc:docker://docker.io/library/alpine:3.22
+        - name: GITEA_RUNNER_REGISTRATION_TOKEN_FILE
+          value: /registration/token
+      resources:
+        requests:
+          cpu: 100m
+          memory: 128Mi
+        limits:
+          cpu: "1"
+          memory: 1Gi
+      volumeMounts:
+        - name: config
+          mountPath: /config
+          readOnly: true
+        - name: registration
+          mountPath: /registration
+          readOnly: true
+        - name: runner-state
+          mountPath: /runner-state
+  volumes:
+    - name: config
+      configMap:
+        name: kata-ephemeral-runner-config
+    - name: registration
+      secret:
+        secretName: gitea-runner-registration
+    - name: runner-state
+      emptyDir:
+        medium: Memory
+        sizeLimit: 16Mi
+    - name: docker-data
+      emptyDir:
+        sizeLimit: 8Gi
+    - name: loopback-data
+      emptyDir:
+        medium: Memory
+        sizeLimit: 6Gi
@@ -0,0 +1,24 @@
+# This file is maintained automatically by "terraform init".
+# Manual edits may be lost in future updates.
+
+provider "registry.terraform.io/bpg/proxmox" {
+  version     = "0.111.1"
+  constraints = "0.111.1"
+  hashes = [
+    "h1:ML2D3UUZTM99yrll/EBXj7wBYMb8xmQgomqFNybEoxY=",
+    "zh:18fb7c31a08dde6bffa1a4d4a211e604d6d17eec7092fd59331b3db3c6f3742c",
+    "zh:1cd60761538289d4dd2a1086b3ae62a7b0bdd4b1a2f824e9a44e243413168dba",
+    "zh:2eb76f6fc8299b6820ff678c8252332cc3366e226b5ae2e61748fd2449c1ed92",
+    "zh:45e6f7ebd0bf48911d37060359a4f359b5743b3092e985295733990e406d0416",
+    "zh:4aa8ba912eae37975d2e983394d173e595ca34fc76b5bf220b37d0e99d76e98c",
+    "zh:58e0789923103a77d502a0a9fc3eb920625e8eb935ec2d4ac0d006aebd1d186c",
+    "zh:6df8aa85fb8865915537e946c19b02538ad188018a629759c213c6f03730f642",
+    "zh:6ed47bc00d0913a1d0880618fa1376115e9edab6b4a658c081061a7f0e4ca360",
+    "zh:c5b10ff4f33df7e4c29e8f1127d49845b561b37b57517e844fb0954d7923d65e",
+    "zh:d016510e14b738499f0db9d9b3aafe82fc6877fb4ab4e9f831fb68a8d70a1385",
+    "zh:d941f394069bbf24351b363da1c64383f487067aaee0a84f9b96476d4912e212",
+    "zh:ddf271dbc2632ae8ffa8de3972f243ee47d260cb2ac90aa784f2746d98e21a0f",
+    "zh:ed0caa3501c42f611b7e9622c9b1df69fd85dc25a3cd88d3076381829688cd62",
+    "zh:f26e0763dbe6a6b2195c94b44696f2110f7f55433dc142839be16b9697fa5597",
+  ]
+}
@@ -0,0 +1,95 @@
+locals {
+  ssh_public_key = trimspace(file(pathexpand(var.ssh_public_key_file)))
+}
+
+data "proxmox_file" "ubuntu_noble" {
+  content_type = "vztmpl"
+  datastore_id = var.template_datastore
+  node_name    = var.proxmox_node
+  file_name    = var.template_file_name
+}
+
+resource "proxmox_virtual_environment_container" "kata_lxc_lab" {
+  node_name = var.proxmox_node
+  vm_id     = var.container_id
+
+  description   = "Disposable privileged LXC for validating k3s and Kata with direct host KVM access."
+  unprivileged  = false
+  started       = true
+  start_on_boot = false
+  tags          = ["disposable", "kata", "lxc", "terraform"]
+
+  cpu {
+    cores = 4
+  }
+
+  memory {
+    dedicated = 8192
+    swap      = 0
+  }
+
+  disk {
+    datastore_id = var.root_datastore
+    size         = 16
+  }
+
+  features {
+    fuse    = true
+    keyctl  = true
+    mknod   = true
+    nesting = true
+  }
+
+  device_passthrough {
+    path = "/dev/kvm"
+    mode = "0660"
+  }
+
+  device_passthrough {
+    path = "/dev/vhost-net"
+    mode = "0660"
+  }
+
+  device_passthrough {
+    path = "/dev/vhost-vsock"
+    mode = "0660"
+  }
+
+  device_passthrough {
+    path = "/dev/kmsg"
+    mode = "0660"
+  }
+
+  device_passthrough {
+    path = "/dev/net/tun"
+    mode = "0666"
+  }
+
+  initialization {
+    hostname = var.container_name
+
+    ip_config {
+      ipv4 {
+        address = "dhcp"
+      }
+    }
+
+    user_account {
+      keys = [local.ssh_public_key]
+    }
+  }
+
+  network_interface {
+    name   = "eth0"
+    bridge = "vmbr0"
+  }
+
+  operating_system {
+    template_file_id = data.proxmox_file.ubuntu_noble.id
+    type             = "ubuntu"
+  }
+
+  wait_for_ip {
+    ipv4 = true
+  }
+}
@@ -0,0 +1,7 @@
+output "container_id" {
+  value = proxmox_virtual_environment_container.kata_lxc_lab.vm_id
+}
+
+output "ipv4" {
+  value = proxmox_virtual_environment_container.kata_lxc_lab.ipv4
+}
@@ -0,0 +1,15 @@
+provider "proxmox" {
+  endpoint  = var.proxmox_endpoint
+  api_token = var.proxmox_api_token
+
+  ssh {
+    agent       = false
+    username    = "root"
+    private_key = file(pathexpand(var.proxmox_ssh_private_key_file))
+
+    node {
+      name    = var.proxmox_node
+      address = var.proxmox_node_address
+    }
+  }
+}
@@ -0,0 +1,54 @@
+variable "proxmox_endpoint" {
+  type    = string
+  default = "https://pve1.ad.ddupan.top:8006"
+}
+
+variable "proxmox_api_token" {
+  type      = string
+  sensitive = true
+}
+
+variable "proxmox_node" {
+  type    = string
+  default = "pve2"
+}
+
+variable "proxmox_node_address" {
+  type    = string
+  default = "192.168.10.7"
+}
+
+variable "proxmox_ssh_private_key_file" {
+  type    = string
+  default = "~/.ssh/id_ed25519"
+}
+
+variable "ssh_public_key_file" {
+  type    = string
+  default = "~/.ssh/id_ed25519.pub"
+}
+
+variable "container_id" {
+  type    = number
+  default = 147
+}
+
+variable "container_name" {
+  type    = string
+  default = "kata-lxc-lab"
+}
+
+variable "template_datastore" {
+  type    = string
+  default = "laptop"
+}
+
+variable "template_file_name" {
+  type    = string
+  default = "ubuntu-24.04-standard_24.04-2_amd64.tar.zst"
+}
+
+variable "root_datastore" {
+  type    = string
+  default = "local-lvm"
+}
@@ -0,0 +1,10 @@
+terraform {
+  required_version = ">= 1.10"
+
+  required_providers {
+    proxmox = {
+      source  = "bpg/proxmox"
+      version = "0.111.1"
+    }
+  }
+}