部署 dynamic runner placement v2 #154
@@ -4,6 +4,9 @@ metadata:
|
|||||||
name: dynamic-runner
|
name: dynamic-runner
|
||||||
namespace: flux-system
|
namespace: flux-system
|
||||||
spec:
|
spec:
|
||||||
|
# Keep the v2 rollout paused while the merge commit's v1 CI drains. Remove
|
||||||
|
# this after the controller has been resumed and the placement smokes pass.
|
||||||
|
suspend: true
|
||||||
dependsOn:
|
dependsOn:
|
||||||
- name: external-secrets
|
- name: external-secrets
|
||||||
- name: spire
|
- name: spire
|
||||||
|
|||||||
@@ -1,10 +1,10 @@
|
|||||||
# Gitea dynamic runner
|
# Gitea dynamic runner
|
||||||
|
|
||||||
本目录部署单副本 Go controller,在同一进程运行 RunnerService scheduler、原生
|
本目录部署单副本 Go controller,在同一进程运行 RunnerService scheduler、原生
|
||||||
Kubernetes Pod worker、OpenSandbox VM worker 和 SPIFFE mTLS facade:
|
Kubernetes worker、OpenSandbox worker 和 SPIFFE mTLS facade:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
Gitea RunnerService -> scheduler -> JetStream ci.runner.pod
|
Gitea RunnerService -> scheduler -> JetStream ci.runner.container.kubernetes
|
||||||
|
|
|
|
||||||
v
|
v
|
||||||
homelab Kubernetes Pod
|
homelab Kubernetes Pod
|
||||||
@@ -15,7 +15,8 @@ Gitea RunnerService -> scheduler -> JetStream ci.runner.pod
|
|||||||
Gitea
|
Gitea
|
||||||
```
|
```
|
||||||
|
|
||||||
Pod backend 不经过 OpenSandbox。VM backend 后续启用时才访问 VyOS 暴露的
|
`container+kubernetes` placement 不经过 OpenSandbox;`vm+opensandbox` placement
|
||||||
|
访问 VyOS 暴露的
|
||||||
OpenSandbox Lifecycle API;本目录不修改 sandbox 平台侧 ESO、Bao Terraform 或
|
OpenSandbox Lifecycle API;本目录不修改 sandbox 平台侧 ESO、Bao Terraform 或
|
||||||
OpenSandbox chart 所有权边界。
|
OpenSandbox chart 所有权边界。
|
||||||
|
|
||||||
@@ -82,9 +83,9 @@ kubectl -n dynamic-runner rollout status deploy/dynamic-runner-controller --time
|
|||||||
kubectl -n dynamic-runner logs deploy/dynamic-runner-controller -f
|
kubectl -n dynamic-runner logs deploy/dynamic-runner-controller -f
|
||||||
```
|
```
|
||||||
|
|
||||||
确认 scheduler 只领取一条 `[self-hosted,pod]` task,然后验证:
|
确认 scheduler 只领取一条 `[self-hosted,container,kubernetes]` task,然后验证:
|
||||||
|
|
||||||
1. assignment message 进入并离开 durable `pod` consumer;
|
1. assignment message 进入并离开 durable `container.kubernetes` consumer;
|
||||||
2. `gitea-task-<task-id>` Pod 创建,取得实际 Pod UID;
|
2. `gitea-task-<task-id>` Pod 创建,取得实际 Pod UID;
|
||||||
3. 同名 `ClusterStaticEntry` 的 parent ID 包含该 UID,SPIFFE ID 使用 repository/job key;
|
3. 同名 `ClusterStaticEntry` 的 parent ID 包含该 UID,SPIFFE ID 使用 repository/job key;
|
||||||
4. 官方 runner v3.5.0 经 facade claim 精确 task,Gitea 实时收到日志和终态;
|
4. 官方 runner v3.5.0 经 facade claim 精确 task,Gitea 实时收到日志和终态;
|
||||||
@@ -96,7 +97,8 @@ ClusterStaticEntry 全部消失;完整自动清理通过前不得提高 `POD_C
|
|||||||
`VM_CAPACITY`。
|
`VM_CAPACITY`。
|
||||||
|
|
||||||
VM backend 已通过 `vm-dev` canary 完成 Docker、kind、SPIFFE 和完整生命周期测试。
|
VM backend 已通过 `vm-dev` canary 完成 Docker、kind、SPIFFE 和完整生命周期测试。
|
||||||
生产调度标签为 `[self-hosted, vm]`,初始保持 `VM_CAPACITY=1`;扩容前先观察实际任务的
|
生产规范标签为 `[self-hosted, vm, opensandbox]`,旧 `[self-hosted, vm]` 仍严格映射到
|
||||||
|
同一 placement。初始保持 `VM_CAPACITY=1`;扩容前先观察实际任务的
|
||||||
资源水位、等待时间以及 OpenSandbox 是否存在 terminal sandbox 残留。
|
资源水位、等待时间以及 OpenSandbox 是否存在 terminal sandbox 残留。
|
||||||
|
|
||||||
## 回滚
|
## 回滚
|
||||||
|
|||||||
@@ -38,12 +38,12 @@ spec:
|
|||||||
mountPath: /trust
|
mountPath: /trust
|
||||||
containers:
|
containers:
|
||||||
- name: controller
|
- name: controller
|
||||||
image: zot.ad.ddupan.top/panxiao81/gitea-dynamic-runner-controller@sha256:7374a08a238ca1c76a9aee33c57520ae0edaa9b5b443708487d5b3a69a61da4e
|
image: zot.ad.ddupan.top/panxiao81/gitea-dynamic-runner-controller@sha256:17edfa094b0fa7fcbe5cd39301c3bfc662431860bb99390c34b05b59bc7af3d7
|
||||||
imagePullPolicy: IfNotPresent
|
imagePullPolicy: IfNotPresent
|
||||||
args: [controller]
|
args: [controller]
|
||||||
env:
|
env:
|
||||||
- name: COMPONENTS
|
- name: COMPONENTS
|
||||||
value: scheduler,pod-worker,vm-worker
|
value: scheduler,kubernetes-worker,opensandbox-worker
|
||||||
- name: GITEA_INSTANCE_URL
|
- name: GITEA_INSTANCE_URL
|
||||||
value: https://git.ddupan.top
|
value: https://git.ddupan.top
|
||||||
- name: GITEA_RUNNER_UUID_FILE
|
- name: GITEA_RUNNER_UUID_FILE
|
||||||
|
|||||||
@@ -35,10 +35,11 @@ ci_worker_password
|
|||||||
|
|
||||||
## CI stream 约定
|
## CI stream 约定
|
||||||
|
|
||||||
controller 首次启动时幂等创建 `CI_RUNNER` stream:`ci.runner.>`、
|
controller 使用 `CI_RUNNER` stream:`ci.runner.>`、
|
||||||
`WorkQueuePolicy`、file storage、24h/10000 条/256 MiB 上限。每类 runner 使用独立
|
`WorkQueuePolicy`、file storage、24h/10000 条/256 MiB 上限。每类 runner 使用独立
|
||||||
subject 和 durable pull consumer;`ci.runner.<backend>.binding` 传递 runner 实际
|
placement subject 和 durable pull consumer;assignment v2 subject 为
|
||||||
领取任务后的身份绑定。同类型的多个 worker 共享 durable consumer。ACK 后消息立即
|
`ci.runner.<workload-class>.<driver>`,当前为 `ci.runner.container.kubernetes` 与
|
||||||
|
`ci.runner.vm.opensandbox`。同 placement 的多个 worker 共享 durable consumer。ACK 后消息立即
|
||||||
删除,不保存 CI 历史。
|
删除,不保存 CI 历史。
|
||||||
|
|
||||||
## 验证
|
## 验证
|
||||||
|
|||||||
Reference in New Issue
Block a user