56 lines
2.5 KiB
Markdown
56 lines
2.5 KiB
Markdown
# OpenSandbox Kata runner
|
|
|
|
The OpenSandbox worker replaces the legacy direct Cloud Hypervisor launcher for
|
|
jobs labelled `self-hosted, vm`. It runs in the sandbox Kubernetes cluster and
|
|
uses two local control planes:
|
|
|
|
1. OpenSandbox Lifecycle API creates a sandbox from the `ci-vm` Pool.
|
|
2. Kubernetes API exposes the concrete BatchSandbox allocation and manages its
|
|
short-lived `ClusterStaticEntry`.
|
|
|
|
## Identity ordering
|
|
|
|
The worker sends the stable repository/job identity and a generated runner name
|
|
as task environment. Once OpenSandbox has allocated a Pool Pod, the worker reads
|
|
the Pod UID and creates an entry with:
|
|
|
|
- parent: `spiffe://ddupan.top/spire/agent/k8s_psat/sandbox-kata/pod/<pod-uid>`;
|
|
- workload: `spiffe://ddupan.top/ci/<owner>/<repository>/<job>`;
|
|
- selector: `unix:uid:2000`.
|
|
|
|
The Pool must run the runner task as UID 2000 and set
|
|
`shareProcessNamespace: true`. Its guest-local SPIRE Agent uses a Pod-bound PSAT
|
|
and exposes the Workload API through the shared `spire-agent-socket` emptyDir.
|
|
The runner image starts through `gitea-opensandbox-runner`, which waits until the
|
|
exact expected SVID is available before it registers with Gitea. This prevents a
|
|
job from starting between Pod allocation and entry reconciliation.
|
|
|
|
The UID selector is the boundary between containers in the same Kata Pod. The
|
|
SPIRE Agent and privileged Docker daemon must not run as UID 2000. The runner may
|
|
access Docker only through a group-owned Unix socket.
|
|
|
|
## Required Pool contract
|
|
|
|
The `ci-vm` Pool template owns infrastructure that callers cannot override in
|
|
Pool mode:
|
|
|
|
- `runtimeClassName: kata-clh-runtime-rs` with block-backed emptyDir storage;
|
|
- runner image containing `gitea-opensandbox-runner` and SPIRE CLI;
|
|
- guest-local SPIRE Agent sidecar and projected audience `spire-server` token;
|
|
- `shareProcessNamespace: true`;
|
|
- runner UID 2000 and a distinct UID for every sidecar;
|
|
- ephemeral Gitea registration token delivery;
|
|
- Docker/BuildKit storage and socket entirely inside the Kata guest.
|
|
|
|
The worker ServiceAccount needs read access to BatchSandboxes and Pods and
|
|
create/get/delete access to ClusterStaticEntries. OpenSandbox API credentials,
|
|
when enabled, are mounted from a Secret and read from
|
|
`OPENSANDBOX_API_KEY_FILE`.
|
|
|
|
## Cleanup
|
|
|
|
On success, failure, timeout, or cancellation the worker deletes the
|
|
ClusterStaticEntry before deleting the sandbox. Both deletes accept `404`, so a
|
|
JetStream redelivery can safely repeat cleanup. The SPIRE Agent registration is
|
|
bound to the Pod UID and is removed by SPIRE after the Pod disappears.
|