fix(spiffe-oidc-discovery-provider): run under restricted PSA/SCC on OpenShift (#920)
* fix(spiffe-oidc-discovery-provider): run under restricted PSA/SCC on OpenShift The OIDC discovery provider does not require any elevated privileges: it runs fine under OpenShift's built-in restricted-v2 SCC (non-root, no privilege escalation, all capabilities dropped, RuntimeDefault seccomp, read-only root filesystem) and mounts only restricted-compatible volumes (csi, configMap, emptyDir, secret, projected, downwardAPI). Despite this, on OpenShift the chart: - downgraded the spire-server namespace from restricted to privileged PSA whenever the OIDC provider was enabled, and - created a fully privileged SecurityContextConstraints (host network/IPC/ PID, privileged container, hostPath, arbitrary seccomp, RunAsAny) bound to the provider's ServiceAccount. Both contradict the chart's own Namespaces documentation, which specifies restricted PSA for spire-server, and violate least privilege for an internet-facing OIDC endpoint. Remove the privileged PSA override for the OIDC provider (spire-server stays restricted; the scc.podSecurityLabelSync=false label is retained) and drop the privileged SCC so the provider falls through to restricted-v2. With spire-server enforcing restricted PSA, the inline-CSI PodSecurity check reads the cluster-scoped CSIDriver's security.openshift.io/csi-ephemeral-volume-profile label. If the CSIDriver is not committed before the spire-server StatefulSet (which mounts the inline upstream.csi.spiffe.io volume) is admitted, the profile defaults to privileged and admission is denied. Under ArgoCD the CSIDriver and the server StatefulSet can land in the same sync wave, racing admission. Annotate the CSIDriver with argocd.argoproj.io/sync-wave: "-1" (OpenShift only) so it is applied before the default-wave server workloads; the annotation is inert for plain helm installs. Signed-off-by: Michael Munch <[email protected]> * ✨ make CSIDriver sync-wave ordering configurable Add syncWave and csiDriverAnnotations values to the spiffe-csi-driver chart so the OpenShift argocd.argoproj.io/sync-wave annotation number can be overridden (e.g. when the chart is nested) and arbitrary annotations can be applied to the CSIDriver. Signed-off-by: Michael Munch <[email protected]> --------- Signed-off-by: Michael Munch <[email protected]> Co-authored-by: kfox1111 <[email protected]>
This commit is contained in:
Michael Munch
co-authored by
kfox1111
kfox1111
parent
0726faa076
commit
ab5e5d8677
@@ -10,9 +10,6 @@
|
|||||||
{{- $labels = mergeOverwrite $labels (include "spire-lib.namespace.default_server_labels" . | fromYaml) }}
|
{{- $labels = mergeOverwrite $labels (include "spire-lib.namespace.default_server_labels" . | fromYaml) }}
|
||||||
{{- if (dig "openshift" false .Values.global) }}
|
{{- if (dig "openshift" false .Values.global) }}
|
||||||
{{- $_ := set $labels "security.openshift.io/scc.podSecurityLabelSync" "false" }}
|
{{- $_ := set $labels "security.openshift.io/scc.podSecurityLabelSync" "false" }}
|
||||||
{{- if (index .Values "spiffe-oidc-discovery-provider").enabled }}
|
|
||||||
{{- $_ := set $labels "pod-security.kubernetes.io/enforce" "privileged" }}
|
|
||||||
{{- end }}
|
|
||||||
{{- end }}
|
{{- end }}
|
||||||
{{- end }}
|
{{- end }}
|
||||||
{{- $labels = mergeOverwrite $labels .Values.global.spire.namespaces.server.labels }}
|
{{- $labels = mergeOverwrite $labels .Values.global.spire.namespaces.server.labels }}
|
||||||
|
|||||||
@@ -25,54 +25,56 @@ A Helm chart to install the SPIFFE CSI driver.
|
|||||||
|
|
||||||
### SPIFFE CSI Driver Chart parameters
|
### SPIFFE CSI Driver Chart parameters
|
||||||
|
|
||||||
| Name | Description | Value |
|
| Name | Description | Value |
|
||||||
| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------- |
|
| --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------- |
|
||||||
| `pluginName` | Set the csi driver name deployed to Kubernetes. | `csi.spiffe.io` |
|
| `pluginName` | Set the csi driver name deployed to Kubernetes. | `csi.spiffe.io` |
|
||||||
| `image.registry` | The OCI registry to pull the image from | `ghcr.io` |
|
| `image.registry` | The OCI registry to pull the image from | `ghcr.io` |
|
||||||
| `image.repository` | The repository within the registry | `spiffe/spiffe-csi-driver` |
|
| `image.repository` | The repository within the registry | `spiffe/spiffe-csi-driver` |
|
||||||
| `image.pullPolicy` | The image pull policy | `IfNotPresent` |
|
| `image.pullPolicy` | The image pull policy | `IfNotPresent` |
|
||||||
| `image.tag` | Overrides the image tag whose default is the chart appVersion | `""` |
|
| `image.tag` | Overrides the image tag whose default is the chart appVersion | `""` |
|
||||||
| `resources` | Resource requests and limits for spiffe-csi-driver and its initContainers | `{}` |
|
| `resources` | Resource requests and limits for spiffe-csi-driver and its initContainers | `{}` |
|
||||||
| `extraEnvVars` | Extra environment variables to be added to the spiffe-csi-driver container | `[]` |
|
| `extraEnvVars` | Extra environment variables to be added to the spiffe-csi-driver container | `[]` |
|
||||||
| `healthChecks.port` | The healthcheck port for spiffe-csi-driver | `9809` |
|
| `healthChecks.port` | The healthcheck port for spiffe-csi-driver | `9809` |
|
||||||
| `updateStrategy.type` | The update strategy to use to replace existing DaemonSet pods with new pods. Can be RollingUpdate or OnDelete. | `RollingUpdate` |
|
| `updateStrategy.type` | The update strategy to use to replace existing DaemonSet pods with new pods. Can be RollingUpdate or OnDelete. | `RollingUpdate` |
|
||||||
| `updateStrategy.rollingUpdate.maxUnavailable` | Max unavailable pods during update. Can be a number or a percentage. | `1` |
|
| `updateStrategy.rollingUpdate.maxUnavailable` | Max unavailable pods during update. Can be a number or a percentage. | `1` |
|
||||||
| `livenessProbe.initialDelaySeconds` | Initial delay seconds for livenessProbe | `5` |
|
| `livenessProbe.initialDelaySeconds` | Initial delay seconds for livenessProbe | `5` |
|
||||||
| `livenessProbe.timeoutSeconds` | Timeout value in seconds for livenessProbe | `5` |
|
| `livenessProbe.timeoutSeconds` | Timeout value in seconds for livenessProbe | `5` |
|
||||||
| `imagePullSecrets` | Image pull secret details for spiffe-csi-driver | `[]` |
|
| `imagePullSecrets` | Image pull secret details for spiffe-csi-driver | `[]` |
|
||||||
| `nameOverride` | Name override for spiffe-csi-driver | `""` |
|
| `nameOverride` | Name override for spiffe-csi-driver | `""` |
|
||||||
| `namespaceOverride` | Namespace to install spiffe-csi-driver | `""` |
|
| `namespaceOverride` | Namespace to install spiffe-csi-driver | `""` |
|
||||||
| `serverNamespaceOverride` | Override the namespace that the spire-server is installed into | `""` |
|
| `serverNamespaceOverride` | Override the namespace that the spire-server is installed into | `""` |
|
||||||
| `validatingAdmissionPolicy.enabled` | When set to auto, the validatingAdmissionPolicy will be enabled when the pluginName == "upstream.csi.spiffe.io" and k8s >= 1.30.0. Valid options are [auto, true, false] | `auto` |
|
| `validatingAdmissionPolicy.enabled` | When set to auto, the validatingAdmissionPolicy will be enabled when the pluginName == "upstream.csi.spiffe.io" and k8s >= 1.30.0. Valid options are [auto, true, false] | `auto` |
|
||||||
| `fullnameOverride` | Full name override for spiffe-csi-driver | `""` |
|
| `fullnameOverride` | Full name override for spiffe-csi-driver | `""` |
|
||||||
| `csiDriverLabels` | Labels to apply to the CSIDriver | `{}` |
|
| `csiDriverLabels` | Labels to apply to the CSIDriver | `{}` |
|
||||||
| `initContainers` | Init Containers to apply to the CSI Driver DaemonSet | `[]` |
|
| `csiDriverAnnotations` | Annotations to apply to the CSIDriver | `{}` |
|
||||||
| `serviceAccount.create` | Specifies whether a service account should be created | `true` |
|
| `syncWave` | The argocd.argoproj.io/sync-wave value applied to the CSIDriver on OpenShift, ensuring the CSI driver reconciles before workloads that depend on its csi-ephemeral-volume-profile label | `-1` |
|
||||||
| `serviceAccount.annotations` | Annotations to add to the service account | `{}` |
|
| `initContainers` | Init Containers to apply to the CSI Driver DaemonSet | `[]` |
|
||||||
| `serviceAccount.name` | The name of the service account to use. If not set and create is true, a name is generated. | `""` |
|
| `serviceAccount.create` | Specifies whether a service account should be created | `true` |
|
||||||
| `podAnnotations` | Pod annotations for spiffe-csi-driver | `{}` |
|
| `serviceAccount.annotations` | Annotations to add to the service account | `{}` |
|
||||||
| `podLabels` | Labels to add to pods | `{}` |
|
| `serviceAccount.name` | The name of the service account to use. If not set and create is true, a name is generated. | `""` |
|
||||||
| `podSecurityContext` | Security context for CSI driver pods | `{}` |
|
| `podAnnotations` | Pod annotations for spiffe-csi-driver | `{}` |
|
||||||
| `securityContext` | Security context for CSI driver containers | `{}` |
|
| `podLabels` | Labels to add to pods | `{}` |
|
||||||
| `hostNetwork` | Enable hostNetwork for the DaemonSet | `false` |
|
| `podSecurityContext` | Security context for CSI driver pods | `{}` |
|
||||||
| `nodeSelector` | Node selector for CSI driver pods | `{}` |
|
| `securityContext` | Security context for CSI driver containers | `{}` |
|
||||||
| `tolerations` | Tolerations for CSI driver pods | `[]` |
|
| `hostNetwork` | Enable hostNetwork for the DaemonSet | `false` |
|
||||||
| `affinity` | Node affinity | `{}` |
|
| `nodeSelector` | Node selector for CSI driver pods | `{}` |
|
||||||
| `nodeDriverRegistrar.image.registry` | The OCI registry to pull the image from | `registry.k8s.io` |
|
| `tolerations` | Tolerations for CSI driver pods | `[]` |
|
||||||
| `nodeDriverRegistrar.image.repository` | The repository within the registry | `sig-storage/csi-node-driver-registrar` |
|
| `affinity` | Node affinity | `{}` |
|
||||||
| `nodeDriverRegistrar.image.pullPolicy` | The image pull policy | `IfNotPresent` |
|
| `nodeDriverRegistrar.image.registry` | The OCI registry to pull the image from | `registry.k8s.io` |
|
||||||
| `nodeDriverRegistrar.image.tag` | Overrides the image tag | `v2.15.0` |
|
| `nodeDriverRegistrar.image.repository` | The repository within the registry | `sig-storage/csi-node-driver-registrar` |
|
||||||
| `nodeDriverRegistrar.extraEnvVars` | Extra environment variables to be added to the nodeDriverRegistrar container | `[]` |
|
| `nodeDriverRegistrar.image.pullPolicy` | The image pull policy | `IfNotPresent` |
|
||||||
| `agentSocketPath` | The unix socket path to the spire-agent | `/run/spire/agent-sockets/spire-agent.sock` |
|
| `nodeDriverRegistrar.image.tag` | Overrides the image tag | `v2.15.0` |
|
||||||
| `kubeletPath` | Path to kubelet file | `/var/lib/kubelet` |
|
| `nodeDriverRegistrar.extraEnvVars` | Extra environment variables to be added to the nodeDriverRegistrar container | `[]` |
|
||||||
| `priorityClassName` | Priority class assigned to daemonset pods. Can be auto set with global.recommendations.priorityClassName. | `""` |
|
| `agentSocketPath` | The unix socket path to the spire-agent | `/run/spire/agent-sockets/spire-agent.sock` |
|
||||||
| `restrictedScc.enabled` | Enables the creation of a SecurityContextConstraint based on the restricted SCC with CSI volume support | `false` |
|
| `kubeletPath` | Path to kubelet file | `/var/lib/kubelet` |
|
||||||
| `restrictedScc.name` | Set the name of the restricted SCC with CSI support | `""` |
|
| `priorityClassName` | Priority class assigned to daemonset pods. Can be auto set with global.recommendations.priorityClassName. | `""` |
|
||||||
| `restrictedScc.version` | Version of the restricted SCC | `2` |
|
| `restrictedScc.enabled` | Enables the creation of a SecurityContextConstraint based on the restricted SCC with CSI volume support | `false` |
|
||||||
| `selinux.enabled` | Enable selinux support | `false` |
|
| `restrictedScc.name` | Set the name of the restricted SCC with CSI support | `""` |
|
||||||
| `selinux.context` | Which selinux context to use | `container_file_t` |
|
| `restrictedScc.version` | Version of the restricted SCC | `2` |
|
||||||
| `selinux.image.registry` | The OCI registry to pull the image from | `registry.access.redhat.com` |
|
| `selinux.enabled` | Enable selinux support | `false` |
|
||||||
| `selinux.image.repository` | The repository within the registry | `ubi10/ubi-minimal` |
|
| `selinux.context` | Which selinux context to use | `container_file_t` |
|
||||||
| `selinux.image.pullPolicy` | The image pull policy | `IfNotPresent` |
|
| `selinux.image.registry` | The OCI registry to pull the image from | `registry.access.redhat.com` |
|
||||||
| `selinux.image.tag` | Overrides the image tag whose default is the chart appVersion | `10.1-1776834797` |
|
| `selinux.image.repository` | The repository within the registry | `ubi10/ubi-minimal` |
|
||||||
|
| `selinux.image.pullPolicy` | The image pull policy | `IfNotPresent` |
|
||||||
|
| `selinux.image.tag` | Overrides the image tag whose default is the chart appVersion | `10.1-1776834797` |
|
||||||
|
|
||||||
|
|||||||
@@ -3,10 +3,19 @@
|
|||||||
{{- $_ := set $labels "security.openshift.io/csi-ephemeral-volume-profile" "restricted" }}
|
{{- $_ := set $labels "security.openshift.io/csi-ephemeral-volume-profile" "restricted" }}
|
||||||
{{- end }}
|
{{- end }}
|
||||||
{{- $labels = mergeOverwrite $labels .Values.csiDriverLabels }}
|
{{- $labels = mergeOverwrite $labels .Values.csiDriverLabels }}
|
||||||
|
{{- $annotations := dict }}
|
||||||
|
{{- if (dig "openshift" false .Values.global) }}
|
||||||
|
{{- $_ := set $annotations "argocd.argoproj.io/sync-wave" (toString .Values.syncWave) }}
|
||||||
|
{{- end }}
|
||||||
|
{{- $annotations = mergeOverwrite $annotations .Values.csiDriverAnnotations }}
|
||||||
apiVersion: storage.k8s.io/v1
|
apiVersion: storage.k8s.io/v1
|
||||||
kind: CSIDriver
|
kind: CSIDriver
|
||||||
metadata:
|
metadata:
|
||||||
name: {{ .Values.pluginName | quote }}
|
name: {{ .Values.pluginName | quote }}
|
||||||
|
{{- with $annotations }}
|
||||||
|
annotations:
|
||||||
|
{{- toYaml . | nindent 4 }}
|
||||||
|
{{- end }}
|
||||||
{{- with $labels }}
|
{{- with $labels }}
|
||||||
labels:
|
labels:
|
||||||
{{- toYaml . | nindent 4 }}
|
{{- toYaml . | nindent 4 }}
|
||||||
|
|||||||
@@ -76,6 +76,12 @@ fullnameOverride: ""
|
|||||||
## @param csiDriverLabels Labels to apply to the CSIDriver
|
## @param csiDriverLabels Labels to apply to the CSIDriver
|
||||||
csiDriverLabels: {}
|
csiDriverLabels: {}
|
||||||
|
|
||||||
|
## @param csiDriverAnnotations Annotations to apply to the CSIDriver
|
||||||
|
csiDriverAnnotations: {}
|
||||||
|
|
||||||
|
## @param syncWave The argocd.argoproj.io/sync-wave value applied to the CSIDriver on OpenShift, ensuring the CSI driver reconciles before workloads that depend on its csi-ephemeral-volume-profile label
|
||||||
|
syncWave: -1
|
||||||
|
|
||||||
## @param initContainers Init Containers to apply to the CSI Driver DaemonSet
|
## @param initContainers Init Containers to apply to the CSI Driver DaemonSet
|
||||||
initContainers: []
|
initContainers: []
|
||||||
|
|
||||||
|
|||||||
-42
@@ -1,42 +0,0 @@
|
|||||||
{{- if eq (.Values.global.openshift | toString) "true" }}
|
|
||||||
apiVersion: security.openshift.io/v1
|
|
||||||
kind: SecurityContextConstraints
|
|
||||||
metadata:
|
|
||||||
name: {{ include "spiffe-oidc-discovery-provider.fullname" . }}
|
|
||||||
readOnlyRootFilesystem: true
|
|
||||||
runAsUser:
|
|
||||||
type: RunAsAny
|
|
||||||
seLinuxContext:
|
|
||||||
type: RunAsAny
|
|
||||||
supplementalGroups:
|
|
||||||
type: RunAsAny
|
|
||||||
users:
|
|
||||||
- system:serviceaccount:{{ include "spiffe-oidc-discovery-provider.namespace" . }}:{{ include "spiffe-oidc-discovery-provider.serviceAccountName" . }}
|
|
||||||
- system:serviceaccount:{{ include "spiffe-oidc-discovery-provider.namespace" . }}:{{ include "spiffe-oidc-discovery-provider.serviceAccountName" . }}-pre-delete
|
|
||||||
volumes:
|
|
||||||
- configMap
|
|
||||||
- csi
|
|
||||||
- downwardAPI
|
|
||||||
- emptyDir
|
|
||||||
- ephemeral
|
|
||||||
- hostPath
|
|
||||||
- projected
|
|
||||||
- secret
|
|
||||||
allowedCapabilities: null
|
|
||||||
allowHostDirVolumePlugin: true
|
|
||||||
allowHostIPC: true
|
|
||||||
allowHostNetwork: true
|
|
||||||
allowHostPID: true
|
|
||||||
allowHostPorts: true
|
|
||||||
allowPrivilegeEscalation: true
|
|
||||||
allowPrivilegedContainer: true
|
|
||||||
defaultAddCapabilities: null
|
|
||||||
fsGroup:
|
|
||||||
type: RunAsAny
|
|
||||||
groups: []
|
|
||||||
priority: null
|
|
||||||
requiredDropCapabilities: null
|
|
||||||
seccompProfiles:
|
|
||||||
- '*'
|
|
||||||
|
|
||||||
{{ end }}
|
|
||||||
@@ -349,6 +349,43 @@ spire-server:
|
|||||||
Expect(objs[serverTmpl]).Should(ContainSubstring("init-jwt-svid-exec"))
|
Expect(objs[serverTmpl]).Should(ContainSubstring("init-jwt-svid-exec"))
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
Describe("spiffe-csi-driver.syncWave", func() {
|
||||||
|
csiTmpl := "spire/charts/spiffe-csi-driver/templates/spiffe-csi-driver.yaml"
|
||||||
|
It("renders the default sync-wave annotation on OpenShift", func() {
|
||||||
|
objs, err := ValueStringRender(chart, `
|
||||||
|
global:
|
||||||
|
openshift: true
|
||||||
|
`)
|
||||||
|
Expect(err).Should(Succeed())
|
||||||
|
Expect(objs[csiTmpl]).Should(ContainSubstring(`argocd.argoproj.io/sync-wave: "-1"`))
|
||||||
|
})
|
||||||
|
It("allows overriding the sync-wave number", func() {
|
||||||
|
objs, err := ValueStringRender(chart, `
|
||||||
|
global:
|
||||||
|
openshift: true
|
||||||
|
spiffe-csi-driver:
|
||||||
|
syncWave: -2
|
||||||
|
`)
|
||||||
|
Expect(err).Should(Succeed())
|
||||||
|
Expect(objs[csiTmpl]).Should(ContainSubstring(`argocd.argoproj.io/sync-wave: "-2"`))
|
||||||
|
})
|
||||||
|
It("allows overriding the annotation via csiDriverAnnotations", func() {
|
||||||
|
objs, err := ValueStringRender(chart, `
|
||||||
|
global:
|
||||||
|
openshift: true
|
||||||
|
spiffe-csi-driver:
|
||||||
|
csiDriverAnnotations:
|
||||||
|
argocd.argoproj.io/sync-wave: "-5"
|
||||||
|
`)
|
||||||
|
Expect(err).Should(Succeed())
|
||||||
|
Expect(objs[csiTmpl]).Should(ContainSubstring(`argocd.argoproj.io/sync-wave: "-5"`))
|
||||||
|
})
|
||||||
|
It("does not render the sync-wave annotation when not on OpenShift", func() {
|
||||||
|
objs, err := ValueStringRender(chart, ``)
|
||||||
|
Expect(err).Should(Succeed())
|
||||||
|
Expect(objs[csiTmpl]).ShouldNot(ContainSubstring("argocd.argoproj.io/sync-wave"))
|
||||||
|
})
|
||||||
|
})
|
||||||
Describe("spire-server.externalServerSubject", func() {
|
Describe("spire-server.externalServerSubject", func() {
|
||||||
It("binds the external server's downstream RBAC to a ServiceAccount subject", func() {
|
It("binds the external server's downstream RBAC to a ServiceAccount subject", func() {
|
||||||
objs, err := ValueStringRender(chart, `
|
objs, err := ValueStringRender(chart, `
|
||||||
|
|||||||
Reference in New Issue
Block a user