Add Configurable Kubelet Address for SPIRE Agent (#709)
* Add kubeletAddress.mode configuration to spire-agent Introduces new enum-based configuration for kubelet connection modes: - auto (default): hostname for OpenShift, localhost otherwise - localhost: SPIRE default behavior (127.0.0.1:10250) - hostname: Connect via node hostname - hostip: Connect via node IP - custom: User-provided configuration Deprecates kubeletConnectByHostname but maintains backward compatibility. Signed-off-by: Oliver Bassett <[email protected]> * Replace connect-by-hostname helper with mode resolution Adds three new helpers: - spire-agent.kubelet-address-mode: Determine mode with backward compat - spire-agent.kubelet-address-mode-resolved: Resolve auto to actual mode - spire-agent.should-set-node-name-env: Determine if node_name_env needed Includes validation of enum values and maintains backward compatibility by keeping the old connect-by-hostname helper as deprecated. Signed-off-by: Oliver Bassett <[email protected]> * Update daemonset to use KUBELET_ADDR env variable - Sets KUBELET_ADDR from downward API for hostname/hostip modes - Maintains MY_NODE_NAME for backward compatibility - No env var set for localhost mode (SPIRE default) - Custom mode allows user control via extraEnvVars - Updates init container env to support both hostname and hostip modes Signed-off-by: Oliver Bassett <[email protected]> * Update workload attestor config and add validation - Changes node_name_env from MY_NODE_NAME to KUBELET_ADDR - Adds validation for kubeletAddress.mode enum - Prevents using both old and new config simultaneously Signed-off-by: Oliver Bassett <[email protected]> * Improve documentation for custom mode Clarifies that custom mode does not validate KUBELET_ADDR presence, allowing for external secret injection and other advanced configuration methods. Signed-off-by: Oliver Bassett <[email protected]> * Fix backward compatibility for kubeletConnectByHostname Two critical fixes for backward compatibility: 1. Helper template priority: Reorder kubelet-address-mode helper to prioritize kubeletConnectByHostname when kubeletAddress.mode is 'auto' or empty. This ensures deprecated config still works. 2. Type-safe validation: Convert kubeletConnectByHostname to string in validation and helper to handle both boolean and string types consistently. Original chart required string type. 3. Smart dual-config validation: Only fail when both configs are explicitly set to non-default values. Allow kubeletConnectByHostname with mode='auto' for backward compatibility. Tested scenarios: - kubeletConnectByHostname='true' maps to hostname mode - kubeletConnectByHostname='false' maps to localhost mode - Both set with mode='auto' allows backward compat to take priority - Both set with different non-defaults triggers validation error - OpenShift auto mode correctly resolves to hostname mode Signed-off-by: Oliver Bassett <[email protected]> * Use parentheses for DEPRECATED tag in values.yaml Change [DEPRECATED] to (DEPRECATED) to avoid conflicts with automated README generator which uses square brackets for special tags. Signed-off-by: Oliver Bassett <[email protected]> * Update generated README documentation Regenerate README.md from values.yaml using documentation generator. Includes new kubeletAddress.mode configuration and deprecation notice for kubeletConnectByHostname. Signed-off-by: Oliver Bassett <[email protected]> * Remove MY_NODE_NAME environment variable Remove MY_NODE_NAME as it is not used within the spire-agent chart. Initially kept for backwards compatibility concerns, but confirmed unnecessary after review. The KUBELET_ADDR environment variable is sufficient for the workload attestor configuration via node_name_env setting. Addresses PR feedback: https://github.com/spiffe/helm-charts-hardened/pull/709#discussion_r1909855869 Signed-off-by: Oliver Bassett <[email protected]> * Fix init container for custom kubelet address mode Address PR #709 feedback by standardizing on KUBELET_ADDR environment variable and passing extraEnvVars to init containers. Changes: 1. Init container env variable: - Renamed NODE_NAME to KUBELET_ADDR for consistency - Made hostip check explicit with 'else if' - Passes extraEnvVars to init container for custom mode support 2. Init container script: - Updated URL construction to use KUBELET_ADDR for all modes - Added validation for custom mode: fails with clear error if KUBELET_ADDR is not set via extraEnvVars - hostname/hostip modes: Use KUBELET_ADDR from downward API - custom mode: Use KUBELET_ADDR from extraEnvVars with validation - localhost mode: Use hardcoded 'localhost' 3. Documentation updates: - Updated custom mode docs to explain extraEnvVars is passed to both main and init containers - Noted init container validation behavior - Updated extraEnvVars param docs to mention init containers Testing verified: - Template rendering for all modes (hostname, hostip, custom, localhost) - Runtime validation: deployed custom mode without KUBELET_ADDR to kind cluster, init container correctly failed with clear error message Addresses: https://github.com/spiffe/helm-charts-hardened/pull/709#discussion_r1909855869 Signed-off-by: Oliver Bassett <[email protected]> * Update generated README for init container changes Regenerate README.md to reflect that extraEnvVars is now passed to both the main container and init containers. Signed-off-by: Oliver Bassett <[email protected]> --------- Signed-off-by: Oliver Bassett <[email protected]> Co-authored-by: kfox1111 <[email protected]>
This commit is contained in:
Oliver Bassett
co-authored by
kfox1111
kfox1111
parent
894cbb1089
commit
97c383b1cb
@@ -116,7 +116,8 @@ A Helm chart to install the SPIRE agent.
|
||||
| `telemetry.datadog.enabled` | Flag to enable datadog monitoring | `false` |
|
||||
| `telemetry.datadog.address` | The address of the datadog service to send metrics to. The default URL for services are `<service-name>.<namespace>.svc` | `datadog.kube-system.svc` |
|
||||
| `telemetry.datadog.port` | The port of the datadog service to send metrics to | `8125` |
|
||||
| `kubeletConnectByHostname` | If true, connect to kubelet using the nodes hostname. If false, uses localhost. If unset, defaults to true on OpenShift and false otherwise. | `""` |
|
||||
| `kubeletConnectByHostname` | (DEPRECATED) Use kubeletAddress.mode instead. If true, connect to kubelet using the nodes hostname. If false, uses localhost. If unset, defaults to true on OpenShift and false otherwise. | `""` |
|
||||
| `kubeletAddress.mode` | How to connect to kubelet for workload attestation | `auto` |
|
||||
| `socketPath` | The unix socket path to the spire-agent | `/run/spire/agent-sockets/spire-agent.sock` |
|
||||
| `socketAlternate.names` | List of alternate names for the socket that workloads might expect to be able to access in the driver mount. | `["socket","spire-agent.sock","api.sock"]` |
|
||||
| `socketAlternate.image.registry` | The OCI registry to pull the image from | `cgr.dev` |
|
||||
@@ -128,7 +129,7 @@ A Helm chart to install the SPIRE agent.
|
||||
| `hostCert.image.pullPolicy` | The image pull policy | `IfNotPresent` |
|
||||
| `hostCert.image.tag` | Overrides the image tag whose default is the chart appVersion | `latest@sha256:abd9d6efa507b657a2fba77e9d6bdeaa3e5dcbc143749f7da62e88cef9b6b629` |
|
||||
| `priorityClassName` | Priority class assigned to daemonset pods. Can be auto set with global.recommendations.priorityClassName. | `""` |
|
||||
| `extraEnvVars` | Extra environment variables to be added to the Spire Agent container | `[]` |
|
||||
| `extraEnvVars` | Extra environment variables to be added to the Spire Agent container and init containers | `[]` |
|
||||
| `extraVolumes` | Extra volumes to be mounted on Spire Agent pods | `[]` |
|
||||
| `extraVolumeMounts` | Extra volume mounts for Spire Agent pods | `[]` |
|
||||
| `extraContainers` | Additional containers to create with Spire Agent pods | `[]` |
|
||||
|
||||
@@ -114,14 +114,66 @@ Create the name of the service account to use
|
||||
{{- print .Values.socketPath }}
|
||||
{{- end }}
|
||||
|
||||
{{- define "spire-agent.connect-by-hostname" -}}
|
||||
{{- if ne .Values.kubeletConnectByHostname "" }}
|
||||
{{- if eq (.Values.kubeletConnectByHostname | toString) "true" }}
|
||||
{{- printf "true" }}
|
||||
{{- else }}
|
||||
{{- printf "false" }}
|
||||
{{/*
|
||||
Determine the kubelet address mode (handles backward compatibility)
|
||||
Returns: auto, localhost, hostname, hostip, or custom
|
||||
Priority:
|
||||
1. If kubeletAddress.mode is set to non-default (not auto/empty), use it
|
||||
2. Else if kubeletConnectByHostname is set, use it (maps to hostname/localhost)
|
||||
3. Else default to auto
|
||||
*/}}
|
||||
{{- define "spire-agent.kubelet-address-mode" -}}
|
||||
{{- if and (hasKey .Values "kubeletAddress") (ne .Values.kubeletAddress.mode "") (ne .Values.kubeletAddress.mode "auto") }}
|
||||
{{- if not (has .Values.kubeletAddress.mode (list "auto" "localhost" "hostname" "hostip" "custom")) }}
|
||||
{{- fail (printf "kubeletAddress.mode must be one of [auto, localhost, hostname, hostip, custom], got: %s" .Values.kubeletAddress.mode) }}
|
||||
{{- end }}
|
||||
{{- else if (dig "openshift" false .Values.global) }}
|
||||
{{- .Values.kubeletAddress.mode }}
|
||||
{{- else if ne (.Values.kubeletConnectByHostname | toString) "" }}
|
||||
{{- if eq (.Values.kubeletConnectByHostname | toString) "true" }}
|
||||
{{- printf "hostname" }}
|
||||
{{- else }}
|
||||
{{- printf "localhost" }}
|
||||
{{- end }}
|
||||
{{- else }}
|
||||
{{- printf "auto" }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Resolve auto mode to actual mode based on platform
|
||||
Returns: localhost, hostname, hostip, or custom (never auto)
|
||||
*/}}
|
||||
{{- define "spire-agent.kubelet-address-mode-resolved" -}}
|
||||
{{- $mode := include "spire-agent.kubelet-address-mode" . }}
|
||||
{{- if eq $mode "auto" }}
|
||||
{{- if (dig "openshift" false .Values.global) }}
|
||||
{{- printf "hostname" }}
|
||||
{{- else }}
|
||||
{{- printf "localhost" }}
|
||||
{{- end }}
|
||||
{{- else }}
|
||||
{{- $mode }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
Check if node_name_env should be set in workload attestor config
|
||||
Returns: "true" if we should set it, empty string otherwise
|
||||
*/}}
|
||||
{{- define "spire-agent.should-set-node-name-env" -}}
|
||||
{{- $resolvedMode := include "spire-agent.kubelet-address-mode-resolved" . }}
|
||||
{{- if or (eq $resolvedMode "hostname") (eq $resolvedMode "hostip") (eq $resolvedMode "custom") }}
|
||||
{{- printf "true" }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
{{/*
|
||||
DEPRECATED: Use spire-agent.kubelet-address-mode-resolved instead
|
||||
Kept for backward compatibility
|
||||
*/}}
|
||||
{{- define "spire-agent.connect-by-hostname" -}}
|
||||
{{- $resolvedMode := include "spire-agent.kubelet-address-mode-resolved" . }}
|
||||
{{- if or (eq $resolvedMode "hostname") (eq $resolvedMode "hostip") }}
|
||||
{{- printf "true" }}
|
||||
{{- else }}
|
||||
{{- printf "false" }}
|
||||
|
||||
@@ -25,6 +25,16 @@
|
||||
{{- if hasPrefix (.Values.socketPath | dir | clean) (.Values.sockets.hostBasePath | clean) }}
|
||||
{{- fail "The sockets.hostBasePath can not be located under the socketPath directory" }}
|
||||
{{- end }}
|
||||
{{- /* Validate kubeletAddress.mode */ -}}
|
||||
{{- if and (hasKey .Values "kubeletAddress") (ne .Values.kubeletAddress.mode "") }}
|
||||
{{- if not (has .Values.kubeletAddress.mode (list "auto" "localhost" "hostname" "hostip" "custom")) }}
|
||||
{{- fail (printf "kubeletAddress.mode must be one of [auto, localhost, hostname, hostip, custom], got: %s" .Values.kubeletAddress.mode) }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- /* Prevent using both old and new config */ -}}
|
||||
{{- if and (ne (.Values.kubeletConnectByHostname | toString) "") (and (hasKey .Values "kubeletAddress") (ne .Values.kubeletAddress.mode "") (ne .Values.kubeletAddress.mode "auto")) }}
|
||||
{{- fail "Both kubeletConnectByHostname (deprecated) and kubeletAddress.mode are set. Please use only kubeletAddress.mode." }}
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- define "spire-agent.yaml-config" -}}
|
||||
agent:
|
||||
@@ -149,8 +159,8 @@ plugins:
|
||||
disable_container_selectors: {{ eq .Values.workloadAttestors.k8s.disableContainerSelectors true}}
|
||||
use_new_container_locator: {{ eq .Values.workloadAttestors.k8s.useNewContainerLocator true }}
|
||||
verbose_container_locator_logs: {{ eq .Values.workloadAttestors.k8s.verboseContainerLocatorLogs true }}
|
||||
{{- if eq (include "spire-agent.connect-by-hostname" .) "true" }}
|
||||
node_name_env: "MY_NODE_NAME"
|
||||
{{- if eq (include "spire-agent.should-set-node-name-env" .) "true" }}
|
||||
node_name_env: "KUBELET_ADDR"
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
|
||||
|
||||
@@ -27,7 +27,8 @@
|
||||
{{- $podSecurityContext = mergeOverwrite (dict "runAsUser" 0 "runAsGroup" 0) .Values.podSecurityContext }}
|
||||
{{- $_ := set $mainSecurityContext "privileged" true }}
|
||||
{{- end }}
|
||||
{{- $cbh := eq (include "spire-agent.connect-by-hostname" .) "true" }}
|
||||
{{- $resolvedMode := include "spire-agent.kubelet-address-mode-resolved" . }}
|
||||
{{- $cbh := or (eq $resolvedMode "hostname") (eq $resolvedMode "hostip") }}
|
||||
{{- $socketAlternateNames := index (include "spire-agent.socket-alternate-names" . | fromYaml) "names" }}
|
||||
{{- $socketPath := include "spire-agent.socket-path" . }}
|
||||
---
|
||||
@@ -105,7 +106,13 @@ spec:
|
||||
else
|
||||
{{- if eq .Values.workloadAttestors.k8s.verification.type "auto" }}
|
||||
{{- if $cbh }}
|
||||
URL="https://$NODE_NAME:10250/spec/"
|
||||
URL="https://$KUBELET_ADDR:10250/spec/"
|
||||
{{- else if eq $resolvedMode "custom" }}
|
||||
if [ -z "$KUBELET_ADDR" ]; then
|
||||
echo "ERROR: kubeletAddress.mode=custom requires KUBELET_ADDR environment variable to be set via extraEnvVars"
|
||||
exit 1
|
||||
fi
|
||||
URL="https://$KUBELET_ADDR:10250/spec/"
|
||||
{{- else }}
|
||||
URL="https://localhost:10250/spec/"
|
||||
{{- end }}
|
||||
@@ -128,10 +135,17 @@ spec:
|
||||
{{- toYaml .Values.resources | nindent 12 }}
|
||||
env:
|
||||
{{- if $cbh }}
|
||||
- name: NODE_NAME
|
||||
- name: KUBELET_ADDR
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
{{- if eq $resolvedMode "hostname" }}
|
||||
fieldPath: spec.nodeName
|
||||
{{- else if eq $resolvedMode "hostip" }}
|
||||
fieldPath: status.hostIP
|
||||
{{- end }}
|
||||
{{- end }}
|
||||
{{- with .Values.extraEnvVars }}
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
volumeMounts:
|
||||
- name: host-cert-isolated
|
||||
@@ -245,12 +259,19 @@ spec:
|
||||
env:
|
||||
- name: PATH
|
||||
value: "/opt/spire/bin:/bin"
|
||||
{{- if $cbh }}
|
||||
- name: MY_NODE_NAME
|
||||
{{- if eq $resolvedMode "hostname" }}
|
||||
- name: KUBELET_ADDR
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: spec.nodeName
|
||||
{{- else if eq $resolvedMode "hostip" }}
|
||||
- name: KUBELET_ADDR
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: status.hostIP
|
||||
{{- end }}
|
||||
{{- /* Note: For localhost and custom modes, KUBELET_ADDR is not set here */ -}}
|
||||
{{- /* For custom mode, user sets KUBELET_ADDR via extraEnvVars */ -}}
|
||||
{{- with .Values.extraEnvVars }}
|
||||
{{- toYaml . | nindent 12 }}
|
||||
{{- end }}
|
||||
|
||||
@@ -276,9 +276,24 @@ telemetry:
|
||||
## @param telemetry.datadog.port The port of the datadog service to send metrics to
|
||||
port: 8125
|
||||
|
||||
## @param kubeletConnectByHostname If true, connect to kubelet using the nodes hostname. If false, uses localhost. If unset, defaults to true on OpenShift and false otherwise.
|
||||
## @param kubeletConnectByHostname (DEPRECATED) Use kubeletAddress.mode instead. If true, connect to kubelet using the nodes hostname. If false, uses localhost. If unset, defaults to true on OpenShift and false otherwise.
|
||||
kubeletConnectByHostname: ""
|
||||
|
||||
kubeletAddress:
|
||||
## @param kubeletAddress.mode How to connect to kubelet for workload attestation
|
||||
## Valid options: [auto, localhost, hostname, hostip, custom]
|
||||
## - auto: hostname for OpenShift, localhost otherwise (default)
|
||||
## - localhost: Use SPIRE default (connects to 127.0.0.1:10250)
|
||||
## - hostname: Connect via node hostname using downward API
|
||||
## - hostip: Connect via node IP using downward API
|
||||
## - custom: Set KUBELET_ADDR manually via extraEnvVars or other injection mechanisms.
|
||||
## Note: extraEnvVars is passed to both the main container and init containers, so
|
||||
## KUBELET_ADDR will be available for certificate verification if using verification
|
||||
## type "auto" or "hostCert". The init container will validate that KUBELET_ADDR is set
|
||||
## and fail with a clear error message if missing. Users must ensure KUBELET_ADDR is
|
||||
## provided at runtime or SPIRE agent will fail to start.
|
||||
mode: auto
|
||||
|
||||
## @param socketPath The unix socket path to the spire-agent
|
||||
socketPath: /run/spire/agent-sockets/spire-agent.sock
|
||||
|
||||
@@ -315,7 +330,7 @@ hostCert:
|
||||
## @param priorityClassName Priority class assigned to daemonset pods. Can be auto set with global.recommendations.priorityClassName.
|
||||
priorityClassName: ""
|
||||
|
||||
## @param extraEnvVars [array] Extra environment variables to be added to the Spire Agent container
|
||||
## @param extraEnvVars [array] Extra environment variables to be added to the Spire Agent container and init containers
|
||||
extraEnvVars: []
|
||||
|
||||
## @param extraVolumes [array] Extra volumes to be mounted on Spire Agent pods
|
||||
|
||||
Reference in New Issue
Block a user