Implement easy Bottom Turtle HA support in the charts (#816)

* Implement easy Bottom Turtle HA support in the charts

Signed-off-by: Kevin Fox <[email protected]>

* Add diagram

Signed-off-by: Kevin Fox <[email protected]>

* Update docs

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes and tightened defaults

Signed-off-by: Kevin Fox <[email protected]>

* More diagrams

Signed-off-by: Kevin Fox <[email protected]>

* More instructions

Signed-off-by: Kevin Fox <[email protected]>

* More instructions

Signed-off-by: Kevin Fox <[email protected]>

* More instructions

Signed-off-by: Kevin Fox <[email protected]>

* More instructions

Signed-off-by: Kevin Fox <[email protected]>

* More instructions

Signed-off-by: Kevin Fox <[email protected]>

* Install some bottom turtle spire bits

Signed-off-by: Kevin Fox <[email protected]>

* Trigger in github

Signed-off-by: Kevin Fox <[email protected]>

* Fix path

Signed-off-by: Kevin Fox <[email protected]>

* Fix path

Signed-off-by: Kevin Fox <[email protected]>

* Fix path

Signed-off-by: Kevin Fox <[email protected]>

* Fix path

Signed-off-by: Kevin Fox <[email protected]>

* Fix path

Signed-off-by: Kevin Fox <[email protected]>

* Fix path

Signed-off-by: Kevin Fox <[email protected]>

* Fix shell code

Signed-off-by: Kevin Fox <[email protected]>

* Add some more testing

Signed-off-by: Kevin Fox <[email protected]>

* Add some more testing

Signed-off-by: Kevin Fox <[email protected]>

* Add some more testing

Signed-off-by: Kevin Fox <[email protected]>

* Add some more debug logging

Signed-off-by: Kevin Fox <[email protected]>

* More logging

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* Some fixes

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* Add x509POP support and more testing

Signed-off-by: Kevin Fox <[email protected]>

* x509pop attestor support and more tests

Signed-off-by: Kevin Fox <[email protected]>

* More updates

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Update docs

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Fix pages artifact upload

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Test some more bits

Signed-off-by: Kevin Fox <[email protected]>

* Initial stab at dynamic registration

Signed-off-by: Kevin Fox <[email protected]>

* Dynamic registration working but not integrated with test

Signed-off-by: Kevin Fox <[email protected]>

* Wire in dynamic registration into the test

Signed-off-by: Kevin Fox <[email protected]>

* Fix missing props

Signed-off-by: Kevin Fox <[email protected]>

* Update the svids to align

Signed-off-by: Kevin Fox <[email protected]>

* Update the svids to align

Signed-off-by: Kevin Fox <[email protected]>

* Fix service name

Signed-off-by: Kevin Fox <[email protected]>

* Look at data

Signed-off-by: Kevin Fox <[email protected]>

* Look at data

Signed-off-by: Kevin Fox <[email protected]>

* Look at data

Signed-off-by: Kevin Fox <[email protected]>

* Fix ca type

Signed-off-by: Kevin Fox <[email protected]>

* Test out new packages

Signed-off-by: Kevin Fox <[email protected]>

* Test out new packages

Signed-off-by: Kevin Fox <[email protected]>

* Test out new packages

Signed-off-by: Kevin Fox <[email protected]>

* Test out new packages

Signed-off-by: Kevin Fox <[email protected]>

* Test out new packages

Signed-off-by: Kevin Fox <[email protected]>

* Test out new packages

Signed-off-by: Kevin Fox <[email protected]>

* Test out new packages

Signed-off-by: Kevin Fox <[email protected]>

* Test out new packages

Signed-off-by: Kevin Fox <[email protected]>

* Test out new packages

Signed-off-by: Kevin Fox <[email protected]>

* Test out new packages

Signed-off-by: Kevin Fox <[email protected]>

* Update ports

Signed-off-by: Kevin Fox <[email protected]>

* Update ports

Signed-off-by: Kevin Fox <[email protected]>

* Work on debugging dynamic registration some more

Signed-off-by: Kevin Fox <[email protected]>

* Fix service account name

Signed-off-by: Kevin Fox <[email protected]>

* Fix service account name

Signed-off-by: Kevin Fox <[email protected]>

* Working... Cleanup.

Signed-off-by: Kevin Fox <[email protected]>

* Working... Cleanup.

Signed-off-by: Kevin Fox <[email protected]>

* Fix broken ssh test

Signed-off-by: Kevin Fox <[email protected]>

* Fix broken ssh test

Signed-off-by: Kevin Fox <[email protected]>

* Simplify a bit

Signed-off-by: Kevin Fox <[email protected]>

* Update to use the released images

Signed-off-by: Kevin Fox <[email protected]>

* Allow x509POP cluster name adding

Signed-off-by: Kevin Fox <[email protected]>

* Restrict cluster registration

Signed-off-by: Kevin Fox <[email protected]>

* Fix var name

Signed-off-by: Kevin Fox <[email protected]>

* Fix missing slash

Signed-off-by: Kevin Fox <[email protected]>

* Make defaults work better

Signed-off-by: Kevin Fox <[email protected]>

* Make defaults work better

Signed-off-by: Kevin Fox <[email protected]>

* Fix readme

Signed-off-by: Kevin Fox <[email protected]>

* updated diagram

Signed-off-by: Kevin Fox <[email protected]>

* Regenerate image

Signed-off-by: Kevin Fox <[email protected]>

* Bump spire versions

Signed-off-by: Kevin Fox <[email protected]>

* Fix issues identified during review

Signed-off-by: Kevin Fox <[email protected]>

* Update docs

Signed-off-by: Kevin Fox <[email protected]>

---------

Signed-off-by: Kevin Fox <[email protected]>
Signed-off-by: kfox1111 <[email protected]>
This commit is contained in:
kfox1111
2026-06-03 12:15:28 -07:00
committed by GitHub
parent a40409ec53
commit 1031167b84
38 changed files with 2274 additions and 554 deletions
+32
View File
@@ -15,6 +15,38 @@ kubeadmConfigPatches:
# admission-control-config-file: /etc/kubernetes/pki/admctrl/admission-control.yaml
nodes:
- role: control-plane
extraMounts:
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-server-a/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-server-a/public
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-server-b/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-server-b/public
- role: worker
extraMounts:
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-server-a/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-server-a/public
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-server-b/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-server-b/public
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-agent-2-a/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-agent-a/public
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-agent-2-b/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-agent-b/public
- role: worker
extraMounts:
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-server-a/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-server-a/public
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-server-b/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-server-b/public
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-agent-3-a/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-agent-a/public
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-agent-3-b/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-agent-b/public
- role: worker
extraMounts:
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-server-a/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-server-a/public
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-server-b/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-server-b/public
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-agent-4-a/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-agent-a/public
- hostPath: /var/run/spiffe/socat/unix/k8s-spire-agent-4-b/public
containerPath: /var/run/spiffe/socat/unix/k8s-spire-agent-b/public
+1 -1
View File
@@ -268,7 +268,7 @@ jobs:
- name: Install and test example
run: |
if [ "${{ matrix.example }}" = "examples/federation" -o "${{ matrix.example }}" = "examples/nested-full" -o "${{ matrix.example }}" = "examples/nested-security" ]; then
if [ "${{ matrix.example }}" = "examples/federation" -o "${{ matrix.example }}" = "examples/nested-full" -o "${{ matrix.example }}" = "examples/nested-security" -o "${{ matrix.example }}" = "examples/bottom-turtle-ha" ]; then
kubectl create namespace spire-mgmt
helm install -n spire-mgmt spire-crds charts/spire-crds
else
@@ -210,7 +210,7 @@ spec:
hostPath:
path: /dev
- name: cid2pid
emtpyDir: {}
emptyDir: {}
{{- if gt (len .Values.extraVolumes) 0 }}
{{- toYaml .Values.extraVolumes | nindent 8 }}
{{- end }}
+57 -1
View File
@@ -4,7 +4,7 @@ description: >
A Helm chart for deploying the complete Spire stack including: spire-server, spire-agent, spiffe-csi-driver, spiffe-oidc-discovery-provider and spire-controller-manager.
type: application
version: 0.28.5
appVersion: "1.14.5"
appVersion: "1.15.1"
keywords: ["spiffe", "spire", "spire-server", "spire-agent", "oidc", "spire-controller-manager"]
home: https://github.com/spiffe/helm-charts-hardened/tree/main/charts/spire
sources:
@@ -124,6 +124,62 @@ dependencies:
condition: spire-ha-agent.enabled
tags:
- haAgentCommon
- name: spire-server
alias: internal-spire-server-bottom-turtle-ha-a
condition: internal-spire-server-bottom-turtle-ha-a.enabled
tags:
- bottomTurtleHAA
repository: file://../spire/charts/spire-server
version: 0.1.0
- name: spire-agent
alias: downstream-spire-agent-bottom-turtle-ha-a
condition: downstream-spire-agent-bottom-turtle-ha-a.enabled
tags:
- bottomTurtleHAA
repository: file://../spire/charts/spire-agent
version: 0.1.0
- name: spiffe-csi-driver
alias: downstream-spiffe-csi-driver-bottom-turtle-ha-a
condition: downstream-spiffe-csi-driver-bottom-turtle-ha-a.enabled
tags:
- bottomTurtleHAA
repository: file://../spire/charts/spiffe-csi-driver
version: 0.1.0
- name: spiffe-csi-driver
alias: upstream-spiffe-csi-driver-bottom-turtle-ha-a
condition: upstream-spiffe-csi-driver-bottom-turtle-ha-a.enabled
tags:
- bottomTurtleHAA
repository: file://../spire/charts/spiffe-csi-driver
version: 0.1.0
- name: spire-server
alias: internal-spire-server-bottom-turtle-ha-b
condition: internal-spire-server-bottom-turtle-ha-b.enabled
tags:
- bottomTurtleHAB
repository: file://../spire/charts/spire-server
version: 0.1.0
- name: spire-agent
alias: downstream-spire-agent-bottom-turtle-ha-b
condition: downstream-spire-agent-bottom-turtle-ha-b.enabled
tags:
- bottomTurtleHAB
repository: file://../spire/charts/spire-agent
version: 0.1.0
- name: spiffe-csi-driver
alias: downstream-spiffe-csi-driver-bottom-turtle-ha-b
condition: downstream-spiffe-csi-driver-bottom-turtle-ha-b.enabled
tags:
- bottomTurtleHAB
repository: file://../spire/charts/spiffe-csi-driver
version: 0.1.0
- name: spiffe-csi-driver
alias: upstream-spiffe-csi-driver-bottom-turtle-ha-b
condition: upstream-spiffe-csi-driver-bottom-turtle-ha-b.enabled
tags:
- bottomTurtleHAB
repository: file://../spire/charts/spiffe-csi-driver
version: 0.1.0
annotations:
artifacthub.io/category: security
artifacthub.io/license: Apache-2.0
+135 -2
View File
@@ -1,6 +1,6 @@
# spire
![Version: 0.28.5](https://img.shields.io/badge/Version-0.28.5-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.14.5](https://img.shields.io/badge/AppVersion-1.14.5-informational?style=flat-square)
![Version: 0.28.5](https://img.shields.io/badge/Version-0.28.5-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.15.1](https://img.shields.io/badge/AppVersion-1.15.1-informational?style=flat-square)
[![Development Phase](https://github.com/spiffe/spiffe/blob/main/.img/maturity/dev.svg)](https://github.com/spiffe/spiffe/blob/main/MATURITY.md#development)
A Helm chart for deploying the complete Spire stack including: spire-server, spire-agent, spiffe-csi-driver, spiffe-oidc-discovery-provider and spire-controller-manager.
@@ -234,6 +234,8 @@ Now you can interact with the Spire agent socket from your own application. The
| `tags.nestedChildFull` | Set the chart mode to a child cluster with its own nested server | `false` |
| `tags.nestedChildSecurity` | Set the chart mode to a child cluster for use with a security cluster | `false` |
| `tags.haAgentCommon` | Set the chart mode to deploy the common portion of a spire-ha-agent setup | `false` |
| `tags.bottomTurtleHAA` | Setup HA side A for use with a Bottom Turtle architecture | `false` |
| `tags.bottomTurtleHAB` | Setup HA side B for use with a Bottom Turtle architecture | `false` |
### Spire agent parameters
@@ -354,6 +356,137 @@ Now you can interact with the Spire agent socket from your own application. The
| `external-spire-server.bundlePublisher.k8sConfigMap.enabled` | Enable local k8s bundle uploader | `false` |
| `external-spire-server.nodeAttestor.k8sPSAT.enabled` | Enable PSAT k8s nodeattestor | `false` |
| `external-spire-server.nodeAttestor.joinToken.enabled` | Enable the join_token nodeattestor | `true` |
| `spiffe-csi-driver.fullnameOverride` | Fullname override | `spiffe-csi-driver` |
| `spiffe-csi-driver.agentSocketPath` | The socket path where Spiffe CSI driver mounts agent socket | `/var/run/spire/agent-sockets/spire-agent.sock` |
| `spiffe-csi-driver.healthChecks.port` | Health check port number for upstream Spire agent | `9814` |
| `spire-ha-agent` | The configuration overrides for a spire-ha-agent | `{}` |
| `spire-ha-agent.fullnameOverride` | Fullname override | `spire-ha-agent` |
### Upstream SPIFFE CSI Driver for Bottom Turtle HA A parameters
| Name | Description | Value |
| --------------------------------------------------------------------------------- | ----------------------------------------------------------- | ----------------------------------------------------------------------- |
| `upstream-spiffe-csi-driver-bottom-turtle-ha-a.fullnameOverride` | Fullname override | `spiffe-csi-driver-upstream-a` |
| `upstream-spiffe-csi-driver-bottom-turtle-ha-a.pluginName` | The plugin name for configuring upstream Spiffe CSI driver | `upstream-a.csi.spiffe.io` |
| `upstream-spiffe-csi-driver-bottom-turtle-ha-a.agentSocketPath` | The socket path where Spiffe CSI driver mounts agent socket | `/var/run/spiffe/socat/unix/k8s-spire-server-a/public/spire-agent.sock` |
| `upstream-spiffe-csi-driver-bottom-turtle-ha-a.healthChecks.port` | The port where Spiffe CSI driver health checks are exposed | `9810` |
| `upstream-spiffe-csi-driver-bottom-turtle-ha-a.validatingAdmissionPolicy.enabled` | Flag to enable validating policy | `true` |
### Upstream SPIFFE CSI Driver for Bottom Turtle HA B parameters
| Name | Description | Value |
| --------------------------------------------------------------------------------- | ----------------------------------------------------------- | ----------------------------------------------------------------------- |
| `upstream-spiffe-csi-driver-bottom-turtle-ha-b.fullnameOverride` | Fullname override | `spiffe-csi-driver-upstream-b` |
| `upstream-spiffe-csi-driver-bottom-turtle-ha-b.pluginName` | The plugin name for configuring upstream Spiffe CSI driver | `upstream-b.csi.spiffe.io` |
| `upstream-spiffe-csi-driver-bottom-turtle-ha-b.agentSocketPath` | The socket path where Spiffe CSI driver mounts agent socket | `/var/run/spiffe/socat/unix/k8s-spire-server-b/public/spire-agent.sock` |
| `upstream-spiffe-csi-driver-bottom-turtle-ha-b.healthChecks.port` | The port where Spiffe CSI driver health checks are exposed | `9812` |
| `upstream-spiffe-csi-driver-bottom-turtle-ha-b.validatingAdmissionPolicy.enabled` | Flag to enable validating policy | `true` |
### Spire server parameters
| Name | Description | Value |
| ------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------- | ----------------------------------------------------------------------------- |
| `internal-spire-server-bottom-turtle-ha-a.nameOverride` | Overrides the name of Spire server pods | `internal-server` |
| `internal-spire-server-bottom-turtle-ha-a.caKeyType` | Key type to use for the ca | `ec-p256` |
| `internal-spire-server-bottom-turtle-ha-a.experimental.enabled` | enable experimental features | `true` |
| `internal-spire-server-bottom-turtle-ha-a.experimental.agentSPIFFEIDAsSelector` | enable adding spiffe_id selectors to all agents | `true` |
| `internal-spire-server-bottom-turtle-ha-a.dynamicRegistration.enabled` | Enable dynamic registration | `true` |
| `internal-spire-server-bottom-turtle-ha-a.dynamicRegistration.allowedIDPrefix` | The allowed ID prefix | `spire/agent/x509pop/k8s` |
| `internal-spire-server-bottom-turtle-ha-a.dynamicRegistration.serviceAccount` | The service account to allow in for dynamic registration | `spire-a-agent` |
| `internal-spire-server-bottom-turtle-ha-a.controllerManager.enabled` | Enable controller manager and provision CRD's | `true` |
| `internal-spire-server-bottom-turtle-ha-a.controllerManager.parentIDTemplate` | parent id template | `spiffe://{{ .TrustDomain }}/k8s_psat/{{ .ClusterName }}/{{ .NodeMeta.UID }}` |
| `internal-spire-server-bottom-turtle-ha-a.controllerManager.identities.clusterSPIFFEIDs.oidc-discovery-provider.autoPopulateDNSNames` | Auto populate dns entries | `false` |
| `internal-spire-server-bottom-turtle-ha-a.controllerManager.identities.clusterSPIFFEIDs.oidc-discovery-provider.type` | The type of the entry | `oidc-discovery-provider-common` |
| `internal-spire-server-bottom-turtle-ha-a.controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.enabled` | Enables the spire-ha-agent identity | `true` |
| `internal-spire-server-bottom-turtle-ha-a.persistence.type` | What type to use for peristence | `emptyDir` |
| `internal-spire-server-bottom-turtle-ha-a.nodeAttestor.k8sPSAT.enabled` | Enable the k8s projected access token node attestor | `false` |
| `internal-spire-server-bottom-turtle-ha-a.nodeAttestor.x509POP.enabled` | Enable the x509 pop node attestor | `true` |
| `internal-spire-server-bottom-turtle-ha-a.nodeAttestor.x509POP.svidPrefix` | What prefix to use when mode is spiffe | `/spire-exchange/k8s` |
| `internal-spire-server-bottom-turtle-ha-a.nodeAttestor.x509POP.agentPathTemplate` | Override the default agent path template | `/{{ .PluginName }}/{{ .SVIDPathTrimmed }}/k8s` |
| `internal-spire-server-bottom-turtle-ha-a.nodeAttestor.x509POP.addClusterName.svidPrefix` | Suffix the cluster name onto the svidPrefix | `true` |
| `internal-spire-server-bottom-turtle-ha-a.nodeAttestor.x509POP.addClusterName.agentPathTemplate` | Suffix the cluster name onto the agentPathTemplate | `true` |
| `internal-spire-server-bottom-turtle-ha-a.upstreamAuthority.spire.enabled` | Enable upstream SPIRE server | `true` |
| `internal-spire-server-bottom-turtle-ha-a.upstreamAuthority.spire.upstreamDriver` | Use an upstream driver for authentication | `upstream-a.csi.spiffe.io` |
| `internal-spire-server-bottom-turtle-ha-a.upstreamAuthority.spire.server.nameOverride` | The name override setting of the root SPIRE server | `root-server` |
| `internal-spire-server-bottom-turtle-ha-a.upstreamAuthority.spire.server.address` | Address for upstream Spire server | `spire-server-a` |
| `internal-spire-server-bottom-turtle-ha-a.upstreamAuthority.spire.server.port` | The port setting of the root SPIRE server | `8081` |
| `internal-spire-server-bottom-turtle-ha-a.bundleConfigMap` | The name of the configmap to store the downstream bundle | `spire-server-a-bundle` |
| `internal-spire-server-bottom-turtle-ha-a.trustSync.enabled` | Enable trust syncing | `true` |
| `internal-spire-server-bottom-turtle-ha-a.trustSync.domains` | the trust domains to sync | `["spire-ha"]` |
### Spire server parameters
| Name | Description | Value |
| ------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| `internal-spire-server-bottom-turtle-ha-b.nameOverride` | Overrides the name of Spire server pods | `internal-server` |
| `internal-spire-server-bottom-turtle-ha-b.caKeyType` | Key type to use for the ca | `ec-p256` |
| `internal-spire-server-bottom-turtle-ha-b.experimental.enabled` | enable experimental features | `true` |
| `internal-spire-server-bottom-turtle-ha-b.experimental.agentSPIFFEIDAsSelector` | enable adding spiffe_id selectors to all agents | `true` |
| `internal-spire-server-bottom-turtle-ha-b.dynamicRegistration.enabled` | Enable dynamic registration | `true` |
| `internal-spire-server-bottom-turtle-ha-b.dynamicRegistration.allowedIDPrefix` | The allowed ID prefix | `spire/agent/x509pop/k8s` |
| `internal-spire-server-bottom-turtle-ha-b.dynamicRegistration.serviceAccount` | The service account to allow in for dynamic registration | `spire-b-agent` |
| `internal-spire-server-bottom-turtle-ha-b.controllerManager.enabled` | Enable controller manager and provision CRD's | `true` |
| `internal-spire-server-bottom-turtle-ha-b.controllerManager.parentIDTemplate` | parent id template | `spiffe://{{ .TrustDomain }}/k8s_psat/{{ .ClusterName }}/{{ .NodeMeta.UID }}` |
| `internal-spire-server-bottom-turtle-ha-b.controllerManager.identities.clusterSPIFFEIDs.oidc-discovery-provider.autoPopulateDNSNames` | Auto populate dns entries | `false` |
| `internal-spire-server-bottom-turtle-ha-b.controllerManager.identities.clusterSPIFFEIDs.oidc-discovery-provider.type` | The type of the entry | `oidc-discovery-provider-common` |
| `internal-spire-server-bottom-turtle-ha-b.controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.enabled` | Enables the spire-ha-agent identity | `true` |
| `internal-spire-server-bottom-turtle-ha-b.persistence.type` | What type to use for peristence | `emptyDir` |
| `internal-spire-server-bottom-turtle-ha-b.nodeAttestor.k8sPSAT.enabled` | Enable the k8s projected access token node attestor | `false` |
| `internal-spire-server-bottom-turtle-ha-b.nodeAttestor.x509POP.enabled` | Enable the x509 pop node attestor | `true` |
| `internal-spire-server-bottom-turtle-ha-b.nodeAttestor.x509POP.svidPrefix` | What prefix to use when mode is spiffe | `/spire-exchange/k8s` |
| `internal-spire-server-bottom-turtle-ha-b.nodeAttestor.x509POP.agentPathTemplate` | Override the default agent path template | `/{{ .PluginName }}/{{ .SVIDPathTrimmed }}/k8s` |
| `internal-spire-server-bottom-turtle-ha-b.nodeAttestor.x509POP.addClusterName.svidPrefix` | Suffix the cluster name onto the svidPrefix | `true` |
| `internal-spire-server-bottom-turtle-ha-b.nodeAttestor.x509POP.addClusterName.agentPathTemplate` | Suffix the cluster name onto the agentPathTemplate | `true` |
| `internal-spire-server-bottom-turtle-ha-b.upstreamAuthority.spire.enabled` | Enable upstream SPIRE server | `true` |
| `internal-spire-server-bottom-turtle-ha-b.upstreamAuthority.spire.upstreamDriver` | Use an upstream driver for authentication | `upstream-b.csi.spiffe.io` |
| `internal-spire-server-bottom-turtle-ha-b.upstreamAuthority.spire.server.nameOverride` | The name override setting of the root SPIRE server | `root-server` |
| `internal-spire-server-bottom-turtle-ha-b.upstreamAuthority.spire.server.address` | Address for upstream Spire server | `spire-server-b` |
| `internal-spire-server-bottom-turtle-ha-b.upstreamAuthority.spire.server.port` | The port setting of the root SPIRE server | `8081` |
| `internal-spire-server-bottom-turtle-ha-b.bundleConfigMap` | The name of the configmap to store the downstream bundle | `spire-server-b-bundle` |
| `internal-spire-server-bottom-turtle-ha-b.trustSync.enabled` | Enable trust syncing | `true` |
| `internal-spire-server-bottom-turtle-ha-b.trustSync.domains` | the trust domains to sync | `["spire-ha"]` |
| `downstream-spire-agent-bottom-turtle-ha-a.nameOverride` | Overrides the name of Spire agent pods | `agent-downstream` |
| `downstream-spire-agent-bottom-turtle-ha-a.server.nameOverride` | The name override setting of the internal SPIRE server | `internal-server` |
| `downstream-spire-agent-bottom-turtle-ha-a.bundleConfigMap` | The name of the configmap that contains the downstream bundle | `spire-server-a-bundle` |
| `downstream-spire-agent-bottom-turtle-ha-a.persistence.hostPath` | Which path to use on the host when persistence.type = hostPath | `/var/lib/spire/k8s/downstream-agent-a` |
| `downstream-spire-agent-bottom-turtle-ha-a.dynamicRegistration.enabled` | Enable dynamic registration | `true` |
| `downstream-spire-agent-bottom-turtle-ha-a.dynamicRegistration.nameOverride` | The name override to use to contact the server | `internal-server` |
| `downstream-spire-agent-bottom-turtle-ha-a.nodeAttestor.k8sPSAT.enabled` | Enable the k8s projected access token node attestor | `false` |
| `downstream-spire-agent-bottom-turtle-ha-a.nodeAttestor.x509POP.enabled` | Enable the x509 pop node attestor | `true` |
| `downstream-spire-agent-bottom-turtle-ha-a.nodeAttestor.x509POP.spiffeEndpointSocket` | Where the socket is to use for mode spiffe | `/var/run/spiffe/socat/unix/k8s-spire-agent-a/public/api.sock` |
| `downstream-spire-agent-bottom-turtle-ha-a.keyManager.memory.enabled` | Enable the memory based Key Manager | `false` |
| `downstream-spire-agent-bottom-turtle-ha-a.keyManager.disk.enabled` | Enable the disk key manager | `true` |
| `downstream-spire-agent-bottom-turtle-ha-a.keyManager.disk.mode` | Where the disk plugin will write out its data | `emptyDir` |
| `downstream-spire-agent-bottom-turtle-ha-a.healthChecks.port` | Health check port | `9981` |
| `downstream-spire-agent-bottom-turtle-ha-a.telemetry.prometheus.port` | Prometheus port to use | `9989` |
| `downstream-spire-agent-bottom-turtle-ha-a.socketPath` | Socket path to use | `/var/run/spire/agent/sockets/a/csi.spiffe.io/public/spire-agent.sock` |
| `downstream-spire-agent-bottom-turtle-ha-a.sockets.hostBasePath` | Path on the host to place sockets | `/var/run/spire/agent/sockets/a` |
| `downstream-spire-agent-bottom-turtle-ha-a.sockets.admin.enabled` | Enable admin socket | `true` |
| `downstream-spire-agent-bottom-turtle-ha-a.sockets.admin.mountOnHost` | Mount admin socket on host | `true` |
| `downstream-spire-agent-bottom-turtle-ha-a.authorizedDelegates` | List of workloads able to use the delegation api | `["/spire-ha-agent"]` |
| `downstream-spire-agent-bottom-turtle-ha-b.nameOverride` | Overrides the name of Spire agent pods | `agent-downstream` |
| `downstream-spire-agent-bottom-turtle-ha-b.server.nameOverride` | The name override setting of the internal SPIRE server | `internal-server` |
| `downstream-spire-agent-bottom-turtle-ha-b.bundleConfigMap` | The name of the configmap that contains the downstream bundle | `spire-server-b-bundle` |
| `downstream-spire-agent-bottom-turtle-ha-b.persistence.hostPath` | Which path to use on the host when persistence.type = hostPath | `/var/lib/spire/k8s/downstream-agent-b` |
| `downstream-spire-agent-bottom-turtle-ha-b.dynamicRegistration.enabled` | Enable dynamic registration | `true` |
| `downstream-spire-agent-bottom-turtle-ha-b.dynamicRegistration.nameOverride` | The name override to use to contact the server | `internal-server` |
| `downstream-spire-agent-bottom-turtle-ha-b.nodeAttestor.k8sPSAT.enabled` | Enable the k8s projected access token node attestor | `false` |
| `downstream-spire-agent-bottom-turtle-ha-b.nodeAttestor.x509POP.enabled` | Enable the x509 pop node attestor | `true` |
| `downstream-spire-agent-bottom-turtle-ha-b.nodeAttestor.x509POP.spiffeEndpointSocket` | Where the socket is to use for mode spiffe | `/var/run/spiffe/socat/unix/k8s-spire-agent-b/public/api.sock` |
| `downstream-spire-agent-bottom-turtle-ha-b.keyManager.memory.enabled` | Enable the memory based Key Manager | `false` |
| `downstream-spire-agent-bottom-turtle-ha-b.keyManager.disk.enabled` | Enable the disk key manager | `true` |
| `downstream-spire-agent-bottom-turtle-ha-b.keyManager.disk.mode` | Where the disk plugin will write out its data | `emptyDir` |
| `downstream-spire-agent-bottom-turtle-ha-b.healthChecks.port` | Health check port | `9982` |
| `downstream-spire-agent-bottom-turtle-ha-b.telemetry.prometheus.port` | Prometheus port to use | `9990` |
| `downstream-spire-agent-bottom-turtle-ha-b.socketPath` | Socket path to use | `/var/run/spire/agent/sockets/b/csi.spiffe.io/public/spire-agent.sock` |
| `downstream-spire-agent-bottom-turtle-ha-b.sockets.hostBasePath` | Path on the host to place sockets | `/var/run/spire/agent/sockets/b` |
| `downstream-spire-agent-bottom-turtle-ha-b.sockets.admin.enabled` | Enable admin socket | `true` |
| `downstream-spire-agent-bottom-turtle-ha-b.sockets.admin.mountOnHost` | Mount admin socket on host | `true` |
| `downstream-spire-agent-bottom-turtle-ha-b.authorizedDelegates` | List of workloads able to use the delegation api | `["/spire-ha-agent"]` |
| `downstream-spiffe-csi-driver-bottom-turtle-ha-a.fullnameOverride` | Fullname override | `spiffe-csi-driver-downstream-a` |
| `downstream-spiffe-csi-driver-bottom-turtle-ha-a.agentSocketPath` | path to agent socket | `/var/run/spire/agent/sockets/a/csi.spiffe.io/public/spire-agent.sock` |
| `downstream-spiffe-csi-driver-bottom-turtle-ha-a.pluginName` | The name of the plugin instance | `a.csi.spiffe.io` |
| `downstream-spiffe-csi-driver-bottom-turtle-ha-a.healthChecks.port` | The health check port | `9814` |
| `downstream-spiffe-csi-driver-bottom-turtle-ha-b.fullnameOverride` | Fullname override | `spiffe-csi-driver-downstream-b` |
| `downstream-spiffe-csi-driver-bottom-turtle-ha-b.agentSocketPath` | path to agent socket | `/var/run/spire/agent/sockets/b/csi.spiffe.io/public/spire-agent.sock` |
| `downstream-spiffe-csi-driver-bottom-turtle-ha-b.pluginName` | The name of the plugin instance | `b.csi.spiffe.io` |
| `downstream-spiffe-csi-driver-bottom-turtle-ha-b.healthChecks.port` | The health check port | `9816` |
+343 -2
View File
@@ -103,6 +103,10 @@ tags:
nestedChildSecurity: false
## @param tags.haAgentCommon Set the chart mode to deploy the common portion of a spire-ha-agent setup
haAgentCommon: false
## @param tags.bottomTurtleHAA Setup HA side A for use with a Bottom Turtle architecture
bottomTurtleHAA: false
## @param tags.bottomTurtleHAB Setup HA side B for use with a Bottom Turtle architecture
bottomTurtleHAB: false
## subcharts
@@ -401,13 +405,350 @@ external-spire-server:
# Used with tags [haAgentCommon]
spiffe-csi-driver:
# enabled: true
## @param spiffe-csi-driver.fullnameOverride Fullname override
fullnameOverride: spiffe-csi-driver
## @param spiffe-csi-driver.agentSocketPath The socket path where Spiffe CSI driver mounts agent socket
agentSocketPath: /var/run/spire/agent-sockets/spire-agent.sock
healthChecks:
## @param spiffe-csi-driver.healthChecks.port Health check port number for upstream Spire agent
port: 9814
## @param spire-ha-agent The configuration overrides for a spire-ha-agent
# Used with tags [haAgentCommon]
spire-ha-agent: {}
spire-ha-agent:
# enabled: true
## @param spire-ha-agent.fullnameOverride Fullname override
fullnameOverride: spire-ha-agent
## @section Upstream SPIFFE CSI Driver for Bottom Turtle HA A parameters
## Parameter values for upstream spiffe-csi-driver-bottom-turtle-ha-a
##
# Used with tags [bottomTurtleHAA]
upstream-spiffe-csi-driver-bottom-turtle-ha-a:
# enabled: true
## @param upstream-spiffe-csi-driver-bottom-turtle-ha-a.fullnameOverride Fullname override
fullnameOverride: spiffe-csi-driver-upstream-a
## @param upstream-spiffe-csi-driver-bottom-turtle-ha-a.pluginName The plugin name for configuring upstream Spiffe CSI driver
pluginName: upstream-a.csi.spiffe.io
## @param upstream-spiffe-csi-driver-bottom-turtle-ha-a.agentSocketPath The socket path where Spiffe CSI driver mounts agent socket
agentSocketPath: /var/run/spiffe/socat/unix/k8s-spire-server-a/public/spire-agent.sock
healthChecks:
## @param upstream-spiffe-csi-driver-bottom-turtle-ha-a.healthChecks.port The port where Spiffe CSI driver health checks are exposed
port: 9810
validatingAdmissionPolicy:
## @param upstream-spiffe-csi-driver-bottom-turtle-ha-a.validatingAdmissionPolicy.enabled Flag to enable validating policy
enabled: true
## @section Upstream SPIFFE CSI Driver for Bottom Turtle HA B parameters
## Parameter values for upstream spiffe-csi-driver-bottom-turtle-ha-b
##
# Used with tags [bottomTurtleHAB]
upstream-spiffe-csi-driver-bottom-turtle-ha-b:
# enabled: true
## @param upstream-spiffe-csi-driver-bottom-turtle-ha-b.fullnameOverride Fullname override
fullnameOverride: spiffe-csi-driver-upstream-b
## @param upstream-spiffe-csi-driver-bottom-turtle-ha-b.pluginName The plugin name for configuring upstream Spiffe CSI driver
pluginName: upstream-b.csi.spiffe.io
## @param upstream-spiffe-csi-driver-bottom-turtle-ha-b.agentSocketPath The socket path where Spiffe CSI driver mounts agent socket
agentSocketPath: /var/run/spiffe/socat/unix/k8s-spire-server-b/public/spire-agent.sock
healthChecks:
## @param upstream-spiffe-csi-driver-bottom-turtle-ha-b.healthChecks.port The port where Spiffe CSI driver health checks are exposed
port: 9812
validatingAdmissionPolicy:
## @param upstream-spiffe-csi-driver-bottom-turtle-ha-b.validatingAdmissionPolicy.enabled Flag to enable validating policy
enabled: true
## @section Spire server parameters
## Parameter values for Spire server
##
# Used with tags [bottomTurtleHAA]
internal-spire-server-bottom-turtle-ha-a:
# enabled: true
## @param internal-spire-server-bottom-turtle-ha-a.nameOverride Overrides the name of Spire server pods
nameOverride: internal-server
## @param internal-spire-server-bottom-turtle-ha-a.caKeyType Key type to use for the ca
caKeyType: ec-p256
experimental:
## @param internal-spire-server-bottom-turtle-ha-a.experimental.enabled enable experimental features
enabled: true
## @param internal-spire-server-bottom-turtle-ha-a.experimental.agentSPIFFEIDAsSelector enable adding spiffe_id selectors to all agents
agentSPIFFEIDAsSelector: true
dynamicRegistration:
## @param internal-spire-server-bottom-turtle-ha-a.dynamicRegistration.enabled Enable dynamic registration
enabled: true
## @param internal-spire-server-bottom-turtle-ha-a.dynamicRegistration.allowedIDPrefix The allowed ID prefix
allowedIDPrefix: "spire/agent/x509pop/k8s"
## @param internal-spire-server-bottom-turtle-ha-a.dynamicRegistration.serviceAccount The service account to allow in for dynamic registration
serviceAccount: spire-a-agent
controllerManager:
## @param internal-spire-server-bottom-turtle-ha-a.controllerManager.enabled Enable controller manager and provision CRD's
enabled: true
## @param internal-spire-server-bottom-turtle-ha-a.controllerManager.parentIDTemplate parent id template
parentIDTemplate: "spiffe://{{ .TrustDomain }}/k8s_psat/{{ .ClusterName }}/{{ .NodeMeta.UID }}"
identities:
clusterSPIFFEIDs:
oidc-discovery-provider:
## @param internal-spire-server-bottom-turtle-ha-a.controllerManager.identities.clusterSPIFFEIDs.oidc-discovery-provider.autoPopulateDNSNames Auto populate dns entries
autoPopulateDNSNames: false
## @param internal-spire-server-bottom-turtle-ha-a.controllerManager.identities.clusterSPIFFEIDs.oidc-discovery-provider.type The type of the entry
type: oidc-discovery-provider-common
spire-ha-agent:
## @param internal-spire-server-bottom-turtle-ha-a.controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.enabled Enables the spire-ha-agent identity
enabled: true
persistence:
## @param internal-spire-server-bottom-turtle-ha-a.persistence.type What type to use for peristence
type: emptyDir
nodeAttestor:
k8sPSAT:
## @param internal-spire-server-bottom-turtle-ha-a.nodeAttestor.k8sPSAT.enabled Enable the k8s projected access token node attestor
enabled: false
x509POP:
## @param internal-spire-server-bottom-turtle-ha-a.nodeAttestor.x509POP.enabled Enable the x509 pop node attestor
enabled: true
## @param internal-spire-server-bottom-turtle-ha-a.nodeAttestor.x509POP.svidPrefix What prefix to use when mode is spiffe
svidPrefix: /spire-exchange/k8s
## @param internal-spire-server-bottom-turtle-ha-a.nodeAttestor.x509POP.agentPathTemplate Override the default agent path template
agentPathTemplate: "/{{ .PluginName }}/{{ .SVIDPathTrimmed }}/k8s"
addClusterName:
## @param internal-spire-server-bottom-turtle-ha-a.nodeAttestor.x509POP.addClusterName.svidPrefix Suffix the cluster name onto the svidPrefix
svidPrefix: true
## @param internal-spire-server-bottom-turtle-ha-a.nodeAttestor.x509POP.addClusterName.agentPathTemplate Suffix the cluster name onto the agentPathTemplate
agentPathTemplate: true
upstreamAuthority:
spire:
## @param internal-spire-server-bottom-turtle-ha-a.upstreamAuthority.spire.enabled Enable upstream SPIRE server
enabled: true
## @param internal-spire-server-bottom-turtle-ha-a.upstreamAuthority.spire.upstreamDriver Use an upstream driver for authentication
upstreamDriver: upstream-a.csi.spiffe.io
server:
## @param internal-spire-server-bottom-turtle-ha-a.upstreamAuthority.spire.server.nameOverride The name override setting of the root SPIRE server
nameOverride: root-server
## @param internal-spire-server-bottom-turtle-ha-a.upstreamAuthority.spire.server.address Address for upstream Spire server
address: "spire-server-a"
## @param internal-spire-server-bottom-turtle-ha-a.upstreamAuthority.spire.server.port The port setting of the root SPIRE server
port: 8081
## @param internal-spire-server-bottom-turtle-ha-a.bundleConfigMap The name of the configmap to store the downstream bundle
bundleConfigMap: spire-server-a-bundle
trustSync:
## @param internal-spire-server-bottom-turtle-ha-a.trustSync.enabled Enable trust syncing
enabled: true
## @param internal-spire-server-bottom-turtle-ha-a.trustSync.domains the trust domains to sync
domains:
- spire-ha
## @section Spire server parameters
## Parameter values for Spire server
##
# Used with tags [bottomTurtleHAB]
internal-spire-server-bottom-turtle-ha-b:
# enabled: true
## @param internal-spire-server-bottom-turtle-ha-b.nameOverride Overrides the name of Spire server pods
nameOverride: internal-server
## @param internal-spire-server-bottom-turtle-ha-b.caKeyType Key type to use for the ca
caKeyType: ec-p256
experimental:
## @param internal-spire-server-bottom-turtle-ha-b.experimental.enabled enable experimental features
enabled: true
## @param internal-spire-server-bottom-turtle-ha-b.experimental.agentSPIFFEIDAsSelector enable adding spiffe_id selectors to all agents
agentSPIFFEIDAsSelector: true
dynamicRegistration:
## @param internal-spire-server-bottom-turtle-ha-b.dynamicRegistration.enabled Enable dynamic registration
enabled: true
## @param internal-spire-server-bottom-turtle-ha-b.dynamicRegistration.allowedIDPrefix The allowed ID prefix
allowedIDPrefix: "spire/agent/x509pop/k8s"
## @param internal-spire-server-bottom-turtle-ha-b.dynamicRegistration.serviceAccount The service account to allow in for dynamic registration
serviceAccount: spire-b-agent
controllerManager:
## @param internal-spire-server-bottom-turtle-ha-b.controllerManager.enabled Enable controller manager and provision CRD's
enabled: true
## @param internal-spire-server-bottom-turtle-ha-b.controllerManager.parentIDTemplate parent id template
parentIDTemplate: "spiffe://{{ .TrustDomain }}/k8s_psat/{{ .ClusterName }}/{{ .NodeMeta.UID }}"
identities:
clusterSPIFFEIDs:
oidc-discovery-provider:
## @param internal-spire-server-bottom-turtle-ha-b.controllerManager.identities.clusterSPIFFEIDs.oidc-discovery-provider.autoPopulateDNSNames Auto populate dns entries
autoPopulateDNSNames: false
## @param internal-spire-server-bottom-turtle-ha-b.controllerManager.identities.clusterSPIFFEIDs.oidc-discovery-provider.type The type of the entry
type: oidc-discovery-provider-common
spire-ha-agent:
## @param internal-spire-server-bottom-turtle-ha-b.controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.enabled Enables the spire-ha-agent identity
enabled: true
persistence:
## @param internal-spire-server-bottom-turtle-ha-b.persistence.type What type to use for peristence
type: emptyDir
nodeAttestor:
k8sPSAT:
## @param internal-spire-server-bottom-turtle-ha-b.nodeAttestor.k8sPSAT.enabled Enable the k8s projected access token node attestor
enabled: false
x509POP:
## @param internal-spire-server-bottom-turtle-ha-b.nodeAttestor.x509POP.enabled Enable the x509 pop node attestor
enabled: true
## @param internal-spire-server-bottom-turtle-ha-b.nodeAttestor.x509POP.svidPrefix What prefix to use when mode is spiffe
svidPrefix: /spire-exchange/k8s
## @param internal-spire-server-bottom-turtle-ha-b.nodeAttestor.x509POP.agentPathTemplate Override the default agent path template
agentPathTemplate: "/{{ .PluginName }}/{{ .SVIDPathTrimmed }}/k8s"
addClusterName:
## @param internal-spire-server-bottom-turtle-ha-b.nodeAttestor.x509POP.addClusterName.svidPrefix Suffix the cluster name onto the svidPrefix
svidPrefix: true
## @param internal-spire-server-bottom-turtle-ha-b.nodeAttestor.x509POP.addClusterName.agentPathTemplate Suffix the cluster name onto the agentPathTemplate
agentPathTemplate: true
upstreamAuthority:
spire:
## @param internal-spire-server-bottom-turtle-ha-b.upstreamAuthority.spire.enabled Enable upstream SPIRE server
enabled: true
## @param internal-spire-server-bottom-turtle-ha-b.upstreamAuthority.spire.upstreamDriver Use an upstream driver for authentication
upstreamDriver: upstream-b.csi.spiffe.io
server:
## @param internal-spire-server-bottom-turtle-ha-b.upstreamAuthority.spire.server.nameOverride The name override setting of the root SPIRE server
nameOverride: root-server
## @param internal-spire-server-bottom-turtle-ha-b.upstreamAuthority.spire.server.address Address for upstream Spire server
address: "spire-server-b"
## @param internal-spire-server-bottom-turtle-ha-b.upstreamAuthority.spire.server.port The port setting of the root SPIRE server
port: 8081
## @param internal-spire-server-bottom-turtle-ha-b.bundleConfigMap The name of the configmap to store the downstream bundle
bundleConfigMap: spire-server-b-bundle
trustSync:
## @param internal-spire-server-bottom-turtle-ha-b.trustSync.enabled Enable trust syncing
enabled: true
## @param internal-spire-server-bottom-turtle-ha-b.trustSync.domains the trust domains to sync
domains:
- spire-ha
# Used with tags [bottomTurtleHAA]
downstream-spire-agent-bottom-turtle-ha-a:
# enabled: true
## @param downstream-spire-agent-bottom-turtle-ha-a.nameOverride Overrides the name of Spire agent pods
nameOverride: agent-downstream
server:
## @param downstream-spire-agent-bottom-turtle-ha-a.server.nameOverride The name override setting of the internal SPIRE server
nameOverride: internal-server
## @param downstream-spire-agent-bottom-turtle-ha-a.bundleConfigMap The name of the configmap that contains the downstream bundle
bundleConfigMap: spire-server-a-bundle
## @param downstream-spire-agent-bottom-turtle-ha-a.persistence.hostPath Which path to use on the host when persistence.type = hostPath
persistence:
hostPath: /var/lib/spire/k8s/downstream-agent-a
dynamicRegistration:
## @param downstream-spire-agent-bottom-turtle-ha-a.dynamicRegistration.enabled Enable dynamic registration
enabled: true
## @param downstream-spire-agent-bottom-turtle-ha-a.dynamicRegistration.nameOverride The name override to use to contact the server
nameOverride: internal-server
nodeAttestor:
k8sPSAT:
## @param downstream-spire-agent-bottom-turtle-ha-a.nodeAttestor.k8sPSAT.enabled Enable the k8s projected access token node attestor
enabled: false
x509POP:
## @param downstream-spire-agent-bottom-turtle-ha-a.nodeAttestor.x509POP.enabled Enable the x509 pop node attestor
enabled: true
## @param downstream-spire-agent-bottom-turtle-ha-a.nodeAttestor.x509POP.spiffeEndpointSocket Where the socket is to use for mode spiffe
spiffeEndpointSocket: /var/run/spiffe/socat/unix/k8s-spire-agent-a/public/api.sock
keyManager:
memory:
## @param downstream-spire-agent-bottom-turtle-ha-a.keyManager.memory.enabled Enable the memory based Key Manager
enabled: false
disk:
## @param downstream-spire-agent-bottom-turtle-ha-a.keyManager.disk.enabled Enable the disk key manager
enabled: true
## @param downstream-spire-agent-bottom-turtle-ha-a.keyManager.disk.mode Where the disk plugin will write out its data
mode: emptyDir
healthChecks:
## @param downstream-spire-agent-bottom-turtle-ha-a.healthChecks.port Health check port
port: 9981
telemetry:
prometheus:
## @param downstream-spire-agent-bottom-turtle-ha-a.telemetry.prometheus.port Prometheus port to use
port: 9989
## @param downstream-spire-agent-bottom-turtle-ha-a.socketPath Socket path to use
socketPath: /var/run/spire/agent/sockets/a/csi.spiffe.io/public/spire-agent.sock
sockets:
## @param downstream-spire-agent-bottom-turtle-ha-a.sockets.hostBasePath Path on the host to place sockets
hostBasePath: /var/run/spire/agent/sockets/a
admin:
## @param downstream-spire-agent-bottom-turtle-ha-a.sockets.admin.enabled Enable admin socket
enabled: true
## @param downstream-spire-agent-bottom-turtle-ha-a.sockets.admin.mountOnHost Mount admin socket on host
mountOnHost: true
## @param downstream-spire-agent-bottom-turtle-ha-a.authorizedDelegates List of workloads able to use the delegation api
authorizedDelegates:
- /spire-ha-agent
# Used with tags [bottomTurtleHAB]
downstream-spire-agent-bottom-turtle-ha-b:
# enabled: true
## @param downstream-spire-agent-bottom-turtle-ha-b.nameOverride Overrides the name of Spire agent pods
nameOverride: agent-downstream
server:
## @param downstream-spire-agent-bottom-turtle-ha-b.server.nameOverride The name override setting of the internal SPIRE server
nameOverride: internal-server
## @param downstream-spire-agent-bottom-turtle-ha-b.bundleConfigMap The name of the configmap that contains the downstream bundle
bundleConfigMap: spire-server-b-bundle
## @param downstream-spire-agent-bottom-turtle-ha-b.persistence.hostPath Which path to use on the host when persistence.type = hostPath
persistence:
hostPath: /var/lib/spire/k8s/downstream-agent-b
dynamicRegistration:
## @param downstream-spire-agent-bottom-turtle-ha-b.dynamicRegistration.enabled Enable dynamic registration
enabled: true
## @param downstream-spire-agent-bottom-turtle-ha-b.dynamicRegistration.nameOverride The name override to use to contact the server
nameOverride: internal-server
nodeAttestor:
k8sPSAT:
## @param downstream-spire-agent-bottom-turtle-ha-b.nodeAttestor.k8sPSAT.enabled Enable the k8s projected access token node attestor
enabled: false
x509POP:
## @param downstream-spire-agent-bottom-turtle-ha-b.nodeAttestor.x509POP.enabled Enable the x509 pop node attestor
enabled: true
## @param downstream-spire-agent-bottom-turtle-ha-b.nodeAttestor.x509POP.spiffeEndpointSocket Where the socket is to use for mode spiffe
spiffeEndpointSocket: /var/run/spiffe/socat/unix/k8s-spire-agent-b/public/api.sock
keyManager:
memory:
## @param downstream-spire-agent-bottom-turtle-ha-b.keyManager.memory.enabled Enable the memory based Key Manager
enabled: false
disk:
## @param downstream-spire-agent-bottom-turtle-ha-b.keyManager.disk.enabled Enable the disk key manager
enabled: true
## @param downstream-spire-agent-bottom-turtle-ha-b.keyManager.disk.mode Where the disk plugin will write out its data
mode: emptyDir
healthChecks:
## @param downstream-spire-agent-bottom-turtle-ha-b.healthChecks.port Health check port
port: 9982
telemetry:
prometheus:
## @param downstream-spire-agent-bottom-turtle-ha-b.telemetry.prometheus.port Prometheus port to use
port: 9990
## @param downstream-spire-agent-bottom-turtle-ha-b.socketPath Socket path to use
socketPath: /var/run/spire/agent/sockets/b/csi.spiffe.io/public/spire-agent.sock
sockets:
## @param downstream-spire-agent-bottom-turtle-ha-b.sockets.hostBasePath Path on the host to place sockets
hostBasePath: /var/run/spire/agent/sockets/b
admin:
## @param downstream-spire-agent-bottom-turtle-ha-b.sockets.admin.enabled Enable admin socket
enabled: true
## @param downstream-spire-agent-bottom-turtle-ha-b.sockets.admin.mountOnHost Mount admin socket on host
mountOnHost: true
## @param downstream-spire-agent-bottom-turtle-ha-b.authorizedDelegates List of workloads able to use the delegation api
authorizedDelegates:
- /spire-ha-agent
# Used with tags [bottomTurtleHAA]
downstream-spiffe-csi-driver-bottom-turtle-ha-a:
# enabled: true
## @param downstream-spiffe-csi-driver-bottom-turtle-ha-a.fullnameOverride Fullname override
fullnameOverride: spiffe-csi-driver-downstream-a
## @param downstream-spiffe-csi-driver-bottom-turtle-ha-a.agentSocketPath path to agent socket
agentSocketPath: /var/run/spire/agent/sockets/a/csi.spiffe.io/public/spire-agent.sock
## @param downstream-spiffe-csi-driver-bottom-turtle-ha-a.pluginName The name of the plugin instance
pluginName: a.csi.spiffe.io
healthChecks:
## @param downstream-spiffe-csi-driver-bottom-turtle-ha-a.healthChecks.port The health check port
port: 9814
# Used with tags [bottomTurtleHAB]
downstream-spiffe-csi-driver-bottom-turtle-ha-b:
# enabled: true
## @param downstream-spiffe-csi-driver-bottom-turtle-ha-b.fullnameOverride Fullname override
fullnameOverride: spiffe-csi-driver-downstream-b
## @param downstream-spiffe-csi-driver-bottom-turtle-ha-b.agentSocketPath path to agent socket
agentSocketPath: /var/run/spire/agent/sockets/b/csi.spiffe.io/public/spire-agent.sock
## @param downstream-spiffe-csi-driver-bottom-turtle-ha-b.pluginName The name of the plugin instance
pluginName: b.csi.spiffe.io
healthChecks:
## @param downstream-spiffe-csi-driver-bottom-turtle-ha-b.healthChecks.port The health check port
port: 9816
+1 -1
View File
@@ -1,6 +1,6 @@
# spire
![Version: 0.28.5](https://img.shields.io/badge/Version-0.28.5-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.14.5](https://img.shields.io/badge/AppVersion-1.14.5-informational?style=flat-square)
![Version: 0.28.5](https://img.shields.io/badge/Version-0.28.5-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.15.1](https://img.shields.io/badge/AppVersion-1.15.1-informational?style=flat-square)
[![Development Phase](https://github.com/spiffe/spiffe/blob/main/.img/maturity/dev.svg)](https://github.com/spiffe/spiffe/blob/main/MATURITY.md#development)
A Helm chart for deploying the complete Spire stack including: spire-server, spire-agent, spiffe-csi-driver, spiffe-oidc-discovery-provider and spire-controller-manager.
@@ -3,7 +3,7 @@ name: spiffe-oidc-discovery-provider
description: A Helm chart to install the SPIFFE OIDC discovery provider.
type: application
version: 0.1.0
appVersion: "1.14.5"
appVersion: "1.15.1"
keywords: ["spiffe", "oidc"]
home: https://github.com/spiffe/helm-charts-hardened/tree/main/charts/spire
sources:
+1 -1
View File
@@ -3,7 +3,7 @@ name: spire-agent
description: A Helm chart to install the SPIRE agent.
type: application
version: 0.1.0
appVersion: "1.14.5"
appVersion: "1.15.1"
keywords: ["spiffe", "spire-agent"]
home: https://github.com/spiffe/helm-charts-hardened/tree/main/charts/spire
sources:
+15 -1
View File
@@ -26,7 +26,7 @@ A Helm chart to install the SPIRE agent.
### Chart parameters
| Name | Description | Value |
| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| `image.registry` | The OCI registry to pull the image from | `ghcr.io` |
| `image.repository` | The repository within the registry | `spiffe/spire-agent` |
| `image.pullPolicy` | The image pull policy | `IfNotPresent` |
@@ -76,6 +76,7 @@ A Helm chart to install the SPIRE agent.
| `fsGroupFix.image.tag` | Overrides the image tag whose default is the chart appVersion | `latest@sha256:90041f375e30f41aa7e0390075d8a69dc61900771d52fa98d63ee5d03d866a58` |
| `keyManager.memory.enabled` | Enable the memory based Key Manager | `true` |
| `keyManager.disk.enabled` | Enable the disk based Key Manager (must have persistence.type set to hostPath when enabled) | `false` |
| `keyManager.disk.mode` | Where to store the data. Supported options are hostPath and emptyDir | `hostPath` |
| `nodeAttestor.k8sPSAT.enabled` | Enable PSAT k8s Node Attestor | `true` |
| `nodeAttestor.httpChallenge.enabled` | Enable the http challenge Node Attestor | `false` |
| `nodeAttestor.httpChallenge.agentname` | Name of this agent. Useful if you have multiple agents bound to different spire servers on the same host and sharing the same port. | `default` |
@@ -95,6 +96,9 @@ A Helm chart to install the SPIRE agent.
| `nodeAttestor.tpmDirect.pubHash.image.tag` | Overrides the image tag | `v1.9.0` |
| `nodeAttestor.awsIID.enabled` | Enable the aws_iid Node Attestor | `false` |
| `nodeAttestor.gcpIIT.enabled` | Enable the gcp_iit Node Attestor | `false` |
| `nodeAttestor.x509POP.enabled` | Enable the x509_pop Node Attestor | `false` |
| `nodeAttestor.x509POP.mode` | Which mode to use. Currently only spiffe is supported | `spiffe` |
| `nodeAttestor.x509POP.spiffeEndpointSocket` | Where the socket is to use for mode spiffe | `/var/run/spiffe/socat/unix/k8s-spire-agent/public/api.sock` |
| `workloadAttestors.unix.enabled` | Enables the Unix workload attestor | `false` |
| `workloadAttestors.k8s.enabled` | Enables the Kubernetes workload attestor | `true` |
| `workloadAttestors.k8s.verification.type` | What kind of verification to do against kubelet. auto will first attempt to use hostCert, and then fall back to apiServerCA. Valid options are [auto, hostCert, apiServerCA, skip] | `skip` |
@@ -103,6 +107,16 @@ A Helm chart to install the SPIRE agent.
| `workloadAttestors.k8s.disableContainerSelectors` | Set to true if using holdApplicationUntilProxyStarts in Istio | `false` |
| `workloadAttestors.k8s.useNewContainerLocator` | If true, enables the new container locator algorithm that has support for cgroups v2. Defaults to true | `true` |
| `workloadAttestors.k8s.verboseContainerLocatorLogs` | If true, enables verbose logging of mountinfo and cgroup information used to locate containers. Defaults to false | `false` |
| `dynamicRegistration.enabled` | Deploys the sidecar helper for dynamic registration | `false` |
| `dynamicRegistration.image.registry` | The OCI registry to pull the image from | `ghcr.io` |
| `dynamicRegistration.image.repository` | The repository within the registry | `spiffe/spire-controller-manager-dynamic-registration/spire-controller-manager-dynamic-registration-agent` |
| `dynamicRegistration.image.pullPolicy` | The image pull policy | `IfNotPresent` |
| `dynamicRegistration.image.tag` | Overrides the image tag to be whatever you need it to be. It will always be the flag you set without modifications | `0.1.0` |
| `dynamicRegistration.audience` | The audience to get the k8s psat for | `spire-controller-manager-dynamic-registration` |
| `dynamicRegistration.serverSPIFFEID` | Expected SPIFFE ID of the server. If blank, it will use a sane default. | `""` |
| `dynamicRegistration.address` | Address for Spire server | `""` |
| `dynamicRegistration.nameOverride` | Override the name for Spire server. Should only be changed when building your own nested chart to ensure names align. | `""` |
| `dynamicRegistration.securityContext` | Security context | `{}` |
| `sds.enabled` | Enables Envoy SDS configuration | `false` |
| `sds.defaultSVIDName` | The TLS Certificate resource name to use for the default X509-SVID with Envoy SDS | `default` |
| `sds.defaultBundleName` | The Validation Context resource name to use for the default X.509 bundle with Envoy SDS | `ROOTCA` |
@@ -110,6 +110,18 @@ Create the name of the service account to use
{{- end }}
{{- end }}
{{- define "spire-agent.dynamic-registration-address" }}
{{- if and (ne (len (dig "spire" "upstreamDynamicRegistrationAddress" "" .Values.global)) 0) .Values.upstream }}
{{- print .Values.global.spire.upstreamDynamicRegistrationAddress }}
{{- else if .Values.dynamicRegistration.address }}
{{- .Values.dynamicRegistration.address }}
{{- else if .Values.dynamicRegistration.nameOverride }}
{{- .Release.Name }}-{{ .Values.dynamicRegistration.nameOverride }}.{{ include "spire-agent.server.namespace" . }}
{{- else }}
{{- .Release.Name }}-server.{{ include "spire-agent.server.namespace" . }}
{{- end }}
{{- end }}
{{- define "spire-agent.socket-path" -}}
{{- print .Values.socketPath }}
{{- end }}
@@ -1,6 +1,7 @@
{{- define "spire-agent.check-config-values" -}}
{{- include "spire-lib.check-strict-mode" (list . "clusterName must be set" (eq (include "spire-lib.cluster-name" .) "example-cluster"))}}
{{- include "spire-lib.check-strict-mode" (list . "trustDomain must be set" (eq (include "spire-lib.trust-domain" .) "example.org"))}}
{{- $trustDomain := include "spire-lib.trust-domain" . }}
{{- include "spire-lib.check-strict-mode" (list . "trustDomain must be set" (eq $trustDomain "example.org"))}}
{{- range $type, $tvals := .Values.customPlugins }}
{{- if not (has $type (list "keyManager" "nodeAttestor" "svidStore" "workloadAttestor")) }}
{{- fail (printf "Unknown plugin type specified: %s" $type) }}
@@ -19,7 +20,7 @@
{{- end }}
{{- end }}
{{- end }}
{{- if and .Values.keyManager.disk.enabled (ne .Values.persistence.type "hostPath") }}
{{- if and .Values.keyManager.disk.enabled (ne .Values.persistence.type "hostPath") (eq .Values.keyManager.disk.mode "hostPath") }}
{{- fail "keyManager.disk.enabled is true but persistence.type is not hostPath. Ensure persistence.type is hostPath when keyManager.disk.enabled is true." }}
{{- end }}
{{- if hasPrefix (.Values.socketPath | dir | clean) (.Values.sockets.hostBasePath | clean) }}
@@ -37,13 +38,20 @@
{{- end }}
{{- end }}
{{- define "spire-agent.yaml-config" -}}
{{- $trustDomain := include "spire-lib.trust-domain" . }}
agent:
{{- if .Values.sockets.admin.enabled }}
admin_socket_path: /tmp/spire-agent/private/admin.sock
{{- end }}
{{- with .Values.authorizedDelegates }}
authorized_delegates:
{{- toYaml . | nindent 4 }}
{{- range . }}
{{- if hasPrefix "/" . }}
- spiffe://{{ $trustDomain }}{{ . }}
{{- else }}
- {{ . }}
{{- end }}
{{- end }}
{{- end }}
data_dir: "/var/lib/spire"
log_level: {{ .Values.logLevel | quote }}
@@ -120,6 +128,14 @@ plugins:
{{- $nodeAttestorUsed = add1 $nodeAttestorUsed }}
{{- end }}
{{- end }}
{{- with .Values.nodeAttestor.x509POP }}
{{- if eq (.enabled | toString) "true" }}
x509pop:
plugin_data:
spiffe_endpoint_socket: unix://{{ .spiffeEndpointSocket }}
{{- $nodeAttestorUsed = add1 $nodeAttestorUsed }}
{{- end }}
{{- end }}
{{- with .Values.nodeAttestor.awsIID }}
{{- if eq (.enabled | toString) "true" }}
aws_iid:
@@ -147,7 +163,11 @@ plugins:
{{- if .Values.keyManager.disk.enabled }}
disk:
plugin_data:
{{- if eq .Values.keyManager.disk.mode "hostPath" }}
directory: {{ .Values.persistence.hostPath }}
{{- else if eq .Values.keyManager.disk.mode "emptyDir" }}
directory: /key-manager
{{- end }}
{{- $keyManagerUsed = add1 $keyManagerUsed }}
{{- end }}
{{- if ne $keyManagerUsed 1 }}
@@ -9,7 +9,7 @@
{{- if hasKey .Values.sds "disableSpiffeCertValidation" }}
{{- fail "disableSpiffeCertValidation was renamed to disableSPIFFECertValidation. Please update your config." }}
{{- end }}
{{- if and .Values.keyManager.disk.enabled (ne .Values.persistence.type "hostPath") }}
{{- if and .Values.keyManager.disk.enabled (ne .Values.persistence.type "hostPath") (eq .Values.keyManager.disk.mode "hostPath") }}
{{- fail "keyManager.disk.enabled is true but persistence.type is not hostPath. Ensure persistence.type is hostPath when keyManager.disk.enabled is true." }}
{{- end }}
{{- range $name := (concat (list "default") (keys .Values.agents)) | uniq }}
@@ -303,7 +303,11 @@ spec:
readOnly: true
{{- if .Values.keyManager.disk.enabled }}
- name: spire-key-manager
{{- if eq .Values.keyManager.disk.mode "emptyDir" }}
mountPath: /key-manager
{{- else }}
mountPath: {{ .Values.persistence.hostPath }}
{{- end }}
readOnly: false
{{- end }}
- name: spire-agent-persistence
@@ -340,6 +344,10 @@ spec:
mountPath: /hostCert
readOnly: true
{{- end }}
{{- if .Values.nodeAttestor.x509POP.enabled }}
- name: x509pop-upstream
mountPath: {{ .Values.nodeAttestor.x509POP.spiffeEndpointSocket | dir }}
{{- end }}
{{- if gt (len .Values.extraVolumeMounts) 0 }}
{{- toYaml .Values.extraVolumeMounts | nindent 12 }}
{{- end }}
@@ -355,6 +363,33 @@ spec:
{{- toYaml .Values.readinessProbe | nindent 12 }}
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- if eq (.Values.dynamicRegistration.enabled | toString) "true" }}
- name: dynamic-registration
securityContext:
{{- include "spire-lib.securitycontext-extended" (dict "root" . "securityContext" .Values.dynamicRegistration.securityContext) | nindent 12 }}
image: {{ template "spire-lib.image" (dict "appVersion" .Values.dynamicRegistration.image.tag "image" .Values.dynamicRegistration.image "global" .Values.global "ubi" false) }}
imagePullPolicy: {{ .Values.dynamicRegistration.image.pullPolicy }}
env:
- name: TOKENFILE
value: /var/run/secrets/tokens/dynamic-registration-agent
- name: SPIFFE_TRUST_DOMAIN
value: {{ include "spire-lib.trust-domain" . | quote }}
- name: SERVERSPIFFEID
value: {{ .Values.dynamicRegistration.serverSPIFFEID | quote }}
- name: APIURL
value: {{ include "spire-agent.dynamic-registration-address" . | quote }}
- name: KEYFILE
value: /key-manager/keys.json
- name: APIPORT
value: "8931"
volumeMounts:
- name: spire-agent-persistence
mountPath: /var/lib/spire
- name: dynamic-registration-psat
mountPath: /var/run/secrets/tokens
- name: spire-key-manager
mountPath: /key-manager
{{- end }}
{{- if gt (len .Values.extraContainers) 0 }}
{{- toYaml .Values.extraContainers | nindent 8 }}
{{- end }}
@@ -376,9 +411,13 @@ spec:
name: {{ include "spire-agent.fullname" . }}
{{- if .Values.keyManager.disk.enabled }}
- name: spire-key-manager
{{- if eq .Values.keyManager.disk.mode "hostPath" }}
hostPath:
path: {{ .Values.persistence.hostPath }}
type: DirectoryOrCreate
{{- else }}
emptyDir: {}
{{- end }}
{{- end }}
{{- if .Values.sockets.admin.mountOnHost }}
- name: spire-agent-admin-socket-dir
@@ -420,6 +459,21 @@ spec:
- name: tpm-direct
emptyDir: {}
{{- end }}
{{- if .Values.nodeAttestor.x509POP.enabled }}
- name: x509pop-upstream
hostPath:
path: {{ .Values.nodeAttestor.x509POP.spiffeEndpointSocket | dir }}
type: DirectoryOrCreate
{{- end }}
{{- if eq (.Values.dynamicRegistration.enabled | toString) "true" }}
- name: dynamic-registration-psat
projected:
sources:
- serviceAccountToken:
path: dynamic-registration-agent
expirationSeconds: 7200
audience: {{ .Values.dynamicRegistration.audience | quote }}
{{- end }}
- name: spire-token
projected:
sources:
@@ -168,6 +168,8 @@ keyManager:
disk:
## @param keyManager.disk.enabled Enable the disk based Key Manager (must have persistence.type set to hostPath when enabled)
enabled: false
## @param keyManager.disk.mode Where to store the data. Supported options are hostPath and emptyDir
mode: hostPath
nodeAttestor:
k8sPSAT:
@@ -219,6 +221,13 @@ nodeAttestor:
gcpIIT:
## @param nodeAttestor.gcpIIT.enabled Enable the gcp_iit Node Attestor
enabled: false
x509POP:
## @param nodeAttestor.x509POP.enabled Enable the x509_pop Node Attestor
enabled: false
## @param nodeAttestor.x509POP.mode Which mode to use. Currently only spiffe is supported
mode: spiffe
## @param nodeAttestor.x509POP.spiffeEndpointSocket Where the socket is to use for mode spiffe
spiffeEndpointSocket: "/var/run/spiffe/socat/unix/k8s-spire-agent/public/api.sock"
# workloadAttestors determine a workload's properties and then generate a set of selectors associated with it.
workloadAttestors:
@@ -244,6 +253,34 @@ workloadAttestors:
## @param workloadAttestors.k8s.verboseContainerLocatorLogs If true, enables verbose logging of mountinfo and cgroup information used to locate containers. Defaults to false
verboseContainerLocatorLogs: false
dynamicRegistration:
## @param dynamicRegistration.enabled Deploys the sidecar helper for dynamic registration
enabled: false
## @param dynamicRegistration.image.registry The OCI registry to pull the image from
## @param dynamicRegistration.image.repository The repository within the registry
## @param dynamicRegistration.image.pullPolicy The image pull policy
## @param dynamicRegistration.image.tag Overrides the image tag to be whatever you need it to be. It will always be the flag you set without modifications
##
image:
registry: ghcr.io
repository: spiffe/spire-controller-manager-dynamic-registration/spire-controller-manager-dynamic-registration-agent
pullPolicy: IfNotPresent
tag: "0.1.0"
## @param dynamicRegistration.audience The audience to get the k8s psat for
audience: spire-controller-manager-dynamic-registration
## @param dynamicRegistration.serverSPIFFEID Expected SPIFFE ID of the server. If blank, it will use a sane default.
serverSPIFFEID: ""
## @param dynamicRegistration.address Address for Spire server
address: ""
## @param dynamicRegistration.nameOverride Override the name for Spire server. Should only be changed when building your own nested chart to ensure names align.
nameOverride: ""
## @param dynamicRegistration.securityContext [object] Security context
securityContext: {}
sds:
## @param sds.enabled Enables Envoy SDS configuration
enabled: false
+1 -1
View File
@@ -3,7 +3,7 @@ name: spire-server
description: A Helm chart to install the SPIRE server.
type: application
version: 0.1.0
appVersion: "1.14.5"
appVersion: "1.15.1"
keywords: ["spiffe", "spire-server", "spire-controller-manager"]
home: https://github.com/spiffe/helm-charts-hardened/tree/main/charts/spire
sources:
+35 -2
View File
@@ -80,7 +80,7 @@ In order to run Tornjak with simple HTTP Connection only, make sure you don't cr
### Chart parameters
| Name | Description | Value |
| -------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| -------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `replicaCount` | SPIRE server currently runs with a sqlite database. Scaling to multiple instances will not work until we use an external database. | `1` |
| `image.registry` | The OCI registry to pull the image from | `ghcr.io` |
| `image.repository` | The repository within the registry | `spiffe/spire-server` |
@@ -122,7 +122,7 @@ In order to run Tornjak with simple HTTP Connection only, make sure you don't cr
| `livenessProbe.timeoutSeconds` | Timeout in seconds for livenessProbe | `3` |
| `readinessProbe.initialDelaySeconds` | Initial delay seconds for readinessProbe | `5` |
| `readinessProbe.periodSeconds` | Period seconds for readinessProbe | `5` |
| `persistence.type` | What type of volume to use for persistence. Valid options pvc (recommended), hostPath, emptyDir (testing only) | `pvc` |
| `persistence.type` | What type of volume to use for persistence. Valid options pvc (recommended), hostPath, emptyDir (testing or nested child only) | `pvc` |
| `persistence.size` | What size volume to use for persistence | `1Gi` |
| `persistence.accessMode` | What access mode to use for persistence. Valid options are ReadWriteOnce (recommended), ReadWriteOncePod, ReadWriteMany (not recommended) | `ReadWriteOnce` |
| `persistence.storageClass` | What storage class to use for persistence | `nil` |
@@ -348,6 +348,10 @@ In order to run Tornjak with simple HTTP Connection only, make sure you don't cr
| `controllerManager.identities.clusterSPIFFEIDs.spike-pilot.enabled` | Enable this identity for controller manager | `true` |
| `controllerManager.identities.clusterSPIFFEIDs.spike-pilot.type` | The type of rule this is. | `spike-pilot` |
| `controllerManager.identities.clusterSPIFFEIDs.spike-pilot.spiffeIDTemplate` | The template to use for this rule. | `spiffe://{{ .TrustDomain }}/spike/pilot/role/superuser` |
| `controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.enabled` | Enable this identity for controller manager | `false` |
| `controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.type` | The type of rule this is. | `spire-ha-agent` |
| `controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.spiffeIDTemplate` | The template to use for this rule. | `spiffe://{{ .TrustDomain }}/spire-ha-agent` |
| `controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.federatesWith` | Federated trust domains to pass to the workload | `["spire-ha"]` |
| `controllerManager.identities.clusterStaticEntries` | Specify ClusterStaticEntry objects. | `{}` |
| `controllerManager.identities.clusterFederatedTrustDomains` | Specify ClusterFederatedTrustDomain objects. | `{}` |
| `controllerManager.validatingWebhookConfiguration.enabled` | Disable only when you have another chart instance on the k8s cluster with webhooks enabled. | `true` |
@@ -451,6 +455,14 @@ In order to run Tornjak with simple HTTP Connection only, make sure you don't cr
| `nodeAttestor.gcpIIT.allowedMetadataKeys` | Instance metadata keys considered for selectors | `[]` |
| `nodeAttestor.gcpIIT.metadataValueMaxSize` | Sets the maximum metadata value size considered by the plugin for selectors | `0` |
| `nodeAttestor.gcpIIT.agentPathTemplate` | A URL path portion format of Agent's SPIFFE ID. Describe in text/template format. | `""` |
| `nodeAttestor.x509POP.enabled` | Enable the x509_popg node attestor | `false` |
| `nodeAttestor.x509POP.mode` | What mode to set the plugin to. Currently only spiffe mode is supported | `spiffe` |
| `nodeAttestor.x509POP.svidPrefix` | What prefix to use when mode is spiffe | `/spire-exchange` |
| `nodeAttestor.x509POP.agentPathTemplate` | Override the default agent path template | `""` |
| `nodeAttestor.x509POP.maxIntermediates` | Maximum number of intermediate certificates allowed in the certificate chain | `4` |
| `nodeAttestor.x509POP.maxRSAKeySize` | Maximum RSA key size in bits allowed in certificates | `8192` |
| `nodeAttestor.x509POP.addClusterName.svidPrefix` | Suffix the cluster name onto the svidPrefix | `false` |
| `nodeAttestor.x509POP.addClusterName.agentPathTemplate` | Suffix the cluster name onto the agentPathTemplate | `false` |
| `bundlePublisher.k8sConfigMap.enabled` | Enable local k8s bundle uploader | `true` |
| `bundlePublisher.k8sConfigMap.namespace` | Namespace to push the bundle into, if blank will default to SPIRE Server namespace | `""` |
| `bundlePublisher.k8sConfigMap.format` | Format of the trust bundle. Can be pem or spiffe | `spiffe` |
@@ -473,6 +485,19 @@ In order to run Tornjak with simple HTTP Connection only, make sure you don't cr
| `bundlePublisher.gcpCloudStorage.bucketName` | Google Cloud Storage bucket name to which the trust bundle is uploaded | `""` |
| `bundlePublisher.gcpCloudStorage.objectName` | Google Cloud Storage object name | `""` |
| `bundlePublisher.gcpCloudStorage.format` | Format in which the trust bundle is stored. Valid options [spiffe, jwks, pem] | `""` |
| `dynamicRegistration.enabled` | Deploys the sidecar helper for dynamic registration | `false` |
| `dynamicRegistration.image.registry` | The OCI registry to pull the image from | `ghcr.io` |
| `dynamicRegistration.image.repository` | The repository within the registry | `spiffe/spire-controller-manager-dynamic-registration/spire-controller-manager-dynamic-registration-server` |
| `dynamicRegistration.image.pullPolicy` | The image pull policy | `IfNotPresent` |
| `dynamicRegistration.image.tag` | Overrides the image tag to be whatever you need it to be. It will always be the flag you set without modifications | `0.1.0` |
| `dynamicRegistration.serviceAccount` | Which service account to allow to register | `spire-agent` |
| `dynamicRegistration.audience` | The expected audience | `spire-controller-manager-dynamic-registration` |
| `dynamicRegistration.entryPrefix` | Unique prefix to bind nodes aliases to the server | `scmnr` |
| `dynamicRegistration.allowedIDPrefix` | Prefix of agents that are allowed to register | `spire/agent/k8s_psat` |
| `dynamicRegistration.registrationPrefix` | prefix to use on all new registration entries | `k8s_psat` |
| `dynamicRegistration.addClusterName.registrationPrefix` | suffix the cluster name onto the registrationPrefix | `true` |
| `dynamicRegistration.addClusterName.allowedIDPrefix` | suffix the cluster name onto the allowedIDPrefix | `true` |
| `dynamicRegistration.securityContext` | Security Context to use | `{}` |
### Tornjak
@@ -515,6 +540,13 @@ In order to run Tornjak with simple HTTP Connection only, make sure you don't cr
| `secrets.aws.accessKeyID` | AWS Access Key ID | `""` |
| `secrets.aws.secretAccessKey` | AWS Secret Access Key | `""` |
| `secrets.gcp.applicationCredentials` | Google Application Credentials | `""` |
| `trustSync.enabled` | Allow configuration of trust syncing | `false` |
| `trustSync.domains` | List of trust domains to sync from parent to child servers | `[]` |
| `trustSync.image.registry` | The OCI registry to pull the image from | `ghcr.io` |
| `trustSync.image.repository` | The repository within the registry | `spiffe/spire-ha-agent/spire-trust-sync` |
| `trustSync.image.pullPolicy` | The image pull policy | `IfNotPresent` |
| `trustSync.image.tag` | Overrides the image tag to be whatever you need it to be. It will always be the flag you set without modifications | `0.0.21` |
| `trustSync.resources` | Resource requests and limits | `{}` |
| `customPlugins.bundlePublisher` | Custom plugins of type BundlePublisher are configured here | `{}` |
| `customPlugins.credentialComposer` | Custom plugins of type CredentialComposer are configured here | `{}` |
| `customPlugins.keyManager` | Custom plugins of type KeyManager are configured here | `{}` |
@@ -527,6 +559,7 @@ In order to run Tornjak with simple HTTP Connection only, make sure you don't cr
| `chown.image.tag` | Overrides the image tag whose default is the chart appVersion | `1.37.0-uclibc` |
| `chown.resources` | Specify resource needs as per https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ | `{}` |
| `experimental.enabled` | Allow configuration of experimental features | `false` |
| `experimental.agentSPIFFEIDAsSelector` | enable adding spiffe_id selectors to all agents | `false` |
| `experimental.cacheReloadInterval` | The amount of time between two reloads of the in-memory entry cache. | `5s` |
| `experimental.eventsBasedCache` | Use events to update the cache with what's changed since the last update. | `false` |
| `experimental.pruneEventsOlderThan` | How old an event can be before being deleted. Used with events based cache. | `12h` |
@@ -289,6 +289,12 @@ Create the name of the service account to use
{{- define "spire-server.upstream-spire-address" }}
{{- if ne (len (dig "spire" "upstreamSpireAddress" "" .Values.global)) 0 }}
{{- print .Values.global.spire.upstreamSpireAddress }}
{{- else if .Values.upstreamAuthority.spire.server.address }}
{{- if contains "." .Values.upstreamAuthority.spire.server.address }}
{{- print .Values.upstreamAuthority.spire.server.address }}
{{- else }}
{{- printf "%s.%s" .Values.upstreamAuthority.spire.server.address (include "spire-lib.trust-domain" .) }}
{{- end }}
{{- else if .Values.upstreamAuthority.spire.server.nameOverride }}
{{- printf "%s-%s" .Release.Name .Values.upstreamAuthority.spire.server.nameOverride }}
{{- else }}
@@ -105,6 +105,7 @@ server:
{{- with .Values.experimental }}
{{- if eq (.enabled | toString) "true" }}
experimental:
agent_spiffe_id_as_selector: {{ .agentSPIFFEIDAsSelector }}
cache_reload_interval: {{ .cacheReloadInterval | quote }}
events_based_cache: {{ .eventsBasedCache }}
prune_events_older_than: {{ .pruneEventsOlderThan | quote }}
@@ -175,7 +176,7 @@ plugins:
{{- end }}
{{- end }}
{{- if or .Values.nodeAttestor.k8sPSAT.enabled .Values.nodeAttestor.externalK8sPSAT.enabled .Values.nodeAttestor.joinToken.enabled .Values.nodeAttestor.httpChallenge.enabled .Values.nodeAttestor.tpmDirect.enabled .Values.nodeAttestor.awsIID.enabled .Values.nodeAttestor.gcpIIT.enabled }}
{{- if or .Values.nodeAttestor.k8sPSAT.enabled .Values.nodeAttestor.externalK8sPSAT.enabled .Values.nodeAttestor.joinToken.enabled .Values.nodeAttestor.httpChallenge.enabled .Values.nodeAttestor.tpmDirect.enabled .Values.nodeAttestor.awsIID.enabled .Values.nodeAttestor.gcpIIT.enabled .Values.nodeAttestor.x509POP.enabled }}
NodeAttestor:
{{- $clusters := default .Values.kubeConfigs .Values.nodeAttestor.externalK8sPSAT.clusters }}
{{- if or (eq (.Values.nodeAttestor.k8sPSAT.enabled | toString) "true") (and (eq (.Values.nodeAttestor.externalK8sPSAT.enabled | toString) "true") (gt (len $clusters) 0)) }}
@@ -255,6 +256,27 @@ plugins:
{{- end }}
{{- end }}
{{- end }}
{{- with .Values.nodeAttestor.x509POP }}
{{- if eq (.enabled | toString) "true" }}
x509pop:
plugin_data:
mode: {{ .mode }}
{{- if .addClusterName.svidPrefix }}
svid_prefix: {{ printf "%s/%s" .svidPrefix (include "spire-lib.cluster-name" $root) | quote }}
{{- else }}
svid_prefix: {{ .svidPrefix | quote }}
{{- end }}
max_intermediates: {{ .maxIntermediates }}
max_rsa_key_size: {{ .maxRSAKeySize }}
{{- if ne .agentPathTemplate "" }}
{{- if .addClusterName.agentPathTemplate }}
agent_path_template: {{ printf "%s/%s" .agentPathTemplate (include "spire-lib.cluster-name" $root) | quote }}
{{- else }}
agent_path_template: {{ .agentPathTemplate | quote }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
{{- with .Values.nodeAttestor.awsIID }}
{{- if eq (.enabled | toString) "true" }}
aws_iid:
@@ -12,6 +12,9 @@ values:
{{- if eq .type "child-servers" }}
matchLabels:
component: server
{{- else if eq .type "oidc-discovery-provider-common" }}
matchLabels:
component: oidc-discovery-provider
{{- else if eq .type "oidc-discovery-provider" }}
matchLabels:
release: {{ .Release.Name }}
@@ -42,6 +45,9 @@ matchLabels:
release: {{ .Release.Name }}
release-namespace: {{ .Release.Namespace }}
component: test-keys
{{- else if eq .type "spire-ha-agent" }}
matchLabels:
"app.kubernetes.io/name": spire-ha-agent
{{- else }}
{}
{{- end }}
@@ -58,8 +64,8 @@ matchLabels:
{{- if eq ($root.Values.controllerManager.enabled | toString) "true" }}
{{- if or (not (hasKey $value "enabled")) (eq ($value.enabled | toString) "true") }}
{{- $type := dig "type" "base" $value }}
{{- if not (has $type (list "base" "raw" "child-servers" "oidc-discovery-provider" "spike-keeper" "spike-nexus" "spike-bootstrap" "spike-pilot" "test-keys")) }}
{{- fail (printf "Type given: %s, must be one of [base, raw, child-servers, oidc-discovery-provider, spike-keeper, spike-nexus, spike-bootstrap, spike-pilot, test-keys]" $type) }}
{{- if not (has $type (list "base" "raw" "spire-ha-agent" "child-servers" "oidc-discovery-provider" "oidc-discovery-provider-common" "spike-keeper" "spike-nexus" "spike-bootstrap" "spike-pilot" "test-keys")) }}
{{- fail (printf "Type given: %s, must be one of [base, raw, spire-ha-agent, child-servers, oidc-discovery-provider, oidc-discovery-provider-common, spike-keeper, spike-nexus, spike-bootstrap, spike-pilot, test-keys]" $type) }}
{{- end }}
{{- $namespaceSelector := deepCopy (dig "namespaceSelector" (dict) $value) }}
{{- if ne $type "raw" }}
@@ -64,7 +64,7 @@ roleRef:
name: {{ include "spire-lib.bundle-configmap" . }}
apiGroup: rbac.authorization.k8s.io
{{- end }}
{{- if and .Values.nodeAttestor.k8sPSAT.enabled }}
{{- if or .Values.nodeAttestor.k8sPSAT.enabled .Values.dynamicRegistration.enabled }}
---
# ClusterRole to allow spire-server node attestor to query Token Review API
kind: ClusterRole
@@ -79,11 +79,13 @@ rules:
- watch
- list
- create
{{- if .Values.nodeAttestor.k8sPSAT.enabled }}
- apiGroups: [""]
resources: [nodes, pods]
verbs:
- get
- list
{{- end }}
---
# Binds above cluster role to spire-server service account
kind: ClusterRoleBinding
@@ -396,6 +396,46 @@ spec:
mountPath: /tmp
readOnly: false
{{- include "spire-controller-manager.containers" . | nindent 8 }}
{{- if eq (.Values.dynamicRegistration.enabled | toString) "true" }}
- name: dynamic-registration
securityContext:
{{- include "spire-lib.securitycontext-extended" (dict "root" . "securityContext" .Values.tornjak.securityContext) | nindent 12 }}
image: {{ template "spire-lib.image" (dict "appVersion" .Values.dynamicRegistration.image.tag "image" .Values.dynamicRegistration.image "global" .Values.global "ubi" false) }}
imagePullPolicy: {{ .Values.dynamicRegistration.image.pullPolicy }}
env:
- name: SPIFFE_TRUST_DOMAIN
value: {{ include "spire-lib.trust-domain" . | quote }}
- name: EXPECTED_AUDIENCE
value: {{ .Values.dynamicRegistration.audience | quote }}
- name: EXPECTED_SERVICE_ACCOUNT
{{- if contains ":" .Values.dynamicRegistration.serviceAccount }}
value: {{ .Values.dynamicRegistration.serviceAccount | quote }}
{{- else }}
value: {{ printf "%s:%s" (include "spire-server.agent-namespace" .) .Values.dynamicRegistration.serviceAccount | quote }}
{{- end }}
- name: ALLOWEDID_PREFIX
{{- if .Values.dynamicRegistration.addClusterName.allowedIDPrefix }}
value: {{ printf "%s/%s" .Values.dynamicRegistration.allowedIDPrefix (include "spire-lib.cluster-name" .) | quote }}
{{- else }}
value: {{ .Values.dynamicRegistration.allowedIDPrefix | quote }}
{{- end }}
- name: ENTRY_PREFIX
value: {{ .Values.dynamicRegistration.entryPrefix | quote }}
- name: REGISTRATION_PREFIX
{{- if .Values.dynamicRegistration.addClusterName.registrationPrefix }}
value: {{ printf "%s/%s" .Values.dynamicRegistration.registrationPrefix (include "spire-lib.cluster-name" .) | quote }}
{{- else }}
value: {{ .Values.dynamicRegistration.registrationPrefix | quote }}
{{- end }}
volumeMounts:
- name: spire-server-socket
mountPath: /tmp/spire-server/private
readOnly: true
ports:
- name: dynamic-https
containerPort: 8931
protocol: TCP
{{- end }}
{{- if eq (.Values.tornjak.enabled | toString) "true" }}
- name: tornjak
securityContext:
@@ -445,6 +485,29 @@ spec:
mountPath: /opt/spire/user
{{- end }}
{{- end }}
{{- if eq (.Values.trustSync.enabled | toString) "true" }}
- name: spire-trust-sync
image: {{ template "spire-lib.image" (dict "appVersion" .Values.trustSync.image.defaultTag "image" .Values.trustSync.image "global" .Values.global "ubi" false) }}
imagePullPolicy: {{ .Values.trustSync.image.pullPolicy }}
args:
- -trust-domains={{ join "," .Values.trustSync.domains }}
env:
- name: SPIFFE_TRUST_DOMAIN
value: {{ include "spire-lib.trust-domain" . | quote }}
- name: SPIRE_SERVER_SOCKET
value: unix:///tmp/spire-server/private/api.sock
- name: SPIFFE_ENDPOINT_SOCKET
value: unix:///run/spire/upstream_agent/spire-agent.sock
volumeMounts:
- mountPath: /run/spire/upstream_agent
name: upstream-agent
readOnly: true
- mountPath: /tmp/spire-server/private
name: spire-server-socket
readOnly: true
securityContext:
{{- include "spire-lib.securitycontext" . | nindent 12 }}
{{- end }}
{{- if gt (len .Values.extraContainers) 0 }}
{{- toYaml .Values.extraContainers | nindent 8 }}
@@ -28,6 +28,14 @@ spec:
protocol: TCP
{{- end }}
{{- end }}
{{- with .Values.dynamicRegistration }}
{{- if eq (.enabled | toString) "true" }}
- name: dynamic-registration
port: 8931
targetPort: dynamic-https
protocol: TCP
{{- end }}
{{- end }}
selector:
{{- include "spire-server.selectorLabels" . | nindent 4 }}
{{- end }}
+98 -1
View File
@@ -144,7 +144,7 @@ readinessProbe:
initialDelaySeconds: 5
periodSeconds: 5
## @param persistence.type What type of volume to use for persistence. Valid options pvc (recommended), hostPath, emptyDir (testing only)
## @param persistence.type What type of volume to use for persistence. Valid options pvc (recommended), hostPath, emptyDir (testing or nested child only)
## @param persistence.size What size volume to use for persistence
## @param persistence.accessMode What access mode to use for persistence. Valid options are ReadWriteOnce (recommended), ReadWriteOncePod, ReadWriteMany (not recommended)
## @param persistence.storageClass What storage class to use for persistence
@@ -788,6 +788,17 @@ controllerManager:
## @param controllerManager.identities.clusterSPIFFEIDs.spike-pilot.spiffeIDTemplate The template to use for this rule.
spiffeIDTemplate: spiffe://{{ .TrustDomain }}/spike/pilot/role/superuser
spire-ha-agent:
## @param controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.enabled Enable this identity for controller manager
enabled: false
## @param controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.type The type of rule this is.
type: spire-ha-agent
## @param controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.spiffeIDTemplate The template to use for this rule.
spiffeIDTemplate: spiffe://{{ .TrustDomain }}/spire-ha-agent
## @param controllerManager.identities.clusterSPIFFEIDs.spire-ha-agent.federatesWith Federated trust domains to pass to the workload
federatesWith:
- spire-ha
# You can specify additional ClusterSPIFFEIDs following this example:
# foo:
# labels:
@@ -1109,6 +1120,24 @@ nodeAttestor:
metadataValueMaxSize: 0
## @param nodeAttestor.gcpIIT.agentPathTemplate A URL path portion format of Agent's SPIFFE ID. Describe in text/template format.
agentPathTemplate: ""
x509POP:
## @param nodeAttestor.x509POP.enabled Enable the x509_popg node attestor
enabled: false
## @param nodeAttestor.x509POP.mode What mode to set the plugin to. Currently only spiffe mode is supported
mode: spiffe
## @param nodeAttestor.x509POP.svidPrefix What prefix to use when mode is spiffe
svidPrefix: /spire-exchange
## @param nodeAttestor.x509POP.agentPathTemplate Override the default agent path template
agentPathTemplate: ""
## @param nodeAttestor.x509POP.maxIntermediates Maximum number of intermediate certificates allowed in the certificate chain
maxIntermediates: 4
## @param nodeAttestor.x509POP.maxRSAKeySize Maximum RSA key size in bits allowed in certificates
maxRSAKeySize: 8192
addClusterName:
## @param nodeAttestor.x509POP.addClusterName.svidPrefix Suffix the cluster name onto the svidPrefix
svidPrefix: false
## @param nodeAttestor.x509POP.addClusterName.agentPathTemplate Suffix the cluster name onto the agentPathTemplate
agentPathTemplate: false
# The secrets needed for this plugin are configured in the secrets: section
bundlePublisher:
@@ -1166,6 +1195,40 @@ bundlePublisher:
## @param bundlePublisher.gcpCloudStorage.format Format in which the trust bundle is stored. Valid options [spiffe, jwks, pem]
format: ""
dynamicRegistration:
## @param dynamicRegistration.enabled Deploys the sidecar helper for dynamic registration
enabled: false
## @param dynamicRegistration.image.registry The OCI registry to pull the image from
## @param dynamicRegistration.image.repository The repository within the registry
## @param dynamicRegistration.image.pullPolicy The image pull policy
## @param dynamicRegistration.image.tag Overrides the image tag to be whatever you need it to be. It will always be the flag you set without modifications
##
image:
registry: ghcr.io
repository: spiffe/spire-controller-manager-dynamic-registration/spire-controller-manager-dynamic-registration-server
pullPolicy: IfNotPresent
tag: "0.1.0"
## @param dynamicRegistration.serviceAccount Which service account to allow to register
serviceAccount: "spire-agent"
## @param dynamicRegistration.audience The expected audience
audience: spire-controller-manager-dynamic-registration
## @param dynamicRegistration.entryPrefix Unique prefix to bind nodes aliases to the server
entryPrefix: "scmnr"
## @param dynamicRegistration.allowedIDPrefix Prefix of agents that are allowed to register
allowedIDPrefix: "spire/agent/k8s_psat"
## @param dynamicRegistration.registrationPrefix prefix to use on all new registration entries
registrationPrefix: "k8s_psat"
addClusterName:
## @param dynamicRegistration.addClusterName.registrationPrefix suffix the cluster name onto the registrationPrefix
registrationPrefix: true
## @param dynamicRegistration.addClusterName.allowedIDPrefix suffix the cluster name onto the allowedIDPrefix
allowedIDPrefix: true
## @param dynamicRegistration.securityContext [object] Security Context to use
securityContext: {}
## @section Tornjak
tornjak:
## @param tornjak.enabled Deploys Tornjak API (backend) (Not for production)
@@ -1291,6 +1354,38 @@ secrets:
## @param secrets.gcp.applicationCredentials Google Application Credentials
applicationCredentials: ""
trustSync:
## @param trustSync.enabled Allow configuration of trust syncing
enabled: false
## @param trustSync.domains List of trust domains to sync from parent to child servers
domains: []
#- spire-ha
#- foo.org
## @param trustSync.image.registry The OCI registry to pull the image from
## @param trustSync.image.repository The repository within the registry
## @param trustSync.image.pullPolicy The image pull policy
## @param trustSync.image.tag Overrides the image tag to be whatever you need it to be. It will always be the flag you set without modifications
##
image:
registry: ghcr.io
repository: spiffe/spire-ha-agent/spire-trust-sync
pullPolicy: IfNotPresent
tag: "0.0.21"
## @param trustSync.resources [object] Resource requests and limits
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
# NOTE: This is unsupported and only to configure currently supported spire built in plugins but plugins unsupported by the chart.
# Upgrades wont be tested for anything under this config. If you need this, please let the chart developers know your needs so we
# can prioritize proper support.
@@ -1336,6 +1431,8 @@ chown:
experimental:
## @param experimental.enabled Allow configuration of experimental features
enabled: false
## @param experimental.agentSPIFFEIDAsSelector enable adding spiffe_id selectors to all agents
agentSPIFFEIDAsSelector: false
## @param experimental.cacheReloadInterval The amount of time between two reloads of the in-memory entry cache.
cacheReloadInterval: 5s
## @param experimental.eventsBasedCache Use events to update the cache with what's changed since the last update.
+138
View File
@@ -0,0 +1,138 @@
## Kubernetes Bottom Turtle HA Setup
In this setup, a bottom turtle HA setup based on spire-ha-agent and then Kubernetes based access is built from the ground up.
What does this mean?
The bottom turtle:
There is a pair of spire servers deployed. Trust is established between the two servers creating an HA Trust Domain without needing any 3rd party
trust sources.
A spire-ha-agent, a spire-agent@a and a spire-agent@b is run on the k8s hosts. This provides a bottom turtle trust source between the services on the os Kubernetes
runs on.
Host services can then use this trust chain to secure communications such as:
* kubelet -> kube-apiserver
* sshd
* log shipper -> centeralized log processor
* os level metrics
* etc
We will not discuss how to do that here, but need to utilize this base to establish trust inside of Kubernetes.
We will bridge os to Kubernetes cluster with some configuration on the host, and deploying the helm charts to utilize and export new services on top.
What do we need to do?
There are two different kinds of services that need permission bridging.
* SPIRE Servers
* Downstream agents
### Root Servers
Setup a pair of HA root servers as described here:
https://github.com/spiffe/bootc/tree/main/demo
Root Servers, A and B:
![Diagram](final-pi5.jpg)
![Diagram](final-pi5.jpg)
### K8s SPIRE Servers
In the following diagram, we see all the parts involved from getting the K8s SPIRE Servers running on the control plane nodes.
![Diagram](diagram-cp.png)
We need to be able to use the hosts workload attestors to attest the SPIRE Servers running inside Kubernetes.
To do so, we will define a workload on the root spire servers, and inject it into the spire servers inside Kubernetes.
Example workload definition:
```
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterStaticEntry
metadata:
name: node1-k8s-spire-server
spec:
parentID: spiffe://${SPIFFE_TRUST_DOMAIN}/node/node1.${SPIFFE_TRUST_DOMAIN}
spiffeID: spiffe://${SPIFFE_TRUST_DOMAIN}/k8s-spire-server/server-${SUBINSTANCE}
downstream: true
selectors:
- systemd:id:spiffe-socat-unix@k8s-spire-server-${SUBINSTANCE}.service
federatesWith:
- spire-ha
```
And on the host, we install spiffe-socat-unix via packages, and then enable the bridges:
```
spiffe-socat-unix@k8s-spire-server-a.service
spiffe-socat-unix@k8s-spire-server-b.service
```
Any process that can access the unix socket will be able to become a spire downstream server. Treat this socket with great care.
Consider only doing this on your control plane nodes, and restricting the spire-server to only run on the control plane nodes for extra isolation.
### Downstream agents
In the following diagram we show how a worker node is aranged.
![Diagram](diagram-worker.png)
We need to be able to use the hosts workload attestors to attest the SPIRE Agents running inside Kubernetes.
To do so, we will define a workload on the root spire servers, and inject it into the spire agents inside Kubernetes.
Example workload definition:
```
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterStaticEntry
metadata:
name: node1-k8s-spire-agent
spec:
parentID: spiffe://${SPIFFE_TRUST_DOMAIN}/node/node1.${SPIFFE_TRUST_DOMAIN}
spiffeID: spiffe://${SPIFFE_TRUST_DOMAIN}/spire-exchange/node1.${SPIFFE_TRUST_DOMAIN}
selectors:
- systemd:id:spiffe-socat-unix@k8s-spire-agent-${SUBINSTANCE}.service
```
And on the host, we install spiffe-socat-unix via packages, and then enable the bridges:
```
spiffe-socat-unix@k8s-spire-agent-a.service
spiffe-socat-unix@k8s-spire-agent-b.service
```
## Install the charts:
We need to install 4 charts.
* spire crds
* side A
* side B
* the common infrasctructure
This allows upgrading Side A or Side B completely independencly from each other, ensuring if there is a problem it will not affect production.
Setup the spire-values.yaml as needed.
```
# Install the common components
helm upgrade --install --create-namespace --namespace spire-mgmt --values "spire-values.yaml" \
spire oci://ghcr.io/spiffe/helm-charts/spire-nested \
--set tags.haAgentCommon=true \
--set "global.spire.namespaces.create=true" \
--set "global.spire.ingressControllerType=ingress-nginx" \
--set "spiffe-oidc-discovery-provider.ingress.enabled=true"
# Install server side a
helm upgrade --install --namespace spire-mgmt --values "spire-values.yaml" \
--wait spire-a oci://ghcr.io/spiffe/helm-charts/spire-nested \
--set tags.bottomTurtleHAA=true \
--set "global.spire.ingressControllerType=ingress-nginx"
# Install server side b
helm upgrade --install --namespace spire-mgmt --values "spire-values.yaml" \
--wait spire-b oci://ghcr.io/spiffe/helm-charts/spire-nested \
--set tags.bottomTurtleHAB=true \
--set "global.spire.ingressControllerType=ingress-nginx"
```
+136
View File
@@ -0,0 +1,136 @@
digraph G {
compound=true;
# --- ROOT SERVERS ---
subgraph cluster_server1 {
label = "node name: n1"
style="filled,solid,bold";
color="#b3b3b3";
fillcolor="#f5f5f5";
labeljust="l";
spire_server_1[label=<<table border="0"><tr><td><b>SPIRE Server A</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestors: tpm,x509pop</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#6c8ebf",fillcolor="#dae8fc"]
}
subgraph cluster_server2 {
label = "node name: n2"
style="filled,solid,bold";
color="#b3b3b3";
fillcolor="#f5f5f5";
labeljust="l";
spire_server_2[label=<<table border="0"><tr><td><b>SPIRE Server B</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestors: tpm,x509pop</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#6c8ebf",fillcolor="#dae8fc"]
}
# --- NODE N3 (SPIRE SERVER A PIPELINE) ---
subgraph cluster_node3 {
label = "node name: n3"
style="filled,solid,bold";
color="#b3b3b3";
fillcolor="#f5f5f5";
labeljust="l";
subgraph cluster_node3_systemd {
label = "systemd managed"
style = "dashed,filled"
color="#939393";
fillcolor="#d5d5d5";
labeljust="l";
spire_agent1[label=<<table border="0"><tr><td><b>SPIRE Agent A</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestor: tpm</font></td></tr><tr><td align="left"><font point-size="9">WorkloadAttestor: systemd</font></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
spire_agent2_n3[label=<<table border="0"><tr><td><b>SPIRE Agent B</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestor: tpm</font></td></tr><tr><td align="left"><font point-size="9">WorkloadAttestor: systemd</font></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
subgraph cluster_tb_n3 {
label=""
style="invis"
spire_ha_agent_n3[label=<<table border="0"><tr><td><b>SPIRE HA Agent</b></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
}
sshd1_n3[label="sshd",shape="box",style="rounded,solid,filled,bold",color="#d6b656",fillcolor="#fff2cc"]
kubelet1_n3[label="kubelet",shape="box",style="rounded,solid,filled,bold",color="#d6b656",fillcolor="#fff2cc"]
k8s_spire_server_a[label=<<table border="0"><tr><td><b>K8s SPIRE Server A Identity</b></td></tr><tr><td align="left"><font point-size="9">systemd: spiffe-socat-unix@k8s-spire-server-a</font></td></tr><tr><td align="left"><font point-size="9">tool: socat</font></td></tr></table>>,shape="box",style="rounded,solid,filled,bold",color="#d79b00",fillcolor="#ffe6cc"]
}
subgraph cluster_node3_k8s {
label = "k8s managed"
style = "dashed,filled"
color="#939393";
fillcolor="#d5d5d5";
labeljust="l";
k8s_upstream_csi_a[label=<<table border="0"><tr><td><b>Upstream CSI A</b></td></tr><tr><td align="left"><font point-size="9">path: /var/run/spiffe/socat/unix/k8s-spire-server-a/public/agent.sock</font></td></tr></table>>,shape="box",style="rounded,solid,filled,bold",color="#d79b00",fillcolor="#ffe6cc"]
spire_server_n1[label=<<table border="0"><tr><td><b>K8s SPIRE Server A</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestors: x509pop</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#6c8ebf",fillcolor="#dae8fc"]
}
}
# --- NODE N4 (SPIRE SERVER B PIPELINE) ---
subgraph cluster_node4 {
label = "node name: n4"
style="filled,solid,bold";
color="#b3b3b3";
fillcolor="#f5f5f5";
labeljust="l";
subgraph cluster_node4_systemd {
label = "systemd managed"
style = "dashed,filled"
color="#939393";
fillcolor="#d5d5d5";
labeljust="l";
spire_agent1_n4[label=<<table border="0"><tr><td><b>SPIRE Agent A</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestor: tpm</font></td></tr><tr><td align="left"><font point-size="9">WorkloadAttestor: systemd</font></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
spire_agent2[label=<<table border="0"><tr><td><b>SPIRE Agent B</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestor: tpm</font></td></tr><tr><td align="left"><font point-size="9">WorkloadAttestor: systemd</font></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
subgraph cluster_tb_n4 {
label=""
style="invis"
spire_ha_agent_n4[label=<<table border="0"><tr><td><b>SPIRE HA Agent</b></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
}
sshd1_n4[label="sshd",shape="box",style="rounded,solid,filled,bold",color="#d6b656",fillcolor="#fff2cc"]
kubelet1_n4[label="kubelet",shape="box",style="rounded,solid,filled,bold",color="#d6b656",fillcolor="#fff2cc"]
k8s_spire_server_b[label=<<table border="0"><tr><td><b>K8s SPIRE Server B Identity</b></td></tr><tr><td align="left"><font point-size="9">systemd: spiffe-socat-unix@k8s-spire-server-b</font></td></tr><tr><td align="left"><font point-size="9">tool: socat</font></td></tr></table>>,shape="box",style="rounded,solid,filled,bold",color="#d79b00",fillcolor="#ffe6cc"]
}
subgraph cluster_node4_k8s {
label = "k8s managed"
style = "dashed,filled"
color="#939393";
fillcolor="#d5d5d5";
labeljust="l";
k8s_upstream_csi_b[label=<<table border="0"><tr><td><b>Upstream CSI B</b></td></tr><tr><td align="left"><font point-size="9">path: /var/run/spiffe/socat/unix/k8s-spire-server-b/public/agent.sock</font></td></tr></table>>,shape="box",style="rounded,solid,filled,bold",color="#d79b00",fillcolor="#ffe6cc"]
spire_server_n2[label=<<table border="0"><tr><td><b>K8s SPIRE Server B</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestors: x509pop</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#6c8ebf",fillcolor="#dae8fc"]
}
}
# --- NETWORKING & LINKS ---
# Upstream Core Cross-Links
spire_server_1 -> spire_agent1[dir=back]
spire_server_1 -> spire_agent1_n4[dir=back]
spire_server_2 -> spire_agent2_n3[dir=back]
spire_server_2 -> spire_agent2[dir=back]
# Node 3 Pipelines
spire_agent1 -> spire_ha_agent_n3[dir=back]
spire_agent2_n3 -> spire_ha_agent_n3[dir=back]
spire_ha_agent_n3 -> sshd1_n3[dir=back]
spire_ha_agent_n3 -> kubelet1_n3[dir=back]
spire_agent1 -> k8s_spire_server_a[dir=back]
k8s_spire_server_a -> k8s_upstream_csi_a[dir=back]
k8s_upstream_csi_a -> spire_server_n1[dir=back]
kubelet1_n3 -> k8s_upstream_csi_a [lhead=cluster_node3_k8s, style=dotted]
spire_server_1 -> spire_server_n1 [dir=back]
# Node 4 Pipelines
spire_agent1_n4 -> spire_ha_agent_n4[dir=back]
spire_agent2 -> spire_ha_agent_n4[dir=back]
spire_ha_agent_n4 -> sshd1_n4[dir=back]
spire_ha_agent_n4 -> kubelet1_n4[dir=back]
spire_agent2 -> k8s_spire_server_b[dir=back]
k8s_spire_server_b -> k8s_upstream_csi_b[dir=back]
k8s_upstream_csi_b -> spire_server_n2[dir=back]
kubelet1_n4 -> k8s_upstream_csi_b [lhead=cluster_node4_k8s, style=dotted]
spire_server_2 -> spire_server_n2 [dir=back]
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 168 KiB

@@ -0,0 +1,150 @@
digraph G {
compound=true;
subgraph cluster_server1 {
label = "node name: n1"
#style = dashed
style="filled,solid,bold";
color="#b3b3b3";
fillcolor="#f5f5f5";
labeljust="l";
spire_server_1[label=<<table border="0"><tr><td><b>SPIRE Server A</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestors: tpm</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#6c8ebf",fillcolor="#dae8fc"]
}
subgraph cluster_ps1 {
label = "Control Plane Node: X"
#style = dashed
style="filled,solid,bold";
color="#b3b3b3";
fillcolor="#f5f5f5";
labeljust="l";
pod_spire_server_1[label=<<table border="0"><tr><td><b>K8s SPIRE Server A</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestors: x509pop</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#6c8ebf",fillcolor="#dae8fc"]
}
subgraph cluster_ps2 {
label = "Control Plane Node: Y"
#style = dashed
style="filled,solid,bold";
color="#b3b3b3";
fillcolor="#f5f5f5";
labeljust="l";
pod_spire_server_2[label=<<table border="0"><tr><td><b>K8s SPIRE Server B</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestors: x509pop</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#6c8ebf",fillcolor="#dae8fc"]
}
subgraph cluster_server2 {
label = "node name: n2"
#style = dashed
style="filled,solid,bold";
color="#b3b3b3";
fillcolor="#f5f5f5";
labeljust="l";
spire_server_2[label=<<table border="0"><tr><td><b>SPIRE Server B</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestors: tpm</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#6c8ebf",fillcolor="#dae8fc"]
}
subgraph cluster_node3 {
label = "node name: n3"
#style = dashed
style="filled,solid,bold";
color="#b3b3b3";
fillcolor="#f5f5f5";
labeljust="l";
subgraph cluster_node1_systemd {
#label = "Systemd"
label = "systemd managed"
style = "dashed,filled"
color="#939393";
fillcolor="#d5d5d5";
labeljust="l";
spire_agent1[label=<<table border="0"><tr><td><b>SPIRE Agent A</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestor: tpm</font></td></tr><tr><td align="left"><font point-size="9">WorkloadAttestor: systemd</font></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
spire_agent2[label=<<table border="0"><tr><td><b>SPIRE Agent B</b></td></tr><tr><td align="left"><font point-size="9">NodeAttestor: tpm</font></td></tr><tr><td align="left"><font point-size="9">WorkloadAttestor: systemd</font></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
subgraph cluster_tb {
label=""
style="invis"
spire_ha_agent[label=<<table border="0"><tr><td><b>SPIRE HA Agent</b></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
subgraph cluster_storage {
#spire_ha_agent_state_a[label="Trust Bundle A", shape=note,style="rounded,solid,filled,bold",fillcolor="#ffffff"]
#spire_ha_agent_state_b[label="Trust Bundle B", shape=note,style="rounded,solid,filled,bold",fillcolor="#ffffff"]
}
}
sshd1[label="sshd",shape="box",style="rounded,solid,filled,bold",color="#d6b656",fillcolor="#fff2cc"]
kubelet1[label="kubelet",shape="box",style="rounded,solid,filled,bold",color="#d6b656",fillcolor="#fff2cc"]
k8s_spire_server_a[label=<<table border="0"><tr><td><b>K8s SPIRE Agent A Identity</b></td></tr><tr><td align="left"><font point-size="9">systemd: spiffe-socat-unix@k8s-spire-agent-a</font></td></tr><tr><td align="left"><font point-size="9">tool: socat</font></td></tr></table>>,shape="box",style="rounded,solid,filled,bold",color="#d79b00",fillcolor="#ffe6cc"]
k8s_spire_server_b[label=<<table border="0"><tr><td><b>K8s SPIRE Agent B Identity</b></td></tr><tr><td align="left"><font point-size="9">systemd: spiffe-socat-unix@k8s-spire-agent-b</font></td></tr><tr><td align="left"><font point-size="9">tool: socat</font></td></tr></table>>,shape="box",style="rounded,solid,filled,bold",color="#d79b00",fillcolor="#ffe6cc"]
}
subgraph cluster_node1_k8s {
#label = "Systemd"
label = "k8s managed"
style = "dashed,filled"
color="#939393";
fillcolor="#d5d5d5";
labeljust="l";
// k8s_upstream_csi_a[label=<<table border="0"><tr><td><b>Downstream CSI A</b></td></tr><tr><td align="left"><font point-size="9">path: /var/run/spiffe/socat/unix/k8s-spire-server-a/public/agent.sock</font></td></tr></table>>,shape="box",style="rounded,solid,filled,bold",color="#d79b00",fillcolor="#ffe6cc"]
// k8s_upstream_csi_b[label=<<table border="0"><tr><td><b>Downstream CSI B</b></td></tr><tr><td align="left"><font point-size="9">path: /var/run/spiffe/socat/unix/k8s-spire-server-b/public/agent.sock</font></td></tr></table>>,shape="box",style="rounded,solid,filled,bold",color="#d79b00",fillcolor="#ffe6cc"]
k8s_downstream_csi[label=<<table border="0"><tr><td><b>Downstream CSI</b></td></tr><tr><td align="left"><font point-size="9">path: /var/run/spire/agent-sockets/spire-agent.sock</font></td></tr></table>>,shape="box",style="rounded,solid,filled,bold",color="#d79b00",fillcolor="#ffe6cc"]
spire_server_n1[label=<<table border="0"><tr><td><b>K8s SPIRE Agent A</b></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
spire_server_n2[label=<<table border="0"><tr><td><b>K8s SPIRE Agent B</b></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
spire_ha_agent_pod[label=<<table border="0"><tr><td><b>SPIRE HA Agent</b></td></tr><tr><td align="left"><font point-size="9">systemd: [email protected]</font></td></tr></table>>,shape="record",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"]
pod1[label="pod",shape="box",style="rounded,solid,filled,bold",color="#d6b656",fillcolor="#fff2cc"]
pod2[label="pod",shape="box",style="rounded,solid,filled,bold",color="#d6b656",fillcolor="#fff2cc"]
pod3[label="pod",shape="box",style="rounded,solid,filled,bold",color="#d6b656",fillcolor="#fff2cc"]
}
}
spire_server_1 -> spire_agent1[dir=back]
spire_server_2 -> spire_agent2[dir=back]
spire_agent1 -> spire_ha_agent[dir=back]
spire_agent2 -> spire_ha_agent[dir=back]
spire_ha_agent -> sshd1[dir=back]
spire_ha_agent -> kubelet1[dir=back]
spire_agent1 -> k8s_spire_server_a[dir=back]
spire_agent2 -> k8s_spire_server_b[dir=back]
spire_server_1 -> pod_spire_server_1[dir=back]
spire_server_2 -> pod_spire_server_2[dir=back]
//k8s_spire_server_a -> k8s_upstream_csi_a[dir=back]
k8s_spire_server_a -> spire_server_n1[dir=back]
k8s_spire_server_b -> spire_server_n2[dir=back]
// k8s_upstream_csi_b -> k8s_spire_server_b
// k8s_upstream_csi_a -> spire_server_n1[dir=back]
// k8s_upstream_csi_b -> spire_server_n2[dir=back]
// k8s_spire_server_b -> spire_server_n2[dir=back]
//kubelet1 -> cluster_node1_k8s
kubelet1 -> spire_server_n1 [lhead=cluster_node1_k8s, style=dotted]
pod_spire_server_1 -> spire_server_n1 [dir=back]
pod_spire_server_2 -> spire_server_n2 [dir=back]
//kubelet1 -> spire_server_n1
//kubelet1 -> spire_server_n2
spire_server_n1 -> spire_ha_agent_pod [dir=back]
spire_server_n2 -> spire_ha_agent_pod [dir=back]
spire_ha_agent_pod -> k8s_downstream_csi [dir=back]
k8s_downstream_csi -> pod1 [dir=back]
k8s_downstream_csi -> pod2 [dir=back]
k8s_downstream_csi -> pod3 [dir=back]
// spire_ha_agent -> spire_ha_agent_state[dir=both, constraint=false]
// spire_ha_agent_state_a -> spire_ha_agent_state_b
//spire_agent1 -> spire_ha_agent_state_a
//spire_agent2 -> spire_ha_agent_state_b
//spire_ha_agent_state_a -> spire_ha_agent
//spire_ha_agent_state_b -> spire_ha_agent
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 177 KiB

@@ -0,0 +1,12 @@
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterStaticEntry
metadata:
name: node1-k8s-spire-server
spec:
parentID: spiffe://${SPIFFE_TRUST_DOMAIN}/agent/node1
spiffeID: spiffe://${SPIFFE_TRUST_DOMAIN}/k8s-spire-server/node1
selectors:
- systemd:id:spiffe-socat-unix@k8s-spire-server-${SUBINSTANCE}.service
downstream: true
federatesWith:
- spire-ha
@@ -0,0 +1,9 @@
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterStaticEntry
metadata:
name: node1-spire-trust-sync-a
spec:
parentID: spiffe://${SPIFFE_TRUST_DOMAIN}/agent/node1
spiffeID: spiffe://${SPIFFE_TRUST_DOMAIN}/spire-trust-sync-a
selectors:
- systemd:id:[email protected]
@@ -0,0 +1,9 @@
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterStaticEntry
metadata:
name: node1-spire-trust-sync-b
spec:
parentID: spiffe://${SPIFFE_TRUST_DOMAIN}/agent/node1
spiffeID: spiffe://${SPIFFE_TRUST_DOMAIN}/spire-trust-sync-b
selectors:
- systemd:id:[email protected]
@@ -0,0 +1,9 @@
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterStaticEntry
metadata:
name: node2-k8s-spire-server
spec:
parentID: spiffe://${SPIFFE_TRUST_DOMAIN}/agent/node1
spiffeID: spiffe://${SPIFFE_TRUST_DOMAIN}/spire-exchange/k8s/production/node2.${SPIFFE_TRUST_DOMAIN}
selectors:
- systemd:id:spiffe-socat-unix@k8s-spire-agent-2-${SUBINSTANCE}.service
@@ -0,0 +1,9 @@
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterStaticEntry
metadata:
name: node3-k8s-spire-server
spec:
parentID: spiffe://${SPIFFE_TRUST_DOMAIN}/agent/node1
spiffeID: spiffe://${SPIFFE_TRUST_DOMAIN}/spire-exchange/k8s/production/node3.${SPIFFE_TRUST_DOMAIN}
selectors:
- systemd:id:spiffe-socat-unix@k8s-spire-agent-3-${SUBINSTANCE}.service
@@ -0,0 +1,9 @@
apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterStaticEntry
metadata:
name: node4-k8s-spire-server
spec:
parentID: spiffe://${SPIFFE_TRUST_DOMAIN}/agent/node1
spiffeID: spiffe://${SPIFFE_TRUST_DOMAIN}/spire-exchange/k8s/production/node4.${SPIFFE_TRUST_DOMAIN}
selectors:
- systemd:id:spiffe-socat-unix@k8s-spire-agent-4-${SUBINSTANCE}.service
Binary file not shown.

After

Width:  |  Height:  |  Size: 332 KiB

+293
View File
@@ -0,0 +1,293 @@
#!/usr/bin/env bash
# shellcheck disable=SC2317
set -xe
SCRIPT="$(readlink -f "$0")"
SCRIPTPATH="$(dirname "${SCRIPT}")"
TESTDIR="${SCRIPTPATH}/../../.github/tests"
#DEPS="${TESTDIR}/dependencies"
# shellcheck source=/dev/null
source "${SCRIPTPATH}/../../.github/scripts/parse-versions.sh"
# shellcheck source=/dev/null
source "${TESTDIR}/common.sh"
CLEANUP=1
for i in "$@"; do
case $i in
-c)
CLEANUP=0
shift # past argument=value
;;
esac
done
if [ "x${GITHUB_JOB}" != "x" ]; then
echo "Running in GitHub"
else
echo "Do not run this script on your own box. For testing, it deploys a testing local spire ha setup using sudo. This is likely not what you want. Only use this script as a reference."
exit 1
fi
teardown() {
echo ---------------------------
docker exec -i chart-testing-worker /bin/bash -c "more /var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/disk-keymanager/keys.json /var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/spire-agent-persistence/agent-data.json | cat"
sudo systemctl status spire-server@a || true
sudo systemctl status spire-server@b || true
sudo spire-server entry show -instance a || true
sudo spire-server entry show -instance b || true
sudo systemctl status spire-controller-manager@a || true
sudo systemctl status spire-controller-manager@b || true
sudo systemctl status spire-agent@a || true
sudo systemctl status spire-agent@b || true
sudo systemctl status spire-trust-sync@a || true
sudo systemctl status spire-trust-sync@b || true
sudo systemctl status spiffe-socat-unix@k8s-spire-server-a || true
sudo systemctl status spiffe-socat-unix@k8s-spire-server-b || true
sudo systemctl status spiffe-socat-unix@k8s-spire-agent-2-a || true
sudo systemctl status spiffe-socat-unix@k8s-spire-agent-2-b || true
sudo systemctl status spiffe-socat-unix@k8s-spire-agent-3-a || true
sudo systemctl status spiffe-socat-unix@k8s-spire-agent-3-b || true
sudo systemctl status spiffe-socat-unix@k8s-spire-agent-4-a || true
sudo systemctl status spiffe-socat-unix@k8s-spire-agent-4-b || true
sudo spire-server bundle list -socketPath /var/run/spire/server/sockets/a/private/api.sock || true
sudo spire-server bundle list -socketPath /var/run/spire/server/sockets/b/private/api.sock || true
kubectl exec -i -n spire-server spire-a-internal-server-0 -- spire-server entry show || true
kubectl exec -i -n spire-server spire-b-internal-server-0 -- spire-server entry show || true
kubectl exec -i -n spire-server spire-a-internal-server-0 -- spire-server agent list -output json | yq e . - -P || true
kubectl exec -i -n spire-server spire-b-internal-server-0 -- spire-server agent list -output json | yq e . - -P || true
print_helm_releases
if [[ "$1" -ne 0 ]]; then
get_namespace_details spire-server spire-system
kubectl describe pod -n spire-system
fi
if [ "${CLEANUP}" -eq 1 ]; then
helm uninstall --namespace spire-mgmt spire-b 2>/dev/null || true
helm uninstall --namespace spire-mgmt spire-a 2>/dev/null || true
helm uninstall --namespace spire-mgmt spire 2>/dev/null || true
kubectl delete ns spire-server 2>/dev/null || true
kubectl delete ns spire-system 2>/dev/null || true
kubectl delete ns spire-mgmt 2>/dev/null || true
fi
}
trap 'EC=$? && trap - SIGTERM && teardown $EC' SIGINT SIGTERM EXIT
wait_for_healthcheck() {
local app="$1"
local socket="$2"
local timeout=30
local count=0
while [ "$count" -lt "$timeout" ]; do
rc=0
sudo "$app" healthcheck -socketPath "$socket" || rc=$?
if [ "$rc" -eq 0 ]; then
return 0
fi
sleep 1
((count++)) || true
done
return 1
}
wait_for_trust_sync() {
local socket="$1"
local timeout=30
local count=0
while [ "$count" -lt "$timeout" ]; do
entries=$(sudo spire-server bundle list -socketPath "$socket" | wc -l)
if [ "$entries" -ne 0 ]; then
return 0
fi
sleep 1
((count++)) || true
done
return 1
}
wait_for_jwt() {
local socket="$1"
local timeout=30
local count=0
while [ "$count" -lt "$timeout" ]; do
rc=0
sudo spire-agent api fetch jwt -audience test -socketPath "$socket" || rc=$?
if [ "$rc" -eq 0 ]; then
return 0
fi
sleep 1
((count++)) || true
done
return 1
}
"${SCRIPTPATH}/../../.github/scripts/prepare-local-chart-deps.sh"
# Get the package repo and install the packages
sudo curl -s -o /etc/apt/sources.list.d/spire-examples.list https://raw.githubusercontent.com/spiffe/spire-examples/refs/heads/main/examples/debs/amd64/spire-examples.list
sudo apt-get update
sudo apt-get install -y spire-common spire-agent spire-server spire-controller-manager spiffe-socat-unix socat spire-trust-sync spiffe-helper
# Set our testing trust domain
sudo sed -i 's/example.org/production.other/' /etc/spiffe/default-trust-domain.env
# register some workloads with the spire server using manifests
sudo mkdir -p /etc/spire/server/a/manifests/ /etc/spire/server/b/manifests/
sudo cp "${SCRIPTPATH}/example-manifests"/* /etc/spire/server/a/manifests/
sudo cp "${SCRIPTPATH}/example-manifests"/* /etc/spire/server/b/manifests/
# For testing, help speed up the sync
sudo rm -f /etc/spire/server/a/manifests/node1-k8s-spire-server.yaml
sudo rm -f /etc/spire/server/b/manifests/node1-k8s-spire-server.yaml
# Since we are running the two root spire servers on the same machine, we need to ensure ports do not conflict for server b
sudo /bin/bash -c 'echo SPIRE_BIND_PORT=8082 > /etc/spire/server/b.env'
sudo /bin/bash -c '(echo METRICS_BIND_ADDRESS="0.0.0.0:9125"; echo HEALTH_PROBE_BIND_ADDRESS="0.0.0.0:9126") > /etc/spire/controller-manager/b.env'
# Startup servers and make sure they are ready
sudo systemctl start spire-server@a spire-server@b spire-controller-manager@a spire-controller-manager@b
wait_for_healthcheck spire-server /run/spire/server/sockets/a/private/api.sock
wait_for_healthcheck spire-server /run/spire/server/sockets/b/private/api.sock
# Configure our agents. For the test, create join tokens for both agents. You should really use a node attestor other then join tokens such as tpm-direct, http_challenge, or a cloud provider one
JOIN_TOKEN_A=$(sudo spire-server token generate -spiffeID spiffe://production.other/agent/node1 -socketPath /run/spire/server/sockets/a/private/api.sock | awk '{print "\""$2"\""}')
JOIN_TOKEN_B=$(sudo spire-server token generate -spiffeID spiffe://production.other/agent/node1 -socketPath /run/spire/server/sockets/b/private/api.sock | awk '{print "\""$2"\""}')
export JOIN_TOKEN_A
export JOIN_TOKEN_B
sudo /bin/bash -c "echo JOIN_TOKEN=${JOIN_TOKEN_A} > /etc/spire/agent/a.env"
sudo /bin/bash -c "echo JOIN_TOKEN=${JOIN_TOKEN_B} > /etc/spire/agent/b.env"
sudo /bin/bash -c "echo SPIRE_SERVER_PORT=8082 >> /etc/spire/agent/b.env"
# Since we are running the two root spire servers on the same machine, we need to configure the trust sync instances to point to the opposite server
sudo /bin/bash -c 'echo "SPIRE_SERVER_SOCKET=/var/run/spire/server/sockets/b/private/api.sock" > /etc/spire/trust-sync/a.conf'
sudo /bin/bash -c 'echo "SPIRE_SERVER_SOCKET=/var/run/spire/server/sockets/a/private/api.sock" > /etc/spire/trust-sync/b.conf'
# Startup the agent
sudo systemctl start spire-agent@a spire-agent@b
sudo systemctl start spire-trust-sync@a spire-trust-sync@b
wait_for_healthcheck spire-agent /var/run/spire/agent/sockets/a/public/api.sock
wait_for_healthcheck spire-agent /var/run/spire/agent/sockets/b/public/api.sock
wait_for_trust_sync /var/run/spire/server/sockets/a/private/api.sock
wait_for_trust_sync /var/run/spire/server/sockets/b/private/api.sock
sudo cp "${SCRIPTPATH}/example-manifests"/node1-k8s-spire-server.yaml /etc/spire/server/a/manifests/
sudo cp "${SCRIPTPATH}/example-manifests"/node1-k8s-spire-server.yaml /etc/spire/server/b/manifests/
# Startup the socat bridge to allow the k8s spire servers to get an identity/trust bundles from the host
sudo systemctl start spiffe-socat-unix@k8s-spire-server-a spiffe-socat-unix@k8s-spire-server-b
wait_for_healthcheck spire-agent /var/run/spiffe/socat/unix/k8s-spire-server-a/public/spire-agent.sock
wait_for_healthcheck spire-agent /var/run/spiffe/socat/unix/k8s-spire-server-b/public/spire-agent.sock
wait_for_jwt /var/run/spiffe/socat/unix/k8s-spire-server-a/public/spire-agent.sock
wait_for_jwt /var/run/spiffe/socat/unix/k8s-spire-server-b/public/spire-agent.sock
# Configure and start up the socat bridges to allow the k8s spire-agents to get an identity/trust bundles from the host.
# We only have one vm mapped to multiple k8s virtual nodes in kind, so we run a pair per k8s virtual node. Normally you would only run one pair per host/vm.
sudo /bin/bash -c "echo SPIFFE_INSTANCE=a > /etc/spiffe/socat/unix/k8s-spire-agent-2-a.conf"
sudo /bin/bash -c "echo SPIFFE_INSTANCE=a > /etc/spiffe/socat/unix/k8s-spire-agent-3-a.conf"
sudo /bin/bash -c "echo SPIFFE_INSTANCE=a > /etc/spiffe/socat/unix/k8s-spire-agent-4-a.conf"
sudo /bin/bash -c "echo SPIFFE_INSTANCE=b > /etc/spiffe/socat/unix/k8s-spire-agent-2-b.conf"
sudo /bin/bash -c "echo SPIFFE_INSTANCE=b > /etc/spiffe/socat/unix/k8s-spire-agent-3-b.conf"
sudo /bin/bash -c "echo SPIFFE_INSTANCE=b > /etc/spiffe/socat/unix/k8s-spire-agent-4-b.conf"
sudo systemctl start spiffe-socat-unix@k8s-spire-agent-2-a spiffe-socat-unix@k8s-spire-agent-2-b
sudo systemctl start spiffe-socat-unix@k8s-spire-agent-3-a spiffe-socat-unix@k8s-spire-agent-3-b
sudo systemctl start spiffe-socat-unix@k8s-spire-agent-4-a spiffe-socat-unix@k8s-spire-agent-4-b
wait_for_healthcheck spire-agent /var/run/spiffe/socat/unix/k8s-spire-agent-2-a/public/api.sock
wait_for_healthcheck spire-agent /var/run/spiffe/socat/unix/k8s-spire-agent-2-b/public/api.sock
wait_for_healthcheck spire-agent /var/run/spiffe/socat/unix/k8s-spire-agent-3-a/public/api.sock
wait_for_healthcheck spire-agent /var/run/spiffe/socat/unix/k8s-spire-agent-3-b/public/api.sock
wait_for_healthcheck spire-agent /var/run/spiffe/socat/unix/k8s-spire-agent-4-a/public/api.sock
wait_for_healthcheck spire-agent /var/run/spiffe/socat/unix/k8s-spire-agent-4-b/public/api.sock
wait_for_jwt /var/run/spiffe/socat/unix/k8s-spire-agent-2-a/public/api.sock
wait_for_jwt /var/run/spiffe/socat/unix/k8s-spire-agent-2-b/public/api.sock
wait_for_jwt /var/run/spiffe/socat/unix/k8s-spire-agent-3-a/public/api.sock
wait_for_jwt /var/run/spiffe/socat/unix/k8s-spire-agent-3-b/public/api.sock
wait_for_jwt /var/run/spiffe/socat/unix/k8s-spire-agent-4-a/public/api.sock
wait_for_jwt /var/run/spiffe/socat/unix/k8s-spire-agent-4-b/public/api.sock
# Deploy an ingress controller
IP=$(kubectl get nodes chart-testing-control-plane -o go-template='{{ range .status.addresses }}{{ if eq .type "InternalIP" }}{{ .address }}{{ end }}{{ end }}')
helm upgrade --install ingress-nginx ingress-nginx --version "$VERSION_INGRESS_NGINX" --repo "$HELM_REPO_INGRESS_NGINX" \
--namespace ingress-nginx \
--create-namespace \
--set "controller.extraArgs.enable-ssl-passthrough=,controller.admissionWebhooks.enabled=false,controller.service.type=ClusterIP,controller.service.externalIPs[0]=$IP" \
--set controller.ingressClassResource.default=true \
--wait
# Test the ingress controller. Should 404 as there is no services yet.
common_test_url "$IP"
# Get the host IP And add spire-server-[ab].${trust_domain} records to it so the spire-servers can talk back to root servers running on the host
HOSTIP=$(ip addr show docker0 | grep 'inet ' | awk '{print $2}' | cut -d/ -f1)
kubectl get configmap -n kube-system coredns -o yaml | grep hosts || kubectl get configmap -n kube-system coredns -o yaml | sed "/ready/a\ hosts {\n fallthrough\n }" | kubectl apply -f -
kubectl get configmap -n kube-system coredns -o yaml | grep production.other || kubectl get configmap -n kube-system coredns -o yaml | sed "/hosts/a\ $HOSTIP spire-server-a.production.other\n $HOSTIP oidc-discovery.production.other\n $HOSTIP spire-server-b.production.other\n" | kubectl apply -f -
kubectl rollout restart -n kube-system deployment/coredns
kubectl rollout status -n kube-system -w --timeout=1m deploy/coredns
# Install the common components
helm upgrade --install --create-namespace --namespace spire-mgmt --values "${COMMON_TEST_YOUR_VALUES},${SCRIPTPATH}/spire-values.yaml" \
spire charts/spire-nested \
--set tags.haAgentCommon=true \
--set "global.spire.namespaces.create=true" \
--set "global.spire.ingressControllerType=ingress-nginx" \
--set "spiffe-oidc-discovery-provider.ingress.enabled=true"
# Install server side a
helm upgrade --install --namespace spire-mgmt --values "${COMMON_TEST_YOUR_VALUES},${SCRIPTPATH}/spire-values.yaml" \
--wait spire-a charts/spire-nested \
--set tags.bottomTurtleHAA=true \
--set "global.spire.ingressControllerType=ingress-nginx"
docker exec -i chart-testing-worker /bin/bash -c "more /var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/disk-keymanager/keys.json /var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/spire-agent-persistence/agent-data.json | cat"
# Rollout just to sped up the tests
kubectl patch deployment spiffe-oidc-discovery-provider -n spire-server --type='strategic' -p '{"spec": {"strategy": {"type": "Recreate", "rollingUpdate": null}}}'
kubectl rollout restart daemonset -n spire-system spire-ha-agent
kubectl rollout status daemonset -n spire-system spire-ha-agent
kubectl rollout restart deployment -n spire-server spiffe-oidc-discovery-provider
kubectl rollout status deployment -n spire-server spiffe-oidc-discovery-provider --timeout=1m
kubectl wait -n spire-server --for=condition=ready pod -l "app.kubernetes.io/name=spiffe-oidc-discovery-provider" --field-selector=status.phase=Running --timeout=90s
curl -k --resolve "oidc-discovery.production.other:443:$IP" "https://oidc-discovery.production.other/.well-known/openid-configuration" -s --fail
# Install server side b
helm upgrade --install --namespace spire-mgmt --values "${COMMON_TEST_YOUR_VALUES},${SCRIPTPATH}/spire-values.yaml" \
--wait spire-b charts/spire-nested \
--set tags.bottomTurtleHAB=true \
--set internal-spire-server-bottom-turtle-ha-b.upstreamAuthority.spire.server.port=8082 \
--set "global.spire.ingressControllerType=ingress-nginx"
docker ps
docker exec -i chart-testing-worker /bin/bash -c "more /var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/disk-keymanager/keys.json /var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/spire-agent-persistence/agent-data.json | cat"
# From here on out, we sanity check that everything is working properly with both servers running.
ENTRIES="$(kubectl exec -i -n spire-server spire-b-internal-server-0 -- spire-server entry show)"
if [[ "${ENTRIES}" == "Found 0 entries" ]]; then
echo "${ENTRIES}"
exit 1
fi
ENTRIES="$(kubectl exec -i -n spire-server spire-a-internal-server-0 -- spire-server entry show)"
if [[ "${ENTRIES}" == "Found 0 entries" ]]; then
echo "${ENTRIES}"
exit 1
fi
kubectl get pods -A -o wide
helm test --namespace spire-mgmt spire-a
helm test --namespace spire-mgmt spire-b
curl -k --resolve "oidc-discovery.production.other:443:$IP" "https://oidc-discovery.production.other/.well-known/openid-configuration" -s --fail
#Test out running only on side b since we know already only both servers work together, and that only side a works if we made it this far.
helm delete -n spire-mgmt spire-a
kubectl rollout restart daemonset -n spire-system spire-ha-agent
kubectl rollout status daemonset -n spire-system spire-ha-agent
kubectl rollout restart deployment -n spire-server spiffe-oidc-discovery-provider
kubectl rollout status deployment -n spire-server spiffe-oidc-discovery-provider --timeout=1m
curl -k --resolve "oidc-discovery.production.other:443:$IP" "https://oidc-discovery.production.other/.well-known/openid-configuration" -s --fail
@@ -0,0 +1,13 @@
global:
spire:
recommendations:
enabled: true
namespaces:
create: false
#ingressControllerType: ""
#clusterName: example-cluster
#trustDomain: example.org
#caSubject:
# country: ""
# organization: ""
# commonName: ""
@@ -123,16 +123,13 @@ echo "${IP} spire-server.production.other spiffe-step-ssh.production.other spiff
echo Hosts:
cat /etc/hosts
curl -L https://raw.githubusercontent.com/kfox1111/spire-examples/refs/heads/spiffe-step-ssh/examples/spiffe-step-ssh/scripts/demo.sh | sudo bash
# Get the package repo and install the packages
curl -o /tmp/stepcli.deb https://dl.smallstep.com/gh-release/cli/gh-release-header/v0.30.2/step-cli_0.30.2-1_amd64.deb -L
sudo dpkg -i /tmp/stepcli.deb
sudo curl -s -o /etc/apt/sources.list.d/spire-examples.list https://raw.githubusercontent.com/spiffe/spire-examples/refs/heads/main/examples/debs/amd64/spire-examples.list
sudo apt-get update
sudo apt-get install -y spire-common spire-agent spiffe-step-ssh spiffe-helper
sudo mkdir -p /usr/libexec/spiffe-step-ssh
sudo mkdir -p /etc/systemd/system/sshd.service.d
sudo curl -L -o /usr/libexec/spiffe-step-ssh/update.sh https://raw.githubusercontent.com/kfox1111/spire-examples/refs/heads/spiffe-step-ssh/examples/spiffe-step-ssh/scripts/update.sh
sudo curl -L -o /etc/systemd/system/[email protected] https://raw.githubusercontent.com/kfox1111/spire-examples/refs/heads/spiffe-step-ssh/examples/spiffe-step-ssh/systemd/[email protected]
sudo curl -L -o /etc/systemd/system/spiffe-step-ssh-cleanup.service https://raw.githubusercontent.com/kfox1111/spire-examples/refs/heads/spiffe-step-ssh/examples/spiffe-step-ssh/systemd/spiffe-step-ssh-cleanup.service
sudo curl -L -o /etc/systemd/system/sshd.service.d/10-spiffe-step-ssh.conf https://raw.githubusercontent.com/kfox1111/spire-examples/refs/heads/spiffe-step-ssh/examples/spiffe-step-ssh/conf/10-spiffe-step-ssh.conf
sudo mkdir -p /etc/spire/agent
sudo cp "${SCRIPTPATH}/spire-agent.conf" /etc/spire/agent/main.conf
PASSWORD=$(openssl rand -base64 48)