spiffe · kfox1111 · Dec 29, 2023 · Dec 30, 2023 · Dec 31, 2023 · Dec 31, 2023
@@ -12,7 +12,6 @@ aliases:
 
 By default no SPIRE services are exposed outside the Kubernetes cluster. The below sections cover how to expose them.
 
-
 # Exposable Services
 
 ## Production Services

@@ -8,7 +8,6 @@ aliases:
     - /docs/latest/helm-charts-hardened/namespaces
 ---
 
-
 ## Namespace Creation Options
 
 | Value                                 | Default Value | Description                                                                                  |

@@ -0,0 +1,172 @@
+---
+title: Nested SPIRE
+short: Nested SPIRE
+description: Nested SPIRE Architectures
+kind: spire-helm-charts-hardened-advanced
+weight: 100
+aliases:
+    - /docs/latest/helm-charts-hardened-advanced/nested-spire
+---
+
+## Nested Considerations
+
+### Architectures
+
+The charts can be used to deploy many different styles of Nested SPIRE. A few possibilities are explained below.
+
+### SPIRE Controller Manager
+
+When multiple charts are installed at the same time with it enabled, they must use different classes. This is setup by default. Do not override without understanding the situation.
+
+### TTLs
+
+
+*fixme* note here about tradeoffs between longer ca's more stable less risky for networking. longer ca time more risk of security issues.
+
+
+The TTL of the workload certificates is limited by the root instances `spire-server.caTTL` and the TTL of the intermediate CA's it produces, default `spire-server.controllerManager.identity.default.ttl`
+
+The root CA will generate a new root at about 1/2 the `spire-server.caTTL`.
+
+# Kubernetes Integrated Root
+
+If your thinking about using nesting in the future, its easiest to start with a nested root deployment rather then a standalone instance.
+
+We start with deploying the SPIRE instance that includes a root server.
+
+## Setup Root Instance
+
+![Image](/img/spire-helm-charts-hardened/root-k8s.png)
+
+
+### Install the CRDs.
+```shell
+helm upgrade --install --create-namespace -n spire-mgmt spire-crds spire-crds \
+ --repo https://spiffe.github.io/helm-charts-hardened/
+```
+
+Write out your-values.yaml as described in the [Install](../../spire-helm-charts-hardened-about/installation/#production-deployment) instructions steps 1 through 3.
+
+Create a file named root-values.yaml
+
+### No child clusters/VMs
+If you do not have a need for any child clusters or VMs, you can turn off the external SPIRE server instance by adding the following to root-values.yaml:
+```
+tags:
+  nestedRoot: true
+
+external-spire-server:
+  enabled: false
+```
+
+Install the root server:
+
+```shell
+helm upgrade --install -n spire-mgmt spire spire-nested --repo https://spiffe.github.io/helm-charts-hardened/ \
+  -f your-values.yaml -f root-values.yaml
+```
+
+### Child clusters/VMs
+If you do want to have child clusters or VMs, it should be exposed outside the cluster. Ingress is the most common/easy way to do so. Add the following to root-values.yaml:
+```
+tags:
+  nestedRoot: true
+
+spiffe-oidc-discovery-provider:
+  ingress:
+    enabled: true
+
+external-spire-server:
+  ingress:
+    enabled: true
+```
+
+Also, ensure spire-server.$trustdomain is setup in your dns environment to point at your ingress controller, or update the ingress related [settings](../../spire-helm-charts-hardened-about/exposing) 
+
+For each child cluster, run the following on a control plane node and copy the generated content to a file named `<child cluster name>.kubeconfig` where you are installing the root server:
+```
+kubeadm kubeconfig user --client-name=spire-root | tr '\n' ' ' | sed 's/ //g'; echo
+```
+
+Install the root server:
+
+```shell
+helm upgrade --install -n spire-mgmt spire spire-nested --repo https://spiffe.github.io/helm-charts-hardened/ \
+  # Use as many of these lines as you have child clusters. Substitute <child cluster name> for its short name:
+  --set "external-spire-server.kubeConfigs.<child cluster name>.kubeConfigBase64=$(cat <child cluster name>.kubeconfig)" \
+  -f your-values.yaml -f root-values.yaml
+```
+
+## Multi-Cluster
+
+![Image](/img/spire-helm-charts-hardened/multicluster-alternate3.png)
+
+Deploy the root server as described above.
+
+Write out a configuration file named child-values.yaml
+```
+tags:
+  nestedChildFull: true
+
+global:
+  spire:
+    #Update these two values
+    clusterName: changeme
+    upstreamSpireAddress: spire-server.changeme
+```
+
+Make sure you update the two values mentioned in the file. Each cluster should have a unique clusterName, and the upstreamSpireAddress should match the dns entry you set up for the root server.
+
+Install the child server onto the child cluster:
+
+```shell
+helm upgrade --install --create-namespace -n spire-mgmt spire-crds spire-crds \
+ --repo https://spiffe.github.io/helm-charts-hardened/
+helm upgrade --install -n spire-mgmt spire spire-nested --repo https://spiffe.github.io/helm-charts-hardened/ \
+  -f your-values.yaml -f child-values.yaml
+```
+
+> **Note**
+> The child cluster will fail to start some services at this point, as the root server doesn't have have a trust established yet. This is expected.
+
+Next, we will establish the trust between instances.
+
+Example: TODO
+
+## Security Cluster
+
+![Image](/img/spire-helm-charts-hardened/securitycluster.png)
+
+In some cases, you may have a seperate Kubernetes Cluster just for security related services that sits along side one or more workload Kubernetes Clusters. The clusters share the same Datacenter, Availability Zone, Region or whatthever other term that is used to denote the same locality.
+
+Deploy the root server as described above
+
+Write out a configuration file named child-values.yaml
+
+```
+global:
+  spire:
+    # Update this value
+    clusterName: changeme
+
+tags:
+  nestedChildSecurity: true
+
+downstream-spire-agent-security:
+  serviceAccount:
+    server:
+      # Update this value
+      address: spire-server.changeme
+```
+
+Install the child server onto the child cluster:
+
+```shell
+helm upgrade --install --create-namespace -n spire-mgmt spire-crds spire-crds \
+ --repo https://spiffe.github.io/helm-charts-hardened/
+helm upgrade --install -n spire-mgmt spire spire-nested --repo https://spiffe.github.io/helm-charts-hardened/ \
+  -f your-values.yaml -f child-values.yaml
+```
+
+> **Note**
+> The child cluster will fail to start some services at this point, as the root server doesn't have have a trust established yet. This is expected.
@@ -0,0 +1,23 @@
+all: multicluster-alternate.png multicluster-alternate2.png multicluster-alternate3.png multicluster.png securitycluster.png singlehardened.png root-k8s.png
+
+root-k8s.png: root-k8s.dot
+	dot -Tpng root-k8s.dot -o root-k8s.png
+
+multicluster-alternate.png: multicluster-alternate.dot
+	dot -Tpng multicluster-alternate.dot -o multicluster-alternate.png
+
+multicluster-alternate2.png: multicluster-alternate2.dot
+	dot -Tpng multicluster-alternate2.dot -o multicluster-alternate2.png
+
+multicluster-alternate3.png: multicluster-alternate3.dot
+	dot -Tpng multicluster-alternate3.dot -o multicluster-alternate3.png
+
+multicluster.png: multicluster.dot
+	dot -Tpng multicluster.dot -o multicluster.png
+
+securitycluster.png: securitycluster.dot
+	dot -Tpng securitycluster.dot -o securitycluster.png
+
+singlehardened.png: singlehardened.dot
+	dot -Tpng singlehardened.dot -o singlehardened.png
+
@@ -0,0 +1,128 @@
+digraph G {
+  compound=true;
+  subgraph cluster_root {
+    label="Cluster: Root K8S";
+    style="filled,solid,bold";
+    color="#b3b3b3";
+    fillcolor="#f5f5f5";
+    subgraph cluster_root_release {
+      label="Helm Release: Namespace=spire-mgmt Name=spire"
+      style="filled,dashed,bold";
+      color="#a3a3a3";
+      fillcolor="#e5e5e5";
+      subgraph cluster_root_ns_server {
+        style="filled,dashed,bold";
+        color="#939393";
+        fillcolor="#d5d5d5";
+        label="Namespace: spire-server"
+        subgraph cluster_ns_root_server_obj {
+          style="filled,rounded,bold";
+          color="#6c8ebf";
+          fillcolor="#dae8fc";
+          label="Root SPIRE Server"
+          spireRoot [label="K̶8̶s̶ ̶C̶o̶n̶t̶r̶o̶l̶l̶e̶r̶ ̶M̶a̶n̶a̶g̶e̶r̶",shape="record",style="rounded,solid,filled,bold",fontcolor="#b85450",color="#b85450",fillcolor="#f8cecc"];
+        }
+      }
+    }
+  }
+  subgraph cluster_nested1 {
+    label="Cluster: K8S Workload 1";
+    style="filled,solid,bold";
+    color="#b3b3b3";
+    fillcolor="#f5f5f5";
+    subgraph cluster_nested1_release {
+      label="Helm Release: Namespace=spire-mgmt Name=spire-root"
+      style="filled,dashed,bold";
+      color="#a3a3a3";
+      fillcolor="#e5e5e5";
+      subgraph cluster_nested1_ns1 {
+        style="filled,dashed,bold";
+        color="#939393";
+        fillcolor="#d5d5d5";
+        label="Namespace: spire-system"
+        spireUpstreamAgent1 [label="Upstream Spire Agent/CSI",shape="box",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"];
+      }
+      subgraph cluster_nested1_ns2 {
+        style="filled,dashed,bold";
+        color="#939393";
+        fillcolor="#d5d5d5";
+        label="Namespace: spire-server"
+        subgraph cluster_ns_nested1_server_obj {
+          style="filled,rounded,bold";
+          color="#6c8ebf";
+          fillcolor="#dae8fc";
+          label="Nested SPIRE Server"
+          spireServerNested1 [label="K8s Controller Manager",shape="record",style="rounded,solid,filled,bold",color="#10739e",fillcolor="#b1ddf0"];
+        }
+      }
+      subgraph cluster_nested1_ns3 {
+        style="filled,dashed,bold";
+        color="#939393";
+        fillcolor="#d5d5d5";
+        label="Namespace: spire-system"
+        spireDownstreamAgent1 [label="Downstream Spire Agent/CSI",shape="box",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"];
+      }
+    }
+    subgraph cluster_nested1_user {
+      style="filled,dashed,bold";
+      color="#939393";
+      fillcolor="#d5d5d5";
+      label="Namespace: user"
+      userWorkload1 [label="User Workload",shape="box",style="rounded,solid,filled,bold",color="#d6b656",fillcolor="#fff2cc"];
+    }
+  }
+  subgraph cluster_nested2 {
+    label="Cluster: K8S Workload 2";
+    style="filled,solid,bold";
+    color="#b3b3b3";
+    fillcolor="#f5f5f5";
+    subgraph cluster_nested2_release {
+      label="Helm Release: Namespace=spire-mgmt Name=spire"
+      style="filled,dashed,bold";
+      color="#a3a3a3";
+      fillcolor="#e5e5e5";
+      subgraph cluster_nested2_ns1 {
+        style="filled,dashed,bold";
+        color="#939393";
+        fillcolor="#d5d5d5";
+        label="Namespace: spire-system"
+        spireUpstreamAgent2 [label="Upstream Spire Agent/CSI",shape="box",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"];
+      }
+      subgraph cluster_nested2_ns2 {
+        style="filled,dashed,bold";
+        color="#939393";
+        fillcolor="#d5d5d5";
+        label="Namespace: spire-server"
+        subgraph cluster_ns_nested2_server_obj {
+          style="filled,rounded,bold";
+          color="#6c8ebf";
+          fillcolor="#dae8fc";
+          label="Nested SPIRE Server"
+          spireServerNested2 [label="K8s Controller Manager",shape="record",style="rounded,solid,filled,bold",color="#10739e",fillcolor="#b1ddf0"];
+        }
+      }
+      subgraph cluster_nested2_ns3 {
+        style="filled,dashed,bold";
+        color="#939393";
+        fillcolor="#d5d5d5";
+        label="Namespace: spire-system"
+        spireDownstreamAgent2 [label="Downstream Spire Agent/CSI",shape="box",style="rounded,solid,filled,bold",color="#82b366",fillcolor="#d5e8d4"];
+      }
+    }
+    subgraph cluster_nested2_user {
+      style="filled,dashed,bold";
+      color="#939393";
+      fillcolor="#d5d5d5";
+      label="Namespace: user"
+      userWorkload2 [label="Other User Workload",shape="box",style="rounded,solid,filled,bold",color="#d6b656",fillcolor="#fff2cc"];
+    }
+  }
+  spireRoot -> spireUpstreamAgent1 [ltail=cluster_ns_root_server_obj];
+  spireRoot -> spireUpstreamAgent2 [ltail=cluster_ns_root_server_obj];
+  spireUpstreamAgent1 -> spireServerNested1 [lhead=cluster_ns_nested1_server_obj];
+  spireServerNested1 -> spireDownstreamAgent1 [ltail=cluster_ns_nested1_server_obj];
+  spireDownstreamAgent1 -> userWorkload1;
+  spireUpstreamAgent2 -> spireServerNested2 [lhead=cluster_ns_nested2_server_obj];
+  spireServerNested2 -> spireDownstreamAgent2 [ltail=cluster_ns_nested2_server_obj];
+  spireDownstreamAgent2 -> userWorkload2;
+}