feat: Add vault method for pulling privateKey (#37)

* feat: Add option for Vault for pulling private key - Up CRD for Vault option over K8s secrets for github app private keys - Up readme - Up manager deployment with env vars for Vault - Add rbac permissions for Token Request API
samirtahir91 · Apr 11, 2024 · bd7fcd1 · bd7fcd1
1 parent 582498b
commit bd7fcd1
Show file tree

Hide file tree

Showing 13 changed files with 466 additions and 27 deletions.
diff --git a/Makefile b/Makefile
@@ -62,7 +62,7 @@ vet: ## Run go vet against code.
 
 .PHONY: test
 test: manifests generate fmt vet envtest ## Run tests.
-	KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) --bin-dir $(LOCALBIN) -p path)" go test $$(go list ./... | grep -v -E '/e2e|v1|utils|cmd|test_helpers') -coverprofile cover.out
+	KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) --bin-dir $(LOCALBIN) -p path)" go test $$(go list ./... | grep -v -E '/e2e|v1|utils|cmd|test_helpers|vault') -coverprofile cover.out
 
 # Utilize Kind or modify the e2e tests to load the image locally, enabling compatibility with other vendors.
 .PHONY: test-e2e  # Run the e2e tests against a Kind k8s instance that is spun up.

diff --git a/README.md b/README.md
@@ -12,10 +12,27 @@ It will reconcile a new access token before expiry (1hr).
 ## Description
 Key features:
 - Uses a custom resource `GithubApp` in your destination namespace.
-- Reads `appId`, `installId` and `privateKeySecret` defined in a `GithubApp` resource and requests an access token from Github for the Github App.
+- Reads `appId`, `installId` and either and `privateKeySecret` or `vaultPrivateKey` defined in a `GithubApp` resource and requests an access token from Github for the Github App.
   - It stores the access token in a secret `github-app-access-token-{appId}`
-- The `privateKeySecret` refers to an existing secret in the namespace which holds the base64 encoded PEM of the Github App's private key.
-  - It expects the field `data.privateKey` in the secret to pull the private key from.
+- For pulling a GitHub Apps private key, there are 2 options built-in:
+  - Using a Kubernetes secret:
+    - Use `privateKeySecret` - refers to an existing secret in the namespace which holds the base64 encoded PEM of the Github App's private key.
+    - It expects the field `data.privateKey` in the secret to pull the private key from.
+  - Hashicorp Vault (this takes priority over `privateKeySecret` if both are specified):
+    - **You must base64 encode your private key before saving in Vault**
+      - The operator logic will only accept base64 encoded secrets else it will fail.
+    - This will create a short-lived JWT (10mins TTL) via Kubernetes Token Request API, with an audience you define.
+    - It will then use the JWT and Vault role you define to authenticate with Vault and pull the secret containing the private key.
+    - Configure with the `vaultPrivateKey` block:
+      - `spec.vaultPrivateKey.mountPath` - Secret mount path, i.e "secret"
+      - `spec.vaultPrivateKey.secretPath` - Secret path i.e. "githubapps/{App ID}"
+      - `spec.vaultPrivateKey.secretKey` - Secret key i.e. "privateKey"
+    - Configure Kubernetes auth with Vault
+    - Define a role and optionally audience, service account, namespace etc bound to the role
+    - Configure the environment variables in the controller deployment spec:
+      - `VAULT_ROLE` - The role you have bound for Kubernetes auth for the operator
+      - `VAULT_ROLE_AUDIENCE` - The audience you have bound in Vault
+      - `VAULT_ADDRESS` - FQDN or your Vault server, i.e. `http://vault.default:8200`
 - Deleting the `GithubApp` object will also delete the access token secret it owns.
 - The operator will reconcile an access token for a `GithubApp` when:
     - Modifications are made to the access token secret that is owned by a `GithubApp`.
@@ -99,6 +116,25 @@ spec:
 EOF
 ```
 
+## Example GithubApp object using Vault to pull the private key during run-time
+- Below example will request a new JWT from Kubernetes and use it to fetch the private key from Vault when the github access token expires
+```sh
+kubectl apply -f - <<EOF
+apiVersion: githubapp.samir.io/v1
+kind: GithubApp
+metadata:
+  name: GithubApp-sample
+  namespace: team-1
+spec:
+  appId: 123123
+  installId: 12312312
+  vaultPrivateKey:
+    mountPath: secret
+    secretPath: githubapp/123123
+    secretKey: privateKey
+EOF
+```
+
 ## Getting Started
 
 ### Prerequisites

diff --git a/api/v1/githubapp_types.go b/api/v1/githubapp_types.go
@@ -24,8 +24,9 @@ import (
 type GithubAppSpec struct {
 	AppId             int                    `json:"appId"`
 	InstallId         int                    `json:"installId"`
-	PrivateKeySecret  string                 `json:"privateKeySecret"`
+	PrivateKeySecret  string                 `json:"privateKeySecret,omitempty"`
 	RolloutDeployment *RolloutDeploymentSpec `json:"rolloutDeployment,omitempty"`
+	VaultPrivateKey   *VaultPrivateKeySpec   `json:"vaultPrivateKey,omitempty"`
 }
 
 // GithubAppStatus defines the observed state of GithubApp
@@ -57,6 +58,13 @@ type RolloutDeploymentSpec struct {
 	Labels map[string]string `json:"labels,omitempty"`
 }
 
+// VaultPrivateKeySpec defines the spec for retrieving the private key from Vault
+type VaultPrivateKeySpec struct {
+	MountPath  string `json:"mountPath"`
+	SecretPath string `json:"secretPath"`
+	SecretKey  string `json:"secretKey"`
+}
+
 //+kubebuilder:object:root=true
 
 // GithubAppList contains a list of GithubApp

diff --git a/api/v1/zz_generated.deepcopy.go b/api/v1/zz_generated.deepcopy.go
diff --git a/cmd/main.go b/cmd/main.go
@@ -70,8 +70,8 @@ func main() {
 	// Read DEBUG_LOG from env var
 	debugLog, logVarErr := strconv.ParseBool(os.Getenv("DEBUG_LOG"))
 	if logVarErr != nil {
-	    // Default to false
-	    debugLog = false
+		// Default to false
+		debugLog = false
 	}
 	opts := zap.Options{
 		Development: debugLog,

diff --git a/config/crd/bases/githubapp.samir.io_githubapps.yaml b/config/crd/bases/githubapp.samir.io_githubapps.yaml
@@ -67,10 +67,24 @@ spec:
                       type: string
                     type: object
                 type: object
+              vaultPrivateKey:
+                description: VaultPrivateKeySpec defines the spec for retrieving the
+                  private key from Vault
+                properties:
+                  mountPath:
+                    type: string
+                  secretKey:
+                    type: string
+                  secretPath:
+                    type: string
+                required:
+                - mountPath
+                - secretKey
+                - secretPath
+                type: object
             required:
             - appId
             - installId
-            - privateKeySecret
             type: object
           status:
             description: GithubAppStatus defines the observed state of GithubApp

diff --git a/config/manager/manager.yaml b/config/manager/manager.yaml
@@ -106,5 +106,11 @@ spec:
             value: "15m"
           - name: DEBUG_LOG
             value: "false"
+          - name: VAULT_ROLE
+            value: githubapp
+          - name: VAULT_ROLE_AUDIENCE
+            value: githubapp
+          - name: VAULT_ADDRESS
+            value: "http://vault.default:8200"
       serviceAccountName: controller-manager
       terminationGracePeriodSeconds: 10
diff --git a/config/rbac/role.yaml b/config/rbac/role.yaml
@@ -33,6 +33,20 @@ rules:
   verbs:
   - create
   - patch
+- apiGroups:
+  - ""
+  resources:
+  - serviceaccounts
+  verbs:
+  - create
+  - get
+- apiGroups:
+  - ""
+  resources:
+  - serviceaccounts/token
+  verbs:
+  - create
+  - get
 - apiGroups:
   - githubapp.samir.io
   resources:

diff --git a/go.mod b/go.mod
@@ -4,21 +4,26 @@ go 1.21
 
 require (
 	github.com/golang-jwt/jwt/v4 v4.5.0
+	github.com/hashicorp/vault/api v1.12.2
+	github.com/hashicorp/vault/api/auth/kubernetes v0.6.0
 	github.com/onsi/ginkgo/v2 v2.14.0
 	github.com/onsi/gomega v1.30.0
 	k8s.io/api v0.29.0
 	k8s.io/apimachinery v0.29.0
 	k8s.io/client-go v0.29.0
+	k8s.io/utils v0.0.0-20230726121419-3b25d923346b
 	sigs.k8s.io/controller-runtime v0.17.0
 )
 
 require (
 	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/cenkalti/backoff/v3 v3.0.0 // indirect
 	github.com/cespare/xxhash/v2 v2.2.0 // indirect
 	github.com/davecgh/go-spew v1.1.1 // indirect
 	github.com/emicklei/go-restful/v3 v3.11.0 // indirect
 	github.com/evanphx/json-patch/v5 v5.8.0 // indirect
 	github.com/fsnotify/fsnotify v1.7.0 // indirect
+	github.com/go-jose/go-jose/v3 v3.0.3 // indirect
 	github.com/go-logr/logr v1.4.1 // indirect
 	github.com/go-logr/zapr v1.3.0 // indirect
 	github.com/go-openapi/jsonpointer v0.19.6 // indirect
@@ -33,11 +38,22 @@ require (
 	github.com/google/gofuzz v1.2.0 // indirect
 	github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 // indirect
 	github.com/google/uuid v1.3.0 // indirect
+	github.com/hashicorp/errwrap v1.1.0 // indirect
+	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
+	github.com/hashicorp/go-multierror v1.1.1 // indirect
+	github.com/hashicorp/go-retryablehttp v0.6.6 // indirect
+	github.com/hashicorp/go-rootcerts v1.0.2 // indirect
+	github.com/hashicorp/go-secure-stdlib/parseutil v0.1.6 // indirect
+	github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 // indirect
+	github.com/hashicorp/go-sockaddr v1.0.2 // indirect
+	github.com/hashicorp/hcl v1.0.0 // indirect
 	github.com/imdario/mergo v0.3.6 // indirect
 	github.com/josharian/intern v1.0.0 // indirect
 	github.com/json-iterator/go v1.1.12 // indirect
 	github.com/mailru/easyjson v0.7.7 // indirect
 	github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect
+	github.com/mitchellh/go-homedir v1.1.0 // indirect
+	github.com/mitchellh/mapstructure v1.5.0 // indirect
 	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
 	github.com/modern-go/reflect2 v1.0.2 // indirect
 	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
@@ -46,14 +62,16 @@ require (
 	github.com/prometheus/client_model v0.5.0 // indirect
 	github.com/prometheus/common v0.45.0 // indirect
 	github.com/prometheus/procfs v0.12.0 // indirect
+	github.com/ryanuber/go-glob v1.0.0 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	go.uber.org/multierr v1.11.0 // indirect
 	go.uber.org/zap v1.26.0 // indirect
+	golang.org/x/crypto v0.19.0 // indirect
 	golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e // indirect
 	golang.org/x/net v0.19.0 // indirect
 	golang.org/x/oauth2 v0.12.0 // indirect
-	golang.org/x/sys v0.16.0 // indirect
-	golang.org/x/term v0.15.0 // indirect
+	golang.org/x/sys v0.17.0 // indirect
+	golang.org/x/term v0.17.0 // indirect
 	golang.org/x/text v0.14.0 // indirect
 	golang.org/x/time v0.3.0 // indirect
 	golang.org/x/tools v0.16.1 // indirect
@@ -67,7 +85,6 @@ require (
 	k8s.io/component-base v0.29.0 // indirect
 	k8s.io/klog/v2 v2.110.1 // indirect
 	k8s.io/kube-openapi v0.0.0-20231010175941-2dd684a91f00 // indirect
-	k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect
 	sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
 	sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
 	sigs.k8s.io/yaml v1.4.0 // indirect