From e676c442e54e5ab5cdfa4a2b4c533b755c67fc27 Mon Sep 17 00:00:00 2001 From: dongjiang1989 Date: Mon, 2 Sep 2024 16:19:55 +0800 Subject: [PATCH 1/4] add volume mountpath env expansion Signed-off-by: dongjiang1989 --- .../README.md | 693 ++++++++++++++++++ .../kep.yaml | 39 + 2 files changed, 732 insertions(+) create mode 100644 keps/sig-node/4788-volume-mountpath-env-expansion/README.md create mode 100644 keps/sig-node/4788-volume-mountpath-env-expansion/kep.yaml diff --git a/keps/sig-node/4788-volume-mountpath-env-expansion/README.md b/keps/sig-node/4788-volume-mountpath-env-expansion/README.md new file mode 100644 index 00000000000..9e6237f972b --- /dev/null +++ b/keps/sig-node/4788-volume-mountpath-env-expansion/README.md @@ -0,0 +1,693 @@ +# KEP-478: Support volume mountPath replacing variable references with the values of given EnvVar + + +- [Release Signoff Checklist](#release-signoff-checklist) +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) + - [User Stories (Optional)](#user-stories-optional) + - [Story 1](#story-1) + - [Story 2](#story-2) + - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional) + - [Risks and Mitigations](#risks-and-mitigations) +- [Design Details](#design-details) + - [Test Plan](#test-plan) + - [Prerequisite testing updates](#prerequisite-testing-updates) + - [Unit tests](#unit-tests) + - [Integration tests](#integration-tests) + - [e2e tests](#e2e-tests) + - [Graduation Criteria](#graduation-criteria) + - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) + - [Version Skew Strategy](#version-skew-strategy) +- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire) + - [Feature Enablement and Rollback](#feature-enablement-and-rollback) + - [Rollout, Upgrade and Rollback Planning](#rollout-upgrade-and-rollback-planning) + - [Monitoring Requirements](#monitoring-requirements) + - [Dependencies](#dependencies) + - [Scalability](#scalability) + - [Troubleshooting](#troubleshooting) +- [Implementation History](#implementation-history) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) +- [Infrastructure Needed (Optional)](#infrastructure-needed-optional) + + +## Release Signoff Checklist + + + +Items marked with (R) are required *prior to targeting to a milestone / release*. + +- [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR) +- [ ] (R) KEP approvers have approved the KEP status as `implementable` +- [ ] (R) Design details are appropriately documented +- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors) + - [ ] e2e Tests for all Beta API Operations (endpoints) + - [ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) + - [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free +- [ ] (R) Graduation criteria is in place + - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) +- [ ] (R) Production readiness review completed +- [ ] (R) Production readiness review approved +- [ ] "Implementation History" section is up-to-date for milestone +- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io] +- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes + + + +[kubernetes.io]: https://kubernetes.io/ +[kubernetes/enhancements]: https://git.k8s.io/enhancements +[kubernetes/kubernetes]: https://git.k8s.io/kubernetes +[kubernetes/website]: https://git.k8s.io/website + +## Summary + +The volume `mountPath` expands the mountPath of the given `VolumeMount` by replacing variable references with the values of given EnvVar. [ExpandContainerVolumeMounts](https://github.com/kubernetes/kubernetes/pull/49388) expands the subpath of the given EnvVar, but not support mountPath field. + +This KEP aims to for better performance and simplicity. + +## Motivation + + + +### Goals + + + +### Non-Goals + + + +## Proposal + + + +### User Stories (Optional) + + + +#### Story 1 + +I used a configMap to store the mount path value, but when i call in my pod it doesn't work it seems that the mountPath property can not be assigned to an env variable. + +``` +spec: + containers: + - env: + - name: POD_NAME + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.name + - name: NAMESPACE + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace + - name: LOG_MOUNT_PATH + valueFrom: + configMapKeyRef: + name: my-configMap + key: LOG_MOUNT_PATH + image: myapp:latest + name: myapp + args: + - --log_file=$(LOG_MOUNT_PATH)/myapp.log + volumeMounts: + - mountPath: $(LOG_MOUNT_PATH) #support mountPath env + name: datalog + subPathExpr: $(NAMESPACE)/$(POD_NAME) + volumes: + - hostPath: + path: /data/log + type: "" + name: datalog + +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: my-configMap +data: + LOG_MOUNT_PATH: "/opt/somepath" + +``` + +#### Story 2 + +### Notes/Constraints/Caveats (Optional) + + + +### Risks and Mitigations + + + +## Design Details + + + +### Test Plan + + + +[x] I/we understand the owners of the involved components may require updates to +existing tests to make this code solid enough prior to committing the changes necessary +to implement this enhancement. + +##### Prerequisite testing updates + + + +##### Unit tests + + + + + +- `pkg/kubelet/container: `` - `` + +##### Integration tests + + + + + +- go test -coverprofile pkg/kubelet/container/helpers_test.go container + +##### e2e tests + + + +- test/e2e_node/summary_test.go + +### Graduation Criteria + +#### Alpha + +- Feature implemented behind the feature gate `VolumeMountPathEnvExpansion` +- Initial e2e tests completed and enabled + +#### Beta + +- `VolumeMountPathEnvExpansion` feature gate is enabled by default +- Gather feedback and ensure no breaking changes + +#### GA + +- Feature gate removed + +### Upgrade / Downgrade Strategy + + + +### Version Skew Strategy + + + +## Production Readiness Review Questionnaire + + + +### Feature Enablement and Rollback + + + +###### How can this feature be enabled / disabled in a live cluster? + + + +- [x] Feature gate (also fill in values in `kep.yaml`) + - Feature gate name: `VolumeMountPathEnvExpansion` + - Components depending on the feature gate: `Kubelet` + +###### Does enabling the feature change any default behavior? + + + +###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)? + +Feature gates are typically disabled by setting the flag to `false` and +restarting the component. No other changes should be necessary to disable the +feature. + +###### What happens if we reenable the feature if it was previously rolled back? + +There should be no problems + +###### Are there any tests for feature enablement/disablement? + +- The existing summary subpath/subpathExpr test +- The nested expandable envs tests + +### Rollout, Upgrade and Rollback Planning + + +`VolumeMountPathEnvExpansion` feature gate disabled. + +###### How can a rollout or rollback fail? Can it impact already running workloads? + + + +###### What specific metrics should inform a rollback? + + + +###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested? + + + +###### Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.? + + + +### Monitoring Requirements + + + +###### How can an operator determine if the feature is in use by workloads? + + + +###### How can someone using this feature know that it is working for their instance? + + + +- [ ] Events + - Event Reason: +- [ ] API .status + - Condition name: + - Other field: +- [ ] Other (treat as last resort) + - Details: + +###### What are the reasonable SLOs (Service Level Objectives) for the enhancement? + + + +###### What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service? + + + +- [ ] Metrics + - Metric name: + - [Optional] Aggregation method: + - Components exposing the metric: +- [ ] Other (treat as last resort) + - Details: + +###### Are there any missing metrics that would be useful to have to improve observability of this feature? + + + +### Dependencies + + + +###### Does this feature depend on any specific services running in the cluster? + + + +### Scalability + + + +###### Will enabling / using this feature result in any new API calls? + + + +###### Will enabling / using this feature result in introducing new API types? + + + +###### Will enabling / using this feature result in any new calls to the cloud provider? + + + +###### Will enabling / using this feature result in increasing size or count of the existing API objects? + + + +###### Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs? + + + +###### Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components? + + + +###### Can enabling / using this feature result in resource exhaustion of some node resources (PIDs, sockets, inodes, etc.)? + + + +### Troubleshooting + + + +###### How does this feature react if the API server and/or etcd is unavailable? + +###### What are other known failure modes? + + + +###### What steps should be taken if SLOs are not being met to determine the problem? + +## Implementation History + + + +## Drawbacks + + + +## Alternatives + + + +## Infrastructure Needed (Optional) + + diff --git a/keps/sig-node/4788-volume-mountpath-env-expansion/kep.yaml b/keps/sig-node/4788-volume-mountpath-env-expansion/kep.yaml new file mode 100644 index 00000000000..27acd4659a8 --- /dev/null +++ b/keps/sig-node/4788-volume-mountpath-env-expansion/kep.yaml @@ -0,0 +1,39 @@ +title: Support mountPath replacing variable references with the values of given EnvVar +kep-number: 4788 +authors: + - "@dongjiang1989" +owning-sig: sig-node +participating-sigs: + - sig-node + - sig-security +status: provisional +creation-date: 2024-09-01 +reviewers: + - TBD + - "@dongjiang1989" +approvers: + - TBD + - "@dongjiang1989" + +see-also: + - "/keps/sig-node/4788-mountpath-replacing-env-variable" + +# The target maturity stage in the current dev cycle for this KEP. +stage: alpha + +# The most recent milestone for which work toward delivery of this KEP has been +# done. This can be the current (upcoming) milestone, if it is being actively +# worked on. +latest-milestone: "v1.32" + +# The milestone at which this feature was, or is targeted to be, at each stage. +milestone: + alpha: "v1.32" + +# The following PRR answers are required at alpha release +# List the feature gate name and the components for which it must be enabled +feature-gates: + - name: VolumeMountPathEnvExpansion + components: + - kubelet +disable-supported: true From 21c3d1f546ba36ad9456a59b2cfcbd15b26c165f Mon Sep 17 00:00:00 2001 From: dongjiang1989 Date: Mon, 2 Sep 2024 16:47:36 +0800 Subject: [PATCH 2/4] dongjiang, fix see-also Signed-off-by: dongjiang1989 --- keps/sig-node/4788-volume-mountpath-env-expansion/kep.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keps/sig-node/4788-volume-mountpath-env-expansion/kep.yaml b/keps/sig-node/4788-volume-mountpath-env-expansion/kep.yaml index 27acd4659a8..a12ee1cff39 100644 --- a/keps/sig-node/4788-volume-mountpath-env-expansion/kep.yaml +++ b/keps/sig-node/4788-volume-mountpath-env-expansion/kep.yaml @@ -16,7 +16,7 @@ approvers: - "@dongjiang1989" see-also: - - "/keps/sig-node/4788-mountpath-replacing-env-variable" + - "/keps/sig-node/4788-volume-mountpath-env-expansion" # The target maturity stage in the current dev cycle for this KEP. stage: alpha From 995e57941010f315141e7e38fc62bec04a4d393b Mon Sep 17 00:00:00 2001 From: dongjiang Date: Tue, 3 Sep 2024 21:08:36 +0800 Subject: [PATCH 3/4] =?UTF-8?q?=E6=9B=B4=E6=96=B0=20README.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Itamar Holder <77444623+iholder101@users.noreply.github.com> --- keps/sig-node/4788-volume-mountpath-env-expansion/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/keps/sig-node/4788-volume-mountpath-env-expansion/README.md b/keps/sig-node/4788-volume-mountpath-env-expansion/README.md index 9e6237f972b..594dd0d0575 100644 --- a/keps/sig-node/4788-volume-mountpath-env-expansion/README.md +++ b/keps/sig-node/4788-volume-mountpath-env-expansion/README.md @@ -131,7 +131,7 @@ bogged down. I used a configMap to store the mount path value, but when i call in my pod it doesn't work it seems that the mountPath property can not be assigned to an env variable. -``` +```yaml spec: containers: - env: From 67cb05b557d5128c0aa6b2ec9a9fa02928a85942 Mon Sep 17 00:00:00 2001 From: dongjiang1989 Date: Thu, 5 Sep 2024 09:28:48 +0800 Subject: [PATCH 4/4] make update-toc Signed-off-by: dongjiang1989 --- keps/sig-node/4788-volume-mountpath-env-expansion/README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/keps/sig-node/4788-volume-mountpath-env-expansion/README.md b/keps/sig-node/4788-volume-mountpath-env-expansion/README.md index 594dd0d0575..735f6d92b33 100644 --- a/keps/sig-node/4788-volume-mountpath-env-expansion/README.md +++ b/keps/sig-node/4788-volume-mountpath-env-expansion/README.md @@ -19,6 +19,9 @@ - [Integration tests](#integration-tests) - [e2e tests](#e2e-tests) - [Graduation Criteria](#graduation-criteria) + - [Alpha](#alpha) + - [Beta](#beta) + - [GA](#ga) - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) - [Version Skew Strategy](#version-skew-strategy) - [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire)