feat: Watch Namespaces based on labels and label selectors #9976

davidjumani · 2024-08-30T14:18:28Z

Description

The watchNamespaces setting allows users to restrict the namespaces in which Edge watches resources. Since this is a static list, users need to update it when they need to modify namespaces to watch
This feature aims to dynamically determine the list of namespaces to watch by defining labels on namespaces or filtering namespaces based on a label expression.

This feature identifies namespaces with label selectors. These can be :

Matching labels : All namespaces that contain a predefined list of labels will be added to the list of namespaces to watch. Eg.: All namespaces labelled with gloo-translate=enabled will be watched
Matching expressions : All namespaces that contain labels that match a given criteria will be watched. Eg.: All namespaces are labelled with environment in (prod, dev) will be watched

This will introduce a new setting to select namespaces :

apiVersion: gloo.solo.io/v1
kind: Settings
metadata:
  name: default
spec:
  watchNamespaceSelectors:
  - matchLabels:
      gloo-translate: enabled
  - matchExpressions:
    - key: env
      operator: In
      values:
        - prod
        - dev

API changes

Added the watchNamespaceSelectors to the settings CRC

Code changes

Added a KubeNamespaceWatcher to the setup snapshot emitter. This will trigger a new snapshot if any namespace has been created / deleted / modified and not necessarily if the namespace belongs to the list we watch. The syncer will determine whether to sync based on whether :

The settings CR has changed
The namespaces we watch has chagned

Context

Watch Namespaces based on labels and label selectors
Design doc

Interesting decisions

Testing steps

Added kubernetes e2e tests

Notes for reviewers

Checklist:

I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added tests that prove my fix is effective or that my feature works

…paces

davidjumani · 2024-08-30T14:22:43Z

projects/gloo/pkg/bootstrap/clients/vault/vault.go

@@ -1,11 +1,10 @@
-package clients
+package vault


Moved this file to prevent an import cycle

github-actions · 2024-08-30T14:32:06Z

Visit the preview URL for this PR (updated for commit 19bcb53):

https://gloo-edge--pr9976-watch-namespace-sele-twteibok.web.app

_{(expires Tue, 01 Oct 2024 18:58:43 GMT)}

_{🔥 via Firebase Hosting GitHub Action 🌎}

_{Sign: 77c2b86e287749579b7ff9cadb81e099042ef677}

solo-changelog-bot · 2024-08-30T14:41:42Z

Issues linked to changelog:
solo-io#9274

test/kube2e/helm/artifacts/helm.yaml

davidjumani · 2024-09-17T18:59:57Z

projects/gloo/pkg/validation/server.go

+func HandleResourceDeletion(snapshot *v1snap.ApiSnapshot, resource resources.Resource) error {
+	if _, ok := resource.(*sk_kubernetes.KubeNamespace); ok {
+		// Special case to handle namespace deletion
+		snapshot.RemoveAllResourcesInNamespace(resource.GetMetadata().GetName())
+		return nil
+	} else {
+		return snapshot.RemoveFromResourceList(resource)
+	}
+}
+


Would change to

diff --git a/projects/gloo/pkg/validation/server.go b/projects/gloo/pkg/validation/server.go index 7621d7433..630086eae 100644 --- a/projects/gloo/pkg/validation/server.go +++ b/projects/gloo/pkg/validation/server.go @@ -210,7 +210,9 @@ func (s *validator) Validate(ctx context.Context, req *validation.GlooValidation func HandleResourceDeletion(snapshot *v1snap.ApiSnapshot, resource resources.Resource) error { if _, ok := resource.(*sk_kubernetes.KubeNamespace); ok { // Special case to handle namespace deletion - snapshot.RemoveAllResourcesInNamespace(resource.GetMetadata().GetName()) + snapshot.RemoveMatches(func(metadata *core.Metadata) bool { + return resource.GetMetadata().GetNamespace() == metadata.GetNamespace() + }) return nil } else { return snapshot.RemoveFromResourceList(resource)

after the solo-kit bump

.github/workflows/pr-kubernetes-tests.yaml

pkg/utils/namespaces/namespaces.go

jbohanon · 2024-09-17T20:41:26Z

pkg/utils/namespaces/namespaces.go

+
+	resp, err := clientset.AuthorizationV1().SelfSubjectAccessReviews().Create(ctx, &selfCheck, metav1.CreateOptions{})
+	if err != nil {
+		return &FakeKubeNamespaceWatcher{}


For example, if we failed here and just silently returned a no-op, it would not be clear to a user or dev that we may be running into some RBAC issues. Returning a nil client and an error would lead to better debugging I think

pkg/utils/setuputils/setup_syncer.go

projects/gateway/pkg/services/k8sadmission/validating_admission_webhook.go

jbohanon · 2024-09-17T21:55:51Z

projects/gateway/pkg/services/k8sadmission/validating_admission_webhook.go

+	// Kubernetes' Secrets and namespace deletions are the only non-Solo API resource operations we support validation requests for.
+	// For a these resources, we want to skip validation on operations we don't support. We only support DELETEs.


Suggested change

// Kubernetes' Secrets and namespace deletions are the only non-Solo API resource operations we support validation requests for.

// For a these resources, we want to skip validation on operations we don't support. We only support DELETEs.

// Kubernetes Secrets and Namespace are the only non-Solo API resources we support validation requests for.

// For a these resources, we want to skip validation on operations we don't support. For Namespaces, we

// support DELETE and UPDATE. For Secrets, we support DELETE.

jbohanon · 2024-09-17T22:11:41Z

test/kubernetes/e2e/tests/base/base_suite.go

@@ -49,6 +49,8 @@ type BaseTestingSuite struct {

 func NewBaseTestingSuite(ctx context.Context, testInst *e2e.TestInstallation, testHelper *helper.SoloTestHelper, setup SimpleTestCase, testCase map[string]*TestCase) *BaseTestingSuite {
 	namespace = testInst.Metadata.InstallNamespace
+	ctx = context.WithValue(ctx, "namespace", namespace)


The provider has a reference to the gloogateway.Context which has the InstallationNamespace

…electors

davidjumani · 2024-09-24T20:22:06Z

closing in favour of solo-io#10104

davidjumani added 16 commits August 19, 2024 17:00

watch namespaces by labels

e0dbe40

update settings

f074c85

namespace cache

935e05c

update

8f1a43b

add check

7da343d

sync

576c3a4

fix mutex

9441024

helm changes

ea82628

refactor

aac1301

Merge branch 'main' of github.com:solo-io/gloo into label-wathc-names…

bbfb6ab

…paces

fix

972d09a

refactor

3eefaf5

tests

92151c2

more tests

ea20ee1

cleanup

2b6e9c4

add stats

f6add91

davidjumani requested a review from a team as a code owner August 30, 2024 14:18

Merge branch 'main' into watch-namespace-selectors

8dd0c19

github-actions bot added keep pr updated signals bulldozer to keep pr up to date with base branch work in progress signals bulldozer to keep pr open (don't auto-merge) labels Aug 30, 2024

tilt changes

db9392c

davidjumani commented Aug 30, 2024

View reviewed changes

davidjumani added 2 commits August 30, 2024 10:40

changelog

15f5555

update tests

afb516a

davidjumani changed the title ~~feat: Watch Namespaces based on labels and label selectors~~ [WIP] feat: Watch Namespaces based on labels and label selectors Aug 30, 2024

davidjumani commented Aug 30, 2024

View reviewed changes

test/kube2e/helm/artifacts/helm.yaml Outdated Show resolved Hide resolved

davidjumani and others added 2 commits August 30, 2024 10:48

revert helm test changes

b920e44

Adding changelog file to new location

f1fd370

davidjumani and others added 10 commits September 11, 2024 14:51

fix tests

8c2b902

Merge refs/heads/main into watch-namespace-selectors

d9e2543

revert shortcircuit

7c43113

revert return

d475b9d

Merge refs/heads/main into watch-namespace-selectors

f5fda2f

Merge refs/heads/main into watch-namespace-selectors

58d1aaf

Merge refs/heads/main into watch-namespace-selectors

2e1018d

Merge refs/heads/main into watch-namespace-selectors

a7d6196

Merge refs/heads/main into watch-namespace-selectors

2af864c

Merge refs/heads/main into watch-namespace-selectors

4309efd

davidjumani commented Sep 17, 2024

View reviewed changes

changelog-bot added 2 commits September 17, 2024 21:19

Adding changelog file to new location

61c83dc

Deleting changelog file from old location

81a9629

jbohanon reviewed Sep 17, 2024

View reviewed changes

davidjumani and others added 9 commits September 18, 2024 16:47

address comments

287a17e

fix test

870c8a0

Merge branch 'main' of github.com:solo-io/gloo into watch-namespace-s…

1557cdf

…electors

Merge refs/heads/main into watch-namespace-selectors

78eae93

add comments

d607745

address comments

2521a4b

codegen

374c3ca

remove context stuff

6079884

Merge branch 'main' of github.com:solo-io/gloo into watch-namespace-s…

a139cd2

…electors

davidjumani changed the title ~~[WIP] feat: Watch Namespaces based on labels and label selectors~~ feat: Watch Namespaces based on labels and label selectors Sep 20, 2024

soloio-bulldozer bot added 3 commits September 23, 2024 18:22

Merge refs/heads/main into watch-namespace-selectors

bdab112

Merge refs/heads/main into watch-namespace-selectors

ca0afa7

Merge refs/heads/main into watch-namespace-selectors

19bcb53

davidjumani closed this Sep 24, 2024

github-actions bot mentioned this pull request Nov 14, 2024

[Migrated] feat: Watch Namespaces based on labels and label selectors solo-io/gloo#9976

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Watch Namespaces based on labels and label selectors #9976

feat: Watch Namespaces based on labels and label selectors #9976

davidjumani commented Aug 30, 2024 •

edited

Loading

davidjumani Aug 30, 2024

github-actions bot commented Aug 30, 2024 •

edited

Loading

solo-changelog-bot bot commented Aug 30, 2024

davidjumani Sep 17, 2024

jbohanon Sep 17, 2024

jbohanon Sep 17, 2024

jbohanon Sep 17, 2024

davidjumani commented Sep 24, 2024

		// Kubernetes' Secrets and namespace deletions are the only non-Solo API resource operations we support validation requests for.
		// For a these resources, we want to skip validation on operations we don't support. We only support DELETEs.

-	// Kubernetes' Secrets and namespace deletions are the only non-Solo API resource operations we support validation requests for.
-	// For a these resources, we want to skip validation on operations we don't support. We only support DELETEs.
+	// Kubernetes Secrets and Namespace are the only non-Solo API resources we support validation requests for.
+	// For a these resources, we want to skip validation on operations we don't support. For Namespaces, we
+	// support DELETE and UPDATE. For Secrets, we support DELETE.

feat: Watch Namespaces based on labels and label selectors #9976

feat: Watch Namespaces based on labels and label selectors #9976

Conversation

davidjumani commented Aug 30, 2024 • edited Loading

Description

API changes

Code changes

Context

Interesting decisions

Testing steps

Notes for reviewers

Checklist:

davidjumani Aug 30, 2024

Choose a reason for hiding this comment

github-actions bot commented Aug 30, 2024 • edited Loading

solo-changelog-bot bot commented Aug 30, 2024

davidjumani Sep 17, 2024

Choose a reason for hiding this comment

jbohanon Sep 17, 2024

Choose a reason for hiding this comment

jbohanon Sep 17, 2024

Choose a reason for hiding this comment

jbohanon Sep 17, 2024

Choose a reason for hiding this comment

davidjumani commented Sep 24, 2024

davidjumani commented Aug 30, 2024 •

edited

Loading

github-actions bot commented Aug 30, 2024 •

edited

Loading