Update prometheus-federator to use dependencies from k8s 1.32.X+

[alexandreLamarre]

Related issues:

• Prometheus Federator + Monitoring Charts Reorg Project rancher#48591
• Dependency updates for Prometheus Federator, helm project operator and helm locker rancher#46296

[mallardduck]

BTW - I just realized that since this is targeting 1.32 we should ideally wait to ship this along with Rancher 2.11 that officially supports 2.11. And we're using RC libraries intended for Rancher 2.11 which feels wrong to include this soon. Because we don't do branching right now we can't merge this into main and expect it to not affect the February release cycle.

So what if you changed this to target 1.31 and Rancher 2.10 based libraries that way it doesn't have to be something we only do in 2.11. Otherwise we have to some how implement branch strategy and do partial reverts before the February release cycle.

[alexandreLamarre]

@mallardduck The intention was always to bump dependencies as aggressively as possible and branch off the main branch, and downgrade dependencies selectively

[mallardduck]

@alexandreLamarre - I guess that wasn't super clear. Either way works in the end I guess.

However after merging this we need to be careful about what tag is used to test main. If main needs to match rancher main, then that means the tag needs to logically match that version and leave a gaps. That way we have either a major/minor that correlates to the Rancher version, but versions under it that we can use to fill in gaps later.

i.e. After merging this the tag for main becomes 6.x by default, and covers 2.11; so in turn 5.x is for 2.10 and so on.

I think my misunderstanding there also may mean my opinion in change ordering will be different here. Specifically I think we do need to sort out my effort to split charts+image code first. Since having that complete will make us more prepared for the branching once implemented here.

[mallardduck]

Is this worth keeping - looks commented out.

[mallardduck]

🤔 KUBECONFIG=$KUBECONFIG think this isn't needed?

Or does this help KUBECONFIG=something make integration work better? If so, what if it was done as export KUBECONFIG=$KUBECONFIG in versions, I think that would keep that same context consistent for any make scripts.

[mallardduck]

Think that rancher/klipper-helm:v0.9.4-build20250113 will match the helm-controller we're targeting with this branch.

[mallardduck]

This part may want to be removed and let users control it.

I'm worried that, if:

• User disabled k3s/rke2 helm-controller and
• User has namespace helm-controller mode setup, and
• user has DETECT_K3S_RKE2=true - because one of the namespace ones owns CRDs,
• user manually set helmController.enabled=true to PromFed be the controller

This change will create a breaking change - maybe that's fine but at the very least we need to document/Release Note it for users in that situation to know:

• They should also deploy a helm-controller in namespaced mode to cover PromFed too.

[mallardduck]

Maybe I'm just confused about the wording of the second line comment here?

[alexandreLamarre]

I can double check but I'm fairly certain this matches 1-1 the existing control flow in main.

I also want to double check when we say namespaced helm-controller we mean one that watches all namespaces but has the controller-name set to something prometheus federator specific -- since the embedded helm-controller expects to watch across arbitrary namespaces

[alexandreLamarre]

prometheus-federator/internal/helm-project-operator/crd/crds.go

Lines 267 to 272 in 57444ba

	onK3sRke2 := k8sRuntimeType == "k3s" || k8sRuntimeType == "rke2"
	if onK3sRke2 {
	logrus.Debug("the cluster is running on k3s (or rke2), `helm-controller` CRDs will not be managed by `prometheus-federator`")
	}
	return !onK3sRke2

Every time we attempt to create crds in main we do the following, which automatically disables helm-controller crds

prometheus-federator/internal/helm-project-operator/crd/crds.go

Lines 204 to 206 in 57444ba

	if shouldManageHelmControllerCRDs(cfg) {
	crdDefs = append(crdDefs, helmControllerCrdDefs...)
	}

[alexandreLamarre]

Note that here, in the shared CRD abstraction :

prometheus-federator/cmd/prometheus-federator/main.go

Lines 94 to 97 in 22072cd

	managedCrds = common.ManagedCRDsFromRuntime(common.RuntimeOptions{
	DisableEmbeddedHelmLocker: f.RuntimeOptions.DisableEmbeddedHelmLocker,
	DisableEmbeddedHelmController: true,
	})

We don't actually disable the helm-controller (since we're not overwriting the config passed to the operator), we just tell the crd tracker to not track crds related to helm-controller, so users can still have the embedded helm-controller running, but not manage the crds in prom-fed

[mallardduck]

when we say namespaced helm-controller
Oh - sorry that's right I had a brain fart. The PromFed embedded helm-controller would be a second instance of a "global controller" since it's listening for all namespaces - but only for charts designated to be managed by it.

Where as if they opt to disable the PromFed provided controller then to be compliant with helm-controller policy and still keep PromFed working, the must:
a) use a global controller (either embedded in k3s/rke2, or added to any k8s), OR
b) use namespaced controller(s) so that each namespace they use for PromFed are covered.

[mallardduck]

So yeah - I think we're good and I was confused by the wording initially. I don't have any proposed changes to fix the wording - I probably just needed to read it slower 😆