chore: add required service when serviceMonitor is enabled

[bendikp]

Currently when enabling serviceMonitor the service isn't automatically enabled, and you have to explicitly enable the service in order for the serviceMonitor to work.

I think it would be nice to add the service when you enable serviceMonitor. Especially since the serviceMonitor don't work when you don't have a service.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign a7i for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Hi @bendikp. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ingvagabund]

Some users might deploy their own service monitor. Or, use a different mechanism for consuming services. I wonder whether the charts are expected to provide a "full" installation by default. Or, rather ask users to be more aware of what they do.

[bendikp]

Choosing the abstraction of the charts isn't easy. I guess I'm just used to getting a Service when enabling serviceMonitor. Enabling both service and serviceMonitor seems a bit redundant since they are so tightly coupled.

Having a more functional value like metrics.enabled: true|false might be a solution.

[ingvagabund]

That would cause strange cases like metrics.enabled=true && serviceMonitor.enabled=false.

I wonder if there's any precedence for cases like this. I.e. a higher level abstraction on top of the existing chart configuration.

[rouke-broersma]

That would cause strange cases like metrics.enabled=true && serviceMonitor.enabled=false.
I wonder if there's any precedence for cases like this. I.e. a higher level abstraction on top of the existing chart configuration.

If I look at a couple of the helm charts that I consume, they all do this in a slightly different way, however I think in general most charts enable the metrics service when metrics is enabled, and enable the service monitor when service monitor is also enabled.

Argocd: https://github.com/argoproj/argo-helm/blob/687392688fdbc28eb0813b083e3e63771d138c5a/charts/argo-cd/templates/argocd-application-controller/metrics.yaml#L1

Ingress-nginx: https://github.com/kubernetes/ingress-nginx/blob/1c2aecbf02898845b45cee3d0b493347b440995e/charts/ingress-nginx/templates/controller-service-metrics.yaml#L1

metrics.enabled=true && serviceMonitor.enabled=false is not a strange case, because you can use other tooling than prometheus to consume the metrics. In that case you have no need for a serviceMonitor, but you do want to enable the metrics endpoint and the metrics server.

However some charts do enable the service also when the service monitor is enabled:

Cillium: Cilium: https://github.com/cilium/cilium/blob/685790550b375380dd8fac71cc2a37427d78fc40/install/kubernetes/cilium/templates/cilium-agent/service.yaml#L3

In general the helm construct I see most often in the helm charts I consume is the following:

    metrics:
      enabled: true # Enabled metrics port and enables metrics service
      serviceMonitor:
        enabled: true # Enables ServiceMonitor that consumes metrics service

[ingvagabund]

@rouke-broersma thank you for exploring the existing solutions. Yeah, this change requires to reshape the current charts "api". So when the metrics are enabled, the service monitor (enabled by default) needs to be explicitly disabled if its functionality is not needed.

[ingvagabund]

@bendikp would you be interested in reshaping the current chart configuration structure?

[rouke-broersma]

@rouke-broersma thank you for exploring the existing solutions. Yeah, this change requires to reshape the current charts "api". So when the metrics are enabled, the service monitor (enabled by default) needs to be explicitly disabled if its functionality is not needed.

Just to clear, the most common case is that both metrics.enabled and metrics.serviceMonitor.enabled are false by default and that both need to be explicitly enabled for both to be deployed. It is not the common case that metrics.enabled implies metrics.serviceMonitor.enabled. To be most specific, the common condition is:

{{- if and .Values.metrics.enabled .Values.metrics.serviceMonitor.enabled -}}

While the ask in this PR is:

{{- if or .Values.metrics.enabled .Values.metrics.serviceMonitor.enabled -}}

Eg of the common case:

This enables metrics service and optionally metrics port (on container) but not serviceMonitor:

metrics:
  enabled: true

This does not enable anything:

metrics:
  enabled: false
  serviceMonitor:
    enabled: true

This enables all of metrics port (on container), metrics service and serviceMonitor:

metrics:
  enabled: true
  serviceMonitor:
    enabled: true

[a7i]

/ok-to-test