[Feature Request] Node selector and tolerations support for deployed pods

[kellian-cottart]

Node selector and tolerations support

Issue

At the moment, a user cannot specify the tolerations and the nodes to deploy the services on. In other words, let's say that I want to deploy a vscode on a node with a specific taint code-node:true, this option isn't available.

Wanted behaviors

Similar to MinIO Operator when deploying pools for tenants, we could imagine parameters in Onyxia's Kubernetes tab in Service configuration that allow the user to select the desired tolerations and nodes.

Using the previous example, it would be something similar to the effect of writing this in a Helm values.yaml manifest:

tolerations: 
  - key: "code-node"
    operator: "Equal"
    value: "true"
    effect: "NoSchedule"
nodeSelector:
  code-node: "true"

Possible Implementation 1

These specifications already exist in the catalog's various Helm values.yaml files.

## Used to specify a toleration for a pod
tolerations: [ ]
## nodeSelector parameters. It specifies a map of key-value pairs. For the pod to be
## eligible to run on a node, the node must have each of the
## indicated key-value pairs as labels.
nodeSelector: { }

To add this behavior, we could imagine that the values.schema.json would be enriched with something along the lines of the following json object:

{
    "tolerations": {
        "description": "Configuration of the tolerations",
        "type": "object",
        "properties": {
            "key": {
                "type": "string",
                "x-form": {
                    "value": "{{tolerations.key}}"
                }
            },
            "operator": {
                "type": "string",
                "default": "Equal"
            },
            "value": {
                "type": "string",
                "default": "true",
                "x-form": {
                    "value": "{{tolerations.value}}"
                }
            },
            "effect": {
                "type": "string",
                "default": "NoSchedule",
                "x-form": {
                    "value": "{{tolerations.effect}}"
                }
            }
        }
    },
    "nodeSelector": {
        "description": "Node Selector",
        "type": "object",
        "properties": {
            "key": {
                "type": "string",
                "x-form": {
                    "value": "{{nodeSelector.key}}"
                }
            },
            "value": {
                "type": "boolean",
                "default": true,
                "x-form": {
                    "value": "{{nodeSelector.value}}"
                }
            }
        }
    }
}

Possible Implementation 2

Along the same lines, we could do something similar but it could be region wise, meaning that all pods deployed under the same regions are following the same deployment specifications with tolerations and node selector;

This method is less flexible, but may be simpler to do.