Managing Kubernetes Quota with confidence

Kotary

Table of Contents

• Kotary
  • What is it ?
  • Why not use an admission controller ?
  • Adding or Scaling-Up a Quota
  • Installation
    • Add the CRD
    • Configuration
      • Options
      • Example
    • Deployment
      • Deploy the controller
      • (Optional) Deploy the service monitor
  • Getting Started
    • Update a ResourceQuota
      • Example
      • Status
        • Example of a rejected claim
        • Example of a pending claim
    • Default claim
  • Plan
  • Manage
    • Global
    • Namespaces
    • Namespace Details
  • Versioning

What is it ?

It is an operator that brings a layer of verification and policy to the native ResourceQuotas mechanism. It introduced a new resource call a ResourceQuotaClaims that will let users ask to modify the specification of their quota. The verification includes :

• There are enough resources (CPU and Memory) on the cluster to allocate the claim which will look at the total amount of resources of the cluster (worker node) and the sum of all the other ResourceQuotas
• (Optional) It respects the maximum bound value express as a ratio of the cluster resource, ex: a namespace cannot claim more that a 1/3 of the cluster
• (Optional) In order to have some flexibility it is possible to set an over-commit or under-commit ratio to set what is claimable compared to the actual resources. ex: In a development environment you could choose to allow reserving more resources than what is actually usable in reality.

In order to facilitate the adaption of ResourceQuotaClaims it is possible to enforce a default claim for namespaces. The feature will be activated on namespace that contains the label quota=managed.

Why not use an admission controller ?

It could have been an elegant solution to use the admission controller mechanism in Kubernetes. This would have avoided the use of a Custom Resource Definition by directly asking to modify a ResourceQuotas. In the meantime this would have left out users on managed cluster like EKS, AKS or GKE, this is why we implemented the operator pattern instead.

Adding or Scaling-Up a Quota

Installation

Add the CRD

kubectl apply -f https://raw.githubusercontent.com/ca-gip/kotary/master/artifacts/crd.yml

Configuration

Options

Name	Description	Mandatory	Type	Default
defaultClaimSpec	Default claim that will be added to a watched Namespace	no	ResourceList	cpu:2 memory: 6Gi
ratioMaxAllocationMemory	Maximum amount of Memory claimable by a Namespace	no	Float	1
ratioMaxAllocationCPU	Maximum amount of CPU claimable by a Namespace	no	Float	1
ratioOverCommitMemory	Memory over-commitment	no	Float	1
ratioOverCommitCPU	CPU over-commitment	no	Float	1

Example

In the following sample configuration we set :

• A default claim of 2 CPU and 10Gi of Memory
• 33% of total amount of resource can be claim by a namespace
• An over-commit of 130%

cat <<EOF | kubectl -n kube-system create -f -
apiVersion: v1
kind: ConfigMap
data:
  defaultClaimSpec: |
    cpu: "2"
    memory: "10Gi"
  ratioMaxAllocationMemory: "0.33"
  ratioMaxAllocationCPU: "0.33"
  ratioOverCommitMemory: "1.3"
  ratioOverCommitCPU: "1.3"
metadata:
  name: kotary-config
EOF

Deployment

Deploy the controller

kubectl apply -f https://raw.githubusercontent.com/ca-gip/kotary/master/artifacts/deployment.yml

(Optional) Deploy the service monitor

kubectl apply -f https://raw.githubusercontent.com/ca-gip/kotary/master/artifacts/metrics.yml

Contributing

• Contributing to kotary.

---

Developed with