Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Adding PVC and volumeMount at the same time causes an error #1868

Open
LH2000b opened this issue Dec 24, 2024 · 0 comments
Open

[BUG] Adding PVC and volumeMount at the same time causes an error #1868

LH2000b opened this issue Dec 24, 2024 · 0 comments
Assignees
Labels
kind/bug Something isn't working

Comments

@LH2000b
Copy link

LH2000b commented Dec 24, 2024

What happened:
I have applied the following yaml file. When I added a new rough field, I observed that the Pod did not respond. When I updated other fields again, sts did not respond either. I suspect that the controller was hung by some abnormal state at this time.

apiVersion: apps.kruise.io/v1beta1
kind: StatefulSet
metadata:
  name: krusie-statefulset
  namespace: test-kruise
  labels:
    app: krusie
spec:
  serviceName: "krusie"
  replicas: 1
  selector:
    matchLabels:
      app: krusie
  template:
    metadata:
      annotations:
        cattle.io/timestamp: "2024-12-23T04:59:27Z"
      labels:
        app: krusie
    spec:
      containers:
      - name: krusie-container
        image: metabit-registry-registry-vpc.cn-zhangjiakou.cr.aliyuncs.com/images-proxy/ubuntu:22.04
        command: ["/bin/sh"]
        args: ["-c", "while true; do sleep 3600; done"]
        ports:
        - containerPort: 8080
          name: http
        volumeMounts:
        - name: persistence-volume
          mountPath: /data
// -------------------------ADD CODE 1
        - name: persistence-volume2                                          
          mountPath: /data2
// -------------------------
  volumeClaimTemplates:
  - apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: persistence-volume
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 60Gi
      storageClassName: alicloud-disk-topology-cloud-ssd
      volumeMode: Filesystem
// -----------------------------ADD CODE 2
  - apiVersion: v1                                                                           
    kind: PersistentVolumeClaim
    metadata:
      name: persistence-volume2
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 60Gi
      storageClassName: alicloud-disk-topology-cloud-ssd
      volumeMode: Filesystem
// ------------------------------

I found the following error in the controller log.

StatefulPodControl failed to update Pod error: Pod \"krusie-statefulset-0\"
      is invalid: spec: Forbidden: pod updates may not change fields other than `spec.containers[*].image`,
      `spec.initContainers[*].image`, `spec.activeDeadlineSeconds`, `spec.tolerations`
      (only additions to existing tolerations) or `spec.terminationGracePeriodSeconds`
      (allow it to be set to 1 if it was previously negative)

After step-by-step investigation, it was found that the error was reported from the following link:
https://github.com/openkruise/kruise/blob/master/pkg/controller/statefulset/stateful_pod_control.go#L190
image

It looks like an attempt is made to update a Pod, but in fact, K8s prohibits updating most fields of a Pod, including volume.

When I removed the following line of code, I observed that sts performed the rolling update as I expected.
https://github.com/openkruise/kruise/blob/master/pkg/controller/statefulset/stateful_pod_control.go#L160
image

What you expected to happen:
Modifying the PVC and VolumeMount field can trigger rolling updates normally

How to reproduce it (as minimally and precisely as possible):
Reference
Anything else we need to know?:
nothing

Environment:

  • Kruise version: 1.63~1.73
  • Kubernetes version (use kubectl version): v1.24.6-aliyun.1
  • Install details (e.g. helm install args):
crds:
  managed: true
daemon:
  enablePprof: true
  extraEnvs: null
  log:
    level: '4'
  nodeSelector: {}
  port: 10221
  pprofAddr: localhost:10222
  resources:
    limits:
      cpu: 50m
      memory: 128Mi
    requests:
      cpu: '0'
      memory: '0'
  socketFile: ''
  socketLocation: /var/run
enableKubeCacheMutationDetector: false
featureGates: >-
  ImagePullJobGate=true,ResourcesDeletionProtection=false,PodUnavailableBudgetDeleteGate=false,WorkloadSpread=false,PodProbeMarkerGate=false
installation:
  createNamespace: true
  namespace: kruise-system
  roleListGroups:
    - '*'
manager:
  healthProbe:
    port: 8000
  hostNetwork: false
  image:
    repository: metabit-registry-registry-vpc.cn-zhangjiakou.cr.aliyuncs.com/images-proxy/kruise-manager
    tag: v1.7.3
  log:
    level: '4'
  metrics:
    port: 8080
  nodeAffinity: {}
  nodeSelector: {}
  pprofAddr: localhost:8090
  replicas: 2
  resources:
    limits:
      cpu: 200m
      memory: 512Mi
    requests:
      cpu: 100m
      memory: 256Mi
  resyncPeriod: '0'
  tolerations: null
  webhook:
    port: 9876
serviceAccount:
  annotations: {}
webhookConfiguration:
  timeoutSeconds: 30
  • Others:
@LH2000b LH2000b added the kind/bug Something isn't working label Dec 24, 2024
@ABNER-1 ABNER-1 assigned ABNER-1 and unassigned FillZpp Dec 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants