Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No such file or directory: "subvolume group 'csi' does not exist" #4548

Open
fenggolang opened this issue Apr 6, 2024 · 3 comments
Open
Labels
component/cephfs Issues related to CephFS component/deployment Helm chart, kubernetes templates and configuration Issues/PRs component/docs Issues and PRs related to documentation

Comments

@fenggolang
Copy link

fenggolang commented Apr 6, 2024

Describe the bug

The dynamic creation of PV (Persistent Volume) for CephFS failed.

error info:
image

Environment details

  • Image/version of Ceph CSI driver :
  • Helm chart version : v3.11.0
  • Kernel version : 6.4.0
  • Mounter used for mounting PVC (for cephFS its fuse or kernel. for rbd its
    krbd or rbd-nbd) :
  • Kubernetes cluster version : v1.29.3
  • Ceph cluster version : v16.2.15

Steps to reproduce

[root@k8s-master-172-28-205-201 cephfs]# cat 01-ceph-csi-cephfs-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: csi-cephfs-secret
  namespace: default
stringData:
  # Required for statically provisioned volumes
  # ceph auth get-or-create client.k8s-cephfs-user mon 'allow r' osd 'allow rw pool=cephfs_data' mds 'allow' && ceph auth get-key client.k8s-cephfs-user
  #userID: k8s-cephfs-user
  #userKey: AQC7LxFm8WeZBBAAa4Lo/Yd10V6ZN3OrMNbjdg==
  userID: admin
  userKey: AQDWMQxmDDKaLBAA4JX6+UiEitT+6m1hJPt0QA==

  # Required for dynamically provisioned volumes
  # admin/ceph auth get-key client.admin
  adminID: admin
  adminKey: AQDWMQxmDDKaLBAA4JX6+UiEitT+6m1hJPt0QA==

  # Encryption passphrase
  encryptionPassphrase: test_passphrase
[root@k8s-master-172-28-205-201 cephfs]# cat 02-ceph-csi-cephfs-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: csi-cephfs-storage
provisioner: cephfs.csi.ceph.com
parameters:
  clusterID: 9a6f67b8-f10d-11ee-8839-000c297f0094
  pool: cephfs.cephfs1.data
  fsName: cephfs1
  csi.storage.k8s.io/provisioner-secret-name: csi-cephfs-secret
  csi.storage.k8s.io/provisioner-secret-namespace: default
  csi.storage.k8s.io/controller-expand-secret-name: csi-cephfs-secret
  csi.storage.k8s.io/controller-expand-secret-namespace: default
  csi.storage.k8s.io/node-stage-secret-name: csi-cephfs-secret
  csi.storage.k8s.io/node-stage-secret-namespace: default
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: Immediate
mountOptions:
  - discard
[root@k8s-master-172-28-205-201 cephfs]# cat 03-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: csi-cephfs-pvc1
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: csi-cephfs-storage
[root@k8s-master-172-28-205-201 cephfs]# cat 04-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: csi-cephfs-demo-pod
spec:
  containers:
    - name: web-server
      image: harbor.openeuler-test.club/test/nginx:latest
      volumeMounts:
        - name: mypvc
          mountPath: /var/lib/www
  volumes:
    - name: mypvc
      persistentVolumeClaim:
        claimName: csi-cephfs-pvc1
        readOnly: false
[root@k8s-master-172-28-205-201 cephfs]#
@Madhu-1
Copy link
Collaborator

Madhu-1 commented Apr 8, 2024

@fenggolang you need to create subvolumegroup in the filesystem before creating the PVC

ceph fs subvolumegroup create <filesystemname> csi

@nixpanic nixpanic added component/cephfs Issues related to CephFS component/deployment Helm chart, kubernetes templates and configuration Issues/PRs labels Apr 9, 2024
@pschichtel
Copy link

@Madhu-1 is this something new in 3.11 ?

@Madhu-1
Copy link
Collaborator

Madhu-1 commented Apr 15, 2024

@Madhu-1 is this something new in 3.11 ?

@pschichtel This is part of 3.10 release https://github.com/ceph/ceph-csi/releases/tag/v3.10.0

@nixpanic nixpanic added the component/docs Issues and PRs related to documentation label Apr 15, 2024
nixpanic added a commit to nixpanic/ceph-csi that referenced this issue Apr 17, 2024
It seems possible that the .Destroy() function is called on a nil
pointer. This would cause a panic in the node-plugin.

Depending on how far GenVolFromVolID() comes, the returned rbdVolume can
be connected. If an error occurs, the rbdVolume should not be connected
at all, so call the .Destroy() function in those cases too.

Fixes: ceph#4548
Signed-off-by: Niels de Vos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/cephfs Issues related to CephFS component/deployment Helm chart, kubernetes templates and configuration Issues/PRs component/docs Issues and PRs related to documentation
Projects
None yet
Development

No branches or pull requests

4 participants