Bug: Listed containers in helm file don't seem to exist for 0.4.1

[jdblack]

What happened:

While attempting to perform a new installation of datashim via helm (via terraform) for the first time, the install bombed out.

16:45 $ k get pods -n kube-storage
NAME READY STATUS RESTARTS AGE
csi-attacher-nfsplugin-0 0/2 CrashLoopBackOff 6 (3m54s ago) 13m
csi-attacher-s3-0 0/1 CrashLoopBackOff 6 (3m59s ago) 13m
csi-nodeplugin-nfsplugin-4xkcv 0/2 ImagePullBackOff 6 (4m11s ago) 13m
csi-nodeplugin-nfsplugin-56r82 0/2 ImagePullBackOff 6 (3m44s ago) 13m
csi-nodeplugin-nfsplugin-6fl8j 0/2 ImagePullBackOff 6 (3m49s ago) 13m
csi-nodeplugin-nfsplugin-c7v4h 0/2 ImagePullBackOff 6 (4m5s ago) 13m
csi-nodeplugin-nfsplugin-h5cgw 0/2 ImagePullBackOff 6 (3m44s ago) 13m
csi-nodeplugin-nfsplugin-qpq5g 0/2 ImagePullBackOff 6 (4m8s ago) 13m
csi-provisioner-s3-0 0/1 CrashLoopBackOff 6 (3m32s ago) 13m
csi-s3-8fprp 0/2 ImagePullBackOff 6 (3m52s ago) 13m
csi-s3-drbkz 0/2 ImagePullBackOff 6 (4m4s ago) 13m
csi-s3-kfkgt 0/2 ImagePullBackOff 6 (4m1s ago) 13m
csi-s3-skllw 0/2 ImagePullBackOff 6 (3m49s ago) 13m
csi-s3-wcvbj 0/2 ImagePullBackOff 6 (3m59s ago) 13m
csi-s3-zmtrb 0/2 ImagePullBackOff 6 (3m49s ago) 13m

One image here, for example:
Warning Failed 8m45s (x4 over 10m) kubelet Failed to pull image "quay.io/datashim-io/csi-nfs:8f50a01": rpc error: code = NotFound desc = failed to pull and unpack image "quay.io/datashim-io/csi-nfs:8f50a01": failed to resolve reference "quay.io/datashim-io/csi-nfs:8f50a01": quay.io/datashim-io/csi-nfs:8f50a01: not found

A check at https://quay.io/repository/datashim-io/csi-nfs?tab=tags&tag=latest seems to confirm that this particular image is missing.

What you expected to happen:

A working helm driven deployment similar to the one provided by kube manifest =)

How to reproduce it (as minimally and precisely as possible):

Try to deploy and observer the containers in imagepullbackoff

datashim kube-storage 1 2025-01-21 16:32:34.909518 +0700 +07 failed datashim-charts-0.4.1 0.4.1

Anything else we need to know?:

Environment:

• Datashim version: 0.4.1 (Helm version)
• Kubernetes version (use kubectl version): 1.3.2
• Kubernetes distribution: via kubeadm
• Cloud provider or hardware configuration: Hardware

[srikumar003]

@jdblack Thanks for the report! Checking now

[jdblack]

Thank you!

Were you able to replicate it? I tried digging into the helm charts a little bit, and I had trouble figuring out where it got those image IDs from. I'm a bit confused because in the helm, it seems like nice tag names are being used, rather than hash ids.

Please let me know if there's anything I can followup with!

[srikumar003]

I was able to replicate it on kind and trying to find out why the image IDs crept in (our release github action tags the images with the right version number)

If it helps, the image IDs is the commit id from which the release was generated. But still does not explain what happened

I'll keep digging into it. But, please feel free to share further insights!

[srikumar003]

The problem is fixed for now but I am still investigating.

Background:

1. The release tag for Datashim is applied only to the images for dataset-operator, generate-keys, and cos-uploader components
2. Images for the CSI components csi-nfs and csi-s3 are tagged with the commit ID at the time of the release.
3. All the components are packaged into the helm charts

What happened:

For reasons unknown, the images for csi-nfs and csi-s3 were not tagged correctly, even though they were successfully pushed to the image repository

As a temporary fix, I have tagged the images in the image repository with the correct ID. The helm chart install works and I have verified this. Will post further when I have more insights

[jdblack]

Thank you very much!

…

On Mon, Jan 27, 2025, 5:22 PM Srikumar Venugopal ***@***.***> wrote: The problem is fixed for now but I am still investigating. Background: 1. The release tag for Datashim is applied only to the images for dataset-operator, generate-keys, and cos-uploader components 2. Images for the CSI components csi-nfs and csi-s3 are tagged with the commit ID at the time of the release. 3. All the components are packaged into the helm charts What happened: For reasons unknown, the images for csi-nfs and csi-s3 were not tagged correctly, even though they were successfully pushed to the image repository <https://github.com/datashim-io/datashim/actions/runs/11971781874/job/33380634838> As a temporary fix, I have tagged the images in the image repository with the correct ID. The helm chart install works and I have verified this. Will post further when I have more insights — Reply to this email directly, view it on GitHub <#397 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIO4LLJONZF5O5FG2QL7332MYCHTAVCNFSM6AAAAABVSC3OV6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMJVGM2TSMBZG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>