You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
achave11-ucsc opened this issue
Nov 1, 2024
· 2 comments
Labels
-[priority] Mediumenh[type] New feature or requestinfra[subject] Project infrastructure like CI/CD, build and deployment scriptsneeds spike[process] Needs to be spiked before it can be solvedorange[process] Done by the Azul teamperf[subject] Performance, efficiency or cost
@hannes-ucsc: "We have around ten images to copy. When we used docker pull and docker push to copy them, doing all images in parallel (using Terraforms default concurrency) would overload dockerd/containerd so we added a semaphore to serialize that part. Now that we use skopeo to copy images, we may want to remove the semaphore again or set it to a higher initial value for increased concurrency."
The text was updated successfully, but these errors were encountered:
Spike to test and measure the running time of deploying tempdev.shared with an initial value of 1, 5 and 10, deleting all images from ECR before every run.
hannes-ucsc
added
bug
[type] A defect preventing use of the system as specified
infra
[subject] Project infrastructure like CI/CD, build and deployment scripts
perf
[subject] Performance, efficiency or cost
enh
[type] New feature or request
and removed
bug
[type] A defect preventing use of the system as specified
labels
Nov 6, 2024
hannes-ucsc
changed the title
Images are not copied in parallel
Copy Docker images in parallel when mirroring to ECR
Nov 6, 2024
Skopeo copies six layers in parallel by default. However, many images have one large layer and many small ones, so copying multiple images in parallel might still be worth it. Hence the spike.
-[priority] Mediumenh[type] New feature or requestinfra[subject] Project infrastructure like CI/CD, build and deployment scriptsneeds spike[process] Needs to be spiked before it can be solvedorange[process] Done by the Azul teamperf[subject] Performance, efficiency or cost
@hannes-ucsc: "We have around ten images to copy. When we used
docker pull
anddocker push
to copy them, doing all images in parallel (using Terraforms default concurrency) would overload dockerd/containerd so we added a semaphore to serialize that part. Now that we useskopeo
to copy images, we may want to remove the semaphore again or set it to a higher initial value for increased concurrency."The text was updated successfully, but these errors were encountered: