stop creating tags of the sha256 of the image #1060
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
On every commit pushed to `main`, we were making a docker tag with the sha256 of the image. This causes the AWS stage deploy to be deployed to on every commit, which was not what we wanted (dev was the place for that).
jbuck
approved these changes
Nov 8, 2024
jmhodges
added a commit
that referenced
this pull request
Nov 8, 2024
So, since #1060, we're not able to do deploys. They error like as in: https://github.com/mozilla-services/autograph/actions/runs/11748424011/job/32732461427 I thought the `sha` I was removing was the hash of the image, but it's the git commit's sha! The real problem seems to be we are pushing to both dockerhub and to Google Artifact Registry (GAR) at the same time after our builds on `main`. We wanted only to push to GAR and then on release copy them to dockerhub. That happens becuase the `docker/metadata-action` action gets called with both Docker Hub and GAR in its `images` argument. So, to fix this, I thought really hard about what we want to happen. We currently have two separate deploy systems. In one, we need to push non-`latest` tags to Docker Hub only when we want a staging deploy to occur. In the other, we need to push to a semver-like tag to Google Artifact Registry only when we want a staging release to occur. Production deploys in both require a human to intervene and change the expected deployed version. In the deploy system that uses GAR, we need a push to `latest` on each commit to effect a dev deploy. We also use the `latest` on Docker Hub to represent the current `main` branch's code for use in other people's projects. So, let's sum that up in terms of tags. On each commit to `main`, we want these docker tags and only these docker tags pushed up: * `:latest` to `GAR` * `:sha-<the git commit's sha>` to GAR * `:latest` to Docker Hub On each release, we want these docker tags and only these docker tags pushed up: * `:<git tag of release>` to GAR * `:<git tag of release>` to Docker Hub Those `:<git tag of release>` will be copies of the `sha-<the git commit's sha>` tag that was made on commit to `main` and pushed to GAR. This patch does that by breaking up single `docker/metadata-action` action call to two. One for GAR and one for Docker Hub; both with different sets of tags depending on whether this is a "push" (commit on main) or "release" event. But doing just that doesn't quite solve it. Becuse we use `docker/build-push-action` to both build and push to GAR in one go. To avoid building on release, we have to skip that step, but that step is what was doing all the work of talking to GAR. So, we also need to, only on release, "copy" the previously made `sha-` tag to the `:<git tag of release>` tag within GAR itself. This may be the point where we should split this one GitHub Actions workflow into separate `deploy` and a `release` workflows. There is a lot of thinking and conditional checks. It would cost a significant amount of duplication between these two actions to do so. But I could be told to do so. (The crane install would be especially annoying; maybe we should just go install it, anyway, since this install instructions were for a non-Go environment)
jmhodges
added a commit
that referenced
this pull request
Nov 12, 2024
So, since #1060, we're not able to do deploys. They error like as in: https://github.com/mozilla-services/autograph/actions/runs/11748424011/job/32732461427 I thought the `sha` I was removing was the hash of the image, but it's the git commit's sha! The real problem seems to be we are pushing to both dockerhub and to Google Artifact Registry (GAR) at the same time after our builds on `main`. We wanted only to push to GAR and then on release copy them to dockerhub. That happens becuase the `docker/metadata-action` action gets called with both Docker Hub and GAR in its `images` argument. So, to fix this, I thought really hard about what we want to happen. We currently have two separate deploy systems. In one, we need to push non-`latest` tags to Docker Hub only when we want a staging deploy to occur. In the other, we need to push to a semver-like tag to Google Artifact Registry only when we want a staging release to occur. Production deploys in both require a human to intervene and change the expected deployed version. In the deploy system that uses GAR, we need a push to `latest` on each commit to effect a dev deploy. We also use the `latest` on Docker Hub to represent the current `main` branch's code for use in other people's projects. So, let's sum that up in terms of tags. On each commit to `main`, we want these docker tags and only these docker tags pushed up: * `:latest` to `GAR` * `:sha-<the git commit's sha>` to GAR * `:latest` to Docker Hub On each release, we want these docker tags and only these docker tags pushed up: * `:<git tag of release>` to GAR * `:<git tag of release>` to Docker Hub Those `:<git tag of release>` will be copies of the `sha-<the git commit's sha>` tag that was made on commit to `main` and pushed to GAR. This patch does that by breaking up single `docker/metadata-action` action call to two. One for GAR and one for Docker Hub; both with different sets of tags depending on whether this is a "push" (commit on main) or "release" event. But doing just that doesn't quite solve it. Becuse we use `docker/build-push-action` to both build and push to GAR in one go. To avoid building on release, we have to skip that step, but that step is what was doing all the work of talking to GAR. So, we also need to, only on release, "copy" the previously made `sha-` tag to the `:<git tag of release>` tag within GAR itself. This may be the point where we should split this one GitHub Actions workflow into separate `deploy` and a `release` workflows. There is a lot of thinking and conditional checks. It would cost a significant amount of duplication between these two actions to do so. But I could be told to do so. (The crane install would be especially annoying; maybe we should just go install it, anyway, since this install instructions were for a non-Go environment)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
On every commit pushed to
main, we were making a docker tag with thesha256 of the image.
This causes the AWS stage deploy to be deployed to on every commit,
which was not what we wanted (dev was the place for that).