From dbd1f0f70931f9f5a8e592598e9b83e1ad0da90e Mon Sep 17 00:00:00 2001 From: takahiro-yamada Date: Thu, 17 Oct 2024 05:46:18 +0000 Subject: [PATCH 1/9] Standardize markdown styles --- CHANGELOG.md | 5 ++-- README.md | 1 + RELEASE.md | 53 ++++++++++++++++---------------- docs/commands.md | 16 ++++------ docs/design.md | 10 +++---- docs/development.md | 47 ++++++++++++++--------------- docs/metrics.md | 3 +- docs/runner-pod-api.md | 46 ++++++++++++++-------------- docs/user-manual.md | 68 +++++++++++++++++++++--------------------- 9 files changed, 120 insertions(+), 129 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0afee094..800db3d0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,8 +25,8 @@ This project adheres to [Semantic Versioning](http://semver.org/). We migrated the image repositories of meows to `ghcr.io`. From meows v0.14.0, please use the following images. -- https://github.com/cybozu-go/meows/pkgs/container/meows-controller -- https://github.com/cybozu-go/meows/pkgs/container/meows-runner +- +- The images on Quay.io ([meows-controller](https://quay.io/repository/cybozu/meows-controller), [meows-runner](https://quay.io/repository/cybozu/meows-runner)) will not be updated in the future. @@ -41,7 +41,6 @@ The images on Quay.io ([meows-controller](https://quay.io/repository/cybozu/meow - Support Kubernetes 1.27 ([#178](https://github.com/cybozu-go/meows/pull/1781)) - Build with go 1.21 ([#178](https://github.com/cybozu-go/meows/pull/178)) - ## [0.12.0] - 2023-07-05 ### Changed diff --git a/README.md b/README.md index aedf9b37..aa71b9e0 100644 --- a/README.md +++ b/README.md @@ -42,6 +42,7 @@ You can run jobs in your GitHub Actions workflows on your Kubernetes cluster wit ## Docker images Docker images are available on [ghcr.io](https://github.com/orgs/cybozu-go/packages?repo_name=meows) + - [Controller](https://github.com/cybozu-go/meows/pkgs/container/meows-controller) - [Runner](https://github.com/cybozu-go/meows/pkgs/container/meows-runner) diff --git a/RELEASE.md b/RELEASE.md index f2111656..c6e02bcc 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,15 +1,12 @@ -Release procedure -================= +# Release procedure This document describes how to release a new version of meows. -Versioning ----------- +## Versioning Follow [semantic versioning 2.0.0][semver] to choose the new version number. -Prepare change log entries --------------------------- +## Prepare change log entries Add notable changes since the last release to [CHANGELOG.md](CHANGELOG.md). It should look like: @@ -19,65 +16,67 @@ It should look like: ## [Unreleased] ### Added + - Implement ... (#35) ### Changed + - Fix a bug in ... (#33) ### Removed + - Deprecated `-option` is removed ... (#39) (snip) ``` -Bump version ------------- +## Bump version 1. Determine a new version number. Then set `VERSION` variable. - ```console + ```bash # Set VERSION and confirm it. It should not have "v" prefix. - $ VERSION=x.y.z - $ echo $VERSION + VERSION=x.y.z + echo $VERSION ``` 2. Make a branch to release - ```console - $ git neco dev "bump-$VERSION" + ```bash + git neco dev "bump-$VERSION" ``` 3. Edit `CHANGELOG.md` for the new version ([example][]). 4. Bump image version. - ```console - $ sed -i -E "s/(.*newTag: ).*/\1${VERSION}/" config/controller/kustomization.yaml config/agent/kustomization.yaml - $ sed -i -E "s/(.*Version = ).*/\1\"${VERSION}\"/" constants.go + ```bash + sed -i -E "s/(.*newTag: ).*/\1${VERSION}/" config/controller/kustomization.yaml config/agent/kustomization.yaml + sed -i -E "s/(.*Version = ).*/\1\"${VERSION}\"/" constants.go ``` 5. Commit the change and push it. - ```console - $ git commit -a -m "Bump version to $VERSION" - $ git neco review + ```bash + git commit -a -m "Bump version to $VERSION" + git neco review ``` 6. Merge this branch. 7. Add a git tag to the main HEAD, then push it. - ```console + ```bash # Set VERSION again. - $ VERSION=x.y.z - $ echo $VERSION + VERSION=x.y.z + echo $VERSION - $ git checkout main - $ git pull - $ git tag -a -m "Release v$VERSION" "v$VERSION" + git checkout main + git pull + git tag -a -m "Release v$VERSION" "v$VERSION" # Make sure the release tag exists. - $ git tag -ln | grep $VERSION + git tag -ln | grep $VERSION - $ git push origin "v$VERSION" + git push origin "v$VERSION" ``` GitHub actions will build and push artifacts such as container images and diff --git a/docs/commands.md b/docs/commands.md index 0e8a13ac..353e53a6 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -1,12 +1,10 @@ -CLI options -=========== +# CLI options -`controller` ------------ +## `controller` The CLI allows you to use the following options: -```bash +```console $ controller -h Kubernetes controller for GitHub Actions self-hosted runner @@ -41,9 +39,7 @@ Flags: --zap-stacktrace-level level Zap Level at and above which stacktraces are captured (one of 'info', 'error', 'panic'). ``` - -`slack-agent` -------------- +## `slack-agent` The Slack agent is a server program. This notifies CI results and accepts requests for extending Pods' lifecycles @@ -68,9 +64,7 @@ Flags: -v, --verbose Verbose. ``` - -`meows` ------- +## `meows` This is a tool command to do some operations. It enables to send requests to the slack-agent, or to control the GitHub runners. diff --git a/docs/design.md b/docs/design.md index 7480684b..846c9475 100644 --- a/docs/design.md +++ b/docs/design.md @@ -135,7 +135,7 @@ Runner has the `status` and `busy` state as written [here](https://docs.github.c If the `--ephemeral` option is given to `config.sh` does not repeat the long polling again, and never gets `online` after the assigned job is done. This behavior is useful for ensuring to make a clean environment for each job. -ref: https://docs.github.com/en/actions/hosting-your-own-runners/autoscaling-with-self-hosted-runners#using-ephemeral-runners-for-autoscaling +ref: #### A job is scheduled only on a `online` runner @@ -189,7 +189,7 @@ meows sets the namespaced name of a `RunnerPool` as a custom label. command when the job is failed. The `if: failure()` syntax allows users to run the step only when one of previous steps exit with non-zero code. 1. Publish the timestamp of when to delete this pod in the `/deletion_time` endpoint. - If the job is succeeded or canceled, the `Pod` publishes the current time for + If the job is succeeded or canceled, the `Pod` publishes the current time for delete itself. If the job is failed, the `Pod` publishes the future time for delete itself, for example 20 min later. 1. The Slack agent notifies the result of the job on a Slack channel. @@ -208,15 +208,15 @@ A Runner `Pod` has the following state as a GitHub Actions job runner. for example, booting a couple of VMs needed in a job before the job is assigned. - `running`: `Pod` is running. Registered in GitHub Actions. - `debugging`: The job has finished with failure and Users can enter `Pod` to debug. -- `stale`: The environment in the `Pod` is dirty. If a runner restarts before completing a job, +- `stale`: The environment in the `Pod` is dirty. If a runner restarts before completing a job, the environment in the `Pod` may be dirty. This state means waiting for the Pod to be removed to prevent Job execution with that stale Pod. In addition, it has the following states as the exit state of the execution result of `Runner.Listener`. - `retryable_error`: If execution fails due to a factor other than a job, restart `Runner.Listener`. -- `updating`: When a new `Runner.Listener` is released, it updates itself and restarts` Runner.Listener`. -- `undefined`: When the exit code of `Runner.Listener` is undefined. It restarts` Runner.Listener`. +- `updating`: When a new `Runner.Listener` is released, it updates itself and restarts `Runner.Listener`. +- `undefined`: When the exit code of `Runner.Listener` is undefined. It restarts `Runner.Listener`. The above states are exposed from `/metrics` endpoint as Prometheus metrics. See [metrics.md](metrics.md). diff --git a/docs/development.md b/docs/development.md index 2f6478e1..12529b44 100644 --- a/docs/development.md +++ b/docs/development.md @@ -1,8 +1,6 @@ -Development guide -================= +# Development guide -Testing -------- +## Testing There are 2 kinds of test included in this repository. @@ -18,7 +16,7 @@ difficult to test in some parts and some parts of the code are not tested intent What kindtest covers is: - Runner `Pod`s are registered to GitHub Actions on a test repository. - - At present, the test repository is a fixed one (`github.com/neco-test/meows-ci`). + - At present, the test repository is a fixed one (`github.com/neco-test/meows-ci`). - GitHub Actions workflows run on the `Pod`s. - Runner `Pod`s send messages to Slack agent. - Slack agent sends messages to Slack. @@ -41,7 +39,8 @@ In order to run the kindtest, you need to prepare as follows. You can run the kindtest as following. 1. Create secret files for kindtest. - ```bash + + ```console $ vi .secret.private-key.pem # Save your GitHub App private key in this file. @@ -56,40 +55,41 @@ You can run the kindtest as following. ``` 2. Install tools. + ```bash - $ make setup + make setup ``` 3. Run kindtest. + ```bash # Start kind cluster. - $ make -C kindtest start + make -C kindtest start # Run test on kind. - $ make -C kindtest test + make -C kindtest test # Stop kind cluster. - $ make -C kindtest stop + make -C kindtest stop ``` - ### Run slack agent manually Then, run a server with the following commands: ```bash # Run server process -$ export SLACK_CHANNEL=# -$ export SLACK_APP_TOKEN= -$ export SLACK_BOT_TOKEN= -$ go run ./cmd/slack-agent -d +export SLACK_CHANNEL=# +export SLACK_APP_TOKEN= +export SLACK_BOT_TOKEN= +go run ./cmd/slack-agent -d ``` You can test both the failure and success messages by actually sending them: ```bash # client -$ cat < /tmp/github.env +cat < /tmp/github.env { "actor": "user", "git_ref": "branch-name", @@ -103,17 +103,16 @@ $ cat < /tmp/github.env EOF # success -$ go run ./cmd/meows slackagent send pod success -f /tmp/github.env +go run ./cmd/meows slackagent send pod success -f /tmp/github.env # failure -$ go run ./cmd/meows slackagent send pod failure --extend -f /tmp/github.env +go run ./cmd/meows slackagent send pod failure --extend -f /tmp/github.env ``` Then, click the button on the Slack message, and check if a receiving log appears on the terminal. -How to run meows for development ----------------------------------------------------- +## How to run meows for development If you need to run the controller on your local environment, this is the easiest way to do that. You can reuse the token for the test repository, which is prepared for CI. @@ -121,9 +120,9 @@ But please be careful that your local environment steals the job that is expecte ```bash # Create secret files for kindtest. -$ vi .secret.private-key.pem -$ vi .secret.env.sh +vi .secret.private-key.pem +vi .secret.env.sh -$ make -C kindtest start -$ make -C kindtest bootstrap +make -C kindtest start +make -C kindtest bootstrap ``` diff --git a/docs/metrics.md b/docs/metrics.md index 26c5be62..8e1bbea3 100644 --- a/docs/metrics.md +++ b/docs/metrics.md @@ -1,5 +1,4 @@ -Metrics -=========== +# Metrics ## Controller diff --git a/docs/runner-pod-api.md b/docs/runner-pod-api.md index 52f4125d..6c04c16a 100644 --- a/docs/runner-pod-api.md +++ b/docs/runner-pod-api.md @@ -1,8 +1,8 @@ -Runner Pod API -============== +# Runner Pod API -- [`PUT /deletion_time`](#put-deletion_time) -- [`GET /status`](#get-status) +- [Runner Pod API](#runner-pod-api) + - [`PUT /deletion_time`](#put-deletion_time) + - [`GET /status`](#get-status) ## `PUT /deletion_time` @@ -14,7 +14,7 @@ This API updates a pod's deletion time. The time format is RFC 3339 in UTC. **Failure responses** -- If the request body is invalid +- If the request body is invalid HTTP status code: 400 Bad Request - If `Content-Type` is not `application/json` HTTP status code: 415 Unsupported Media Type @@ -22,7 +22,7 @@ This API updates a pod's deletion time. The time format is RFC 3339 in UTC. ```console curl -s -XPUT localhost:8080/deletion_time -H "Content-Type: application/json" -d ' { - "deletion_time": "0001-01-01T00:00:00Z" + "deletion_time": "0001-01-01T00:00:00Z" }' ``` @@ -42,32 +42,32 @@ When the pod state is `debugging` (i.e. the pod is finished), it returns a json **Failure responses** - If it fails to get the job information -HTTP status code: 500 Internal Server Error + HTTP status code: 500 Internal Server Error ```console $ # When the pod state is `initializing`, `running` or `stale`: $ curl -s -XGET localhost:8080/status { - "state": "initializing" ... "initializing", "running" or "stale" + "state": "initializing" ... "initializing", "running" or "stale" } $ # When the pod state is `debugging`: $ curl -s -XGET localhost:8080/status { - "state": "debugging", - "result": "failure", ... Job result. "success", "failure, "cancelled" or "unknown". - "finished_at": "2021-01-01T00:00:00Z", ... The time the job was finished. - "deletion_time": "2021-01-01T00:20:00Z", ... Scheduled deletion time. This field remains nil until `PUT /deletion_time` is called. - "extend": true, ... Pod extension is required or not. - "job_info": { - "actor": "user", - "git_ref": "branch/name", - "job_id": "job", - "repository": "owner/repo", - "run_id": 123456789, - "run_number": 987, - "workflow_name": "Work flow" - }, - "slack_channel": "" ... May be blank. The name of the Slack channel specified in the workflow. + "state": "debugging", + "result": "failure", ... Job result. "success", "failure, "cancelled" or "unknown". + "finished_at": "2021-01-01T00:00:00Z", ... The time the job was finished. + "deletion_time": "2021-01-01T00:20:00Z", ... Scheduled deletion time. This field remains nil until `PUT /deletion_time` is called. + "extend": true, ... Pod extension is required or not. + "job_info": { + "actor": "user", + "git_ref": "branch/name", + "job_id": "job", + "repository": "owner/repo", + "run_id": 123456789, + "run_number": 987, + "workflow_name": "Work flow" + }, + "slack_channel": "" ... May be blank. The name of the Slack channel specified in the workflow. } ``` diff --git a/docs/user-manual.md b/docs/user-manual.md index e018c228..a46fb421 100644 --- a/docs/user-manual.md +++ b/docs/user-manual.md @@ -7,15 +7,15 @@ meows depends on the [cert-manager](https://cert-manager.io/docs/). If you are not installing the cert-manager on your Kubernetes cluster, install it as follows: ```bash -$ curl -fsLO https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml -$ kubectl apply -f cert-manager.yaml +curl -fsLO https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml +kubectl apply -f cert-manager.yaml ``` You need to manually create a secret and a configmap in the `meows` namespace at the initial deployment. So make the `meows` namespace to prepare. ```bash -$ kubectl create namespace meows +kubectl create namespace meows ``` ### Create Controller Option ConfigMap (Optional) @@ -24,9 +24,9 @@ You can restrict the organization and repository that meows operates by `meows-c If you want to restrict it in some way, please create a ConfigMap as follows. ```bash -$ kubectl create configmap meows-cm -n meows \ - --from-literal=organization-rule='^neco-test$' \ - --from-literal=repository-rule='^neco-test/.*' +kubectl create configmap meows-cm -n meows \ + --from-literal=organization-rule='^neco-test$' \ + --from-literal=repository-rule='^neco-test/.*' ``` Both `organization-rule` and `repository-rule` accepts golang's regular expressions. @@ -36,8 +36,8 @@ Both `organization-rule` and `repository-rule` accepts golang's regular expressi Deploy the controller as follows. ```bash -$ MEOWS_VERSION=$(curl -s https://api.github.com/repos/cybozu-go/meows/releases/latest | jq -r .tag_name) -$ kustomize build github.com/cybozu-go/meows/config/controller?ref=${MEOWS_VERSION} | kubectl apply -f - +MEOWS_VERSION=$(curl -s https://api.github.com/repos/cybozu-go/meows/releases/latest | jq -r .tag_name) +kustomize build github.com/cybozu-go/meows/config/controller?ref=${MEOWS_VERSION} | kubectl apply -f - ``` ### Deploying Slack Agent (Optional) @@ -48,21 +48,21 @@ The agent requires Slack App tokens, so create a Slack App following [Creating S And create a secret as follows: ```bash -$ SLACK_CHANNEL="#" -$ SLACK_APP_TOKEN= -$ SLACK_BOT_TOKEN= - -$ kubectl create secret generic slack-app-secret -n meows \ - --from-literal=SLACK_CHANNEL=${SLACK_CHANNEL} \ - --from-literal=SLACK_APP_TOKEN=${SLACK_APP_TOKEN} \ - --from-literal=SLACK_BOT_TOKEN=${SLACK_BOT_TOKEN} +SLACK_CHANNEL="#" +SLACK_APP_TOKEN= +SLACK_BOT_TOKEN= + +kubectl create secret generic slack-app-secret -n meows \ + --from-literal=SLACK_CHANNEL=${SLACK_CHANNEL} \ + --from-literal=SLACK_APP_TOKEN=${SLACK_APP_TOKEN} \ + --from-literal=SLACK_BOT_TOKEN=${SLACK_BOT_TOKEN} ``` After that deploy the agent. ```bash -$ MEOWS_VERSION=$(curl -s https://api.github.com/repos/cybozu-go/meows/releases/latest | jq -r .tag_name) -$ kustomize build github.com/cybozu-go/meows/config/controller?ref=${MEOWS_VERSION} | kubectl apply -f - +MEOWS_VERSION=$(curl -s https://api.github.com/repos/cybozu-go/meows/releases/latest | jq -r .tag_name) +kustomize build github.com/cybozu-go/meows/config/controller?ref=${MEOWS_VERSION} | kubectl apply -f - ``` ## Creating RunnerPool @@ -77,8 +77,8 @@ In other words, you need to create a Secret that records the credential in the R If you have not created a RunnerPool's namespace yet, please create it as follows. ```bash -$ RUNNERPOOL_NAMESPACE= -$ kubectl create namespace ${RUNNERPOOL_NAMESPACE} +RUNNERPOOL_NAMESPACE= +kubectl create namespace ${RUNNERPOOL_NAMESPACE} ``` ### Creating GitHub Credential Secret @@ -94,31 +94,32 @@ If you want to use a GitHub App, create a GitHub App and download a private key And create a secret as follows: ```bash -$ RUNNERPOOL_NAMESPACE= -$ GITHUB_APP_ID= -$ GITHUB_APP_INSTALLATION_ID= -$ GITHUB_APP_PRIVATE_KEY_PATH= - -$ kubectl create secret generic meows-github-cred -n ${RUNNERPOOL_NAMESPACE} \ - --from-literal=app-id=${GITHUB_APP_ID} \ - --from-literal=app-installation-id=${GITHUB_APP_INSTALLATION_ID} \ - --from-file=app-private-key=${GITHUB_APP_PRIVATE_KEY_PATH} +RUNNERPOOL_NAMESPACE= +GITHUB_APP_ID= +GITHUB_APP_INSTALLATION_ID= +GITHUB_APP_PRIVATE_KEY_PATH= + +kubectl create secret generic meows-github-cred -n ${RUNNERPOOL_NAMESPACE} \ + --from-literal=app-id=${GITHUB_APP_ID} \ + --from-literal=app-installation-id=${GITHUB_APP_INSTALLATION_ID} \ + --from-file=app-private-key=${GITHUB_APP_PRIVATE_KEY_PATH} ``` If you want to use a Personal Access Token (PAT), create a PAT following [the official documentation](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token). Then: + - Set the `repo` scope, if you want to use a repository-level runner. - Set the `admin:org` scope, if you want to use an organization-level runner. And create a secret as follows: ```bash -$ RUNNERPOOL_NAMESPACE= -$ GITHUB_TOKEN= +RUNNERPOOL_NAMESPACE= +GITHUB_TOKEN= -$ kubectl create secret generic meows-github-cred -n ${RUNNERPOOL_NAMESPACE} \ - --from-literal=token=${GITHUB_TOKEN} +kubectl create secret generic meows-github-cred -n ${RUNNERPOOL_NAMESPACE} \ + --from-literal=token=${GITHUB_TOKEN} ``` NOTE: The meows controller loads the credential when the controller reconcile the RunnerPool creation or when the controller starts. @@ -208,7 +209,6 @@ spec: extendDuration: "30s" # If you want to extend the Pod in case of job failure, set this field. ``` - ```yaml name: slack notification example on: push From 5898dee890a3c68dd394569442d8b3ad53b278e3 Mon Sep 17 00:00:00 2001 From: takahiro-yamada Date: Thu, 17 Oct 2024 06:12:45 +0000 Subject: [PATCH 2/9] Replace `git neco` --- RELEASE.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/RELEASE.md b/RELEASE.md index c6e02bcc..a7d365ca 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -43,7 +43,7 @@ It should look like: 2. Make a branch to release ```bash - git neco dev "bump-$VERSION" + git switch -c "bump-$VERSION" ``` 3. Edit `CHANGELOG.md` for the new version ([example][]). @@ -58,7 +58,7 @@ It should look like: ```bash git commit -a -m "Bump version to $VERSION" - git neco review + git push origin "bump-$VERSION" ``` 6. Merge this branch. From 8405942e3480572a4343e37bf0d063131d86b5e2 Mon Sep 17 00:00:00 2001 From: takahiro-yamada Date: Thu, 17 Oct 2024 10:30:59 +0000 Subject: [PATCH 3/9] Add the maintenance procedure --- docs/maintenance.md | 49 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 docs/maintenance.md diff --git a/docs/maintenance.md b/docs/maintenance.md new file mode 100644 index 00000000..c06136b8 --- /dev/null +++ b/docs/maintenance.md @@ -0,0 +1,49 @@ +# Maintenance + +## How to update supported Kubernetes + +Meows supports the three latest Kubernetes versions. +If a new Kubernetes version is released, please update the followings: + +### 1. Update supported kubernetes and dependencies versions + +- Kubernetes versions: You can check the versions at . + - `k8s-version` in [.github/workflows/main.yaml](/.github/workflows/main.yaml) + - "Supported software" in [README.md](/README.md) +- Tools versions: + - Update `CONTROLLER_GEN_VERSION` in [Makefile](/Makefile) to the latest version from . + - Update `RUNNER_VERSION` in [Dockerfile](/Dockerfile) to the latest version from . + - In [kindtest/Makefile](/kindtest/Makefile): + - Update `KINDTEST_K8S_VERSION` to the latest supported version of kubernetes. + - Update `KUSTOMIZE_VERSION` to the latest version from . + - Update `KIND_VERSION` to the latest version from . + - Update `CERT_MANAGER_VERSION` to the latest version from . +- After saving the changes above, update `ENVTEST_K8S_VERSION` in [Makefile](/Makefile) to the latest patch version among the latest supported kubernetes minor versions listed by running `make setup && tmp/bin/setup-envtest list` at the root of this repository. If the latest minor supported version is `1.30.Z`, find `1.30.Z+` from the output but not `1.31.Z`. +- Other dependencies versions: + - Update `ghcr.io/cybozu/golang` image in [Dockerfile](/Dockerfile) to the latest version from . +- `go.mod` and `go.sum`: + - Run `go get -u ./...`. + +If Kubernetes or controller-runtime API has changed, please update the relevant source code accordingly. + +### 2. Update meows by running `make` + +You can update meows by running the following `make` commands: + +```sh +make setup +make manifests +make build +``` + +### 3. Fix test code if tests fail + +After pushing the change, if the CI fails, fix the tests and push the changes again. + +_e.g._, + +### 4. Release the new version + +After merging the changes above, follow the procedures written in [Release.md](/RELEASE.md) and release the new version. + +_e.g._, From 23b908533a979d4110854debc57d4cb01669d4ae Mon Sep 17 00:00:00 2001 From: takahiro-yamada Date: Fri, 18 Oct 2024 02:12:55 +0000 Subject: [PATCH 4/9] Add --no-install-recommends --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index 89fb06d9..5686443b 100644 --- a/Dockerfile +++ b/Dockerfile @@ -23,7 +23,7 @@ ARG RUNNER_VERSION=2.319.1 ENV DEBIAN_FRONTEND=noninteractive RUN apt-get update -y \ - && apt-get install -y software-properties-common \ + && apt-get install -y --no-install-recommends software-properties-common \ && add-apt-repository -y ppa:git-core/ppa \ && apt-get update -y \ && apt-get install -y --no-install-recommends libyaml-dev \ From a12571bba1465d464c042f5e1095430a82f2a2f2 Mon Sep 17 00:00:00 2001 From: takahiro-yamada Date: Fri, 18 Oct 2024 02:21:52 +0000 Subject: [PATCH 5/9] Add debug commands for GHA --- .github/workflows/main.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.github/workflows/main.yaml b/.github/workflows/main.yaml index 90436b0a..1f3a66e1 100644 --- a/.github/workflows/main.yaml +++ b/.github/workflows/main.yaml @@ -56,6 +56,14 @@ jobs: GIT_SSH_COMMAND: "ssh -i /tmp/deploy-key.pem" - run: make -C kindtest logs if: always() + + - name: Check disk usage + run: df -h + if: always() + - name: Check directory sizes + run: du -sh /usr/local/* /opt/* /usr/* | sort -hr + if: always() + - uses: actions/upload-artifact@v4 if: always() with: From 4a0cbada2cd98237e0ea2c7530ea8e7458a028f4 Mon Sep 17 00:00:00 2001 From: takahiro-yamada Date: Fri, 18 Oct 2024 02:42:22 +0000 Subject: [PATCH 6/9] Fix Dockerfile --- Dockerfile | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/Dockerfile b/Dockerfile index 5686443b..de686a5b 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,10 +1,10 @@ -FROM ghcr.io/cybozu/golang:1.23-jammy as builder +FROM ghcr.io/cybozu/golang:1.23-jammy AS builder WORKDIR /workspace COPY . . RUN make build -FROM ghcr.io/cybozu/ubuntu:22.04 as controller +FROM ghcr.io/cybozu/ubuntu:22.04 AS controller LABEL org.opencontainers.image.source="https://github.com/cybozu-go/meows" COPY --from=builder /workspace/tmp/bin/controller /usr/local/bin @@ -14,7 +14,7 @@ COPY --from=builder /workspace/tmp/bin/meows /usr/local/bin USER 10000:10000 ENTRYPOINT ["controller"] -FROM ghcr.io/cybozu/ubuntu:22.04 as runner +FROM ghcr.io/cybozu/ubuntu:22.04 AS runner LABEL org.opencontainers.image.source="https://github.com/cybozu-go/meows" # Even if the version of the runner is out of date, it will self-update at job execution time. So there is no problem to update it when you notice. @@ -22,8 +22,9 @@ LABEL org.opencontainers.image.source="https://github.com/cybozu-go/meows" ARG RUNNER_VERSION=2.319.1 ENV DEBIAN_FRONTEND=noninteractive +# hadolint ignore=DL3015 RUN apt-get update -y \ - && apt-get install -y --no-install-recommends software-properties-common \ + && apt-get install -y software-properties-common \ && add-apt-repository -y ppa:git-core/ppa \ && apt-get update -y \ && apt-get install -y --no-install-recommends libyaml-dev \ From 08650cd150fb3db5f071d31088d6d3b3b4cfa93b Mon Sep 17 00:00:00 2001 From: takahiro-yamada Date: Fri, 18 Oct 2024 03:05:55 +0000 Subject: [PATCH 7/9] Add debug logs --- .github/workflows/main.yaml | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/.github/workflows/main.yaml b/.github/workflows/main.yaml index 1f3a66e1..94e2c905 100644 --- a/.github/workflows/main.yaml +++ b/.github/workflows/main.yaml @@ -57,11 +57,13 @@ jobs: - run: make -C kindtest logs if: always() - - name: Check disk usage - run: df -h + - name: Check disk usage and docker images + run: | + df -h + docker images if: always() - name: Check directory sizes - run: du -sh /usr/local/* /opt/* /usr/* | sort -hr + run: sudo du -sh /usr/local/* /home/* /opt/* /tmp/* /usr/* /var/* | sort -hr if: always() - uses: actions/upload-artifact@v4 From 9a3842895b5d11c7af427d294980e2855c48008d Mon Sep 17 00:00:00 2001 From: takahiro-yamada Date: Fri, 18 Oct 2024 08:20:57 +0000 Subject: [PATCH 8/9] Add clean up process --- .github/workflows/main.yaml | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/.github/workflows/main.yaml b/.github/workflows/main.yaml index 94e2c905..ffe4a100 100644 --- a/.github/workflows/main.yaml +++ b/.github/workflows/main.yaml @@ -49,6 +49,36 @@ jobs: export SLACK_APP_TOKEN=${{ secrets.SLACK_APP_TOKEN }} export SLACK_BOT_TOKEN=${{ secrets.SLACK_BOT_TOKEN }} EOF + - name: Clean up unnecessary files + run: | + docker image prune -af + sudo rm -rf /home/linuxbrew || true + sudo rm -rf /opt/az || true + sudo rm -rf /opt/microsoft || true + sudo rm -rf /opt/pipx || true + sudo rm -rf /opt/google || true + sudo rm -rf /usr/share/dotnet || true + sudo rm -rf /usr/local/lib/android || true + sudo rm -rf /usr/local/share/boost || true + sudo rm -rf /usr/lib/jvm || true + sudo rm -rf /usr/share/swift || true + sudo rm -rf /usr/local/julia* || true + sudo rm -rf /usr/local/n || true + sudo rm -rf /usr/share/kotlinc || true + sudo rm -rf /usr/local/share/edge_driver || true + sudo rm -rf /usr/local/share/chromedriver-linux64 || true + sudo rm -rf /usr/local/share/gecko_driver || true + sudo rm -rf /usr/share/miniconda || true + sudo rm -rf /usr/local/share/phantomjs* || true + sudo rm -rf /usr/share/sbt || true + sudo rm -rf /usr/local/aws-cli || true + sudo rm -rf /usr/local/aws-sam-cli || true + sudo rm -rf /usr/local/sqlpackage || true + sudo rm -rf /usr/local/bin/minikube || true + sudo rm -rf /usr/local/bin/kustomize || true + sudo rm -rf /usr/local/bin/kubectl || true + sudo rm -rf /usr/local/bin/kind || true + sudo rm -rf /usr/local/bin/helm || true - run: make setup KINDTEST_K8S_VERSION=${{ matrix.k8s-version }} - run: make -C kindtest start KINDTEST_K8S_VERSION=${{ matrix.k8s-version }} - run: make -C kindtest test From dcd0b74581e910493d9b643b47c952873555435d Mon Sep 17 00:00:00 2001 From: takahiro-yamada Date: Mon, 28 Oct 2024 07:30:33 +0000 Subject: [PATCH 9/9] Bump RUNNER_VERSION from 2.319.1 to 2.320.0 --- Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile b/Dockerfile index de686a5b..ef186708 100644 --- a/Dockerfile +++ b/Dockerfile @@ -19,7 +19,7 @@ LABEL org.opencontainers.image.source="https://github.com/cybozu-go/meows" # Even if the version of the runner is out of date, it will self-update at job execution time. So there is no problem to update it when you notice. # TODO: Until https://github.com/cybozu-go/meows/issues/137 is fixed, update it manually. -ARG RUNNER_VERSION=2.319.1 +ARG RUNNER_VERSION=2.320.0 ENV DEBIAN_FRONTEND=noninteractive # hadolint ignore=DL3015