-
Notifications
You must be signed in to change notification settings - Fork 287
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run ARM64 tests in CI #94
Merged
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -8,13 +8,11 @@ on: | |
|
||
jobs: | ||
lint: | ||
name: Lint | ||
name: Lint (${{ matrix.target_arch }}) | ||
runs-on: ubuntu-24.04 | ||
strategy: | ||
fail-fast: true | ||
max-parallel: 2 | ||
matrix: | ||
go: ["stable"] | ||
target_arch: [amd64, arm64] | ||
steps: | ||
- name: Clone code | ||
uses: actions/checkout@v4 | ||
|
@@ -26,79 +24,94 @@ jobs: | |
make lint | ||
|
||
test: | ||
name: Test | ||
name: Test (${{ matrix.target_arch }}) | ||
runs-on: ubuntu-24.04 | ||
strategy: | ||
fail-fast: true | ||
max-parallel: 2 | ||
matrix: | ||
go: ["stable"] | ||
target_arch: [amd64, arm64] | ||
steps: | ||
- name: Clone code | ||
uses: actions/checkout@v4 | ||
- name: Set up environment | ||
uses: ./.github/workflows/env | ||
- name: Build | ||
run: | | ||
echo $PATH | ||
make test | ||
- name: Tests | ||
run: | | ||
make test | ||
run: make test TARGET_ARCH=${{ matrix.target_arch }} | ||
|
||
build-integration-test-binaries: | ||
name: Build integration test binaries | ||
name: Build integration test binaries (${{ matrix.target_arch }}) | ||
runs-on: ubuntu-24.04 | ||
timeout-minutes: 10 | ||
strategy: | ||
matrix: | ||
target_arch: [amd64, arm64] | ||
steps: | ||
- name: Clone code | ||
uses: actions/checkout@v4 | ||
- name: Set up environment | ||
uses: ./.github/workflows/env | ||
- name: Prepare integration test binaries for qemu tests | ||
run: | | ||
make integration-test-binaries | ||
run: make integration-test-binaries TARGET_ARCH=${{ matrix.target_arch }} | ||
- name: Upload integration test binaries | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: integration-test-binaries | ||
name: integration-test-binaries-${{ matrix.target_arch }} | ||
path: support/*.test | ||
|
||
integration-tests: | ||
name: Integration tests (kernel v${{ matrix.kernel-version }}) | ||
name: Integration tests (v${{ matrix.kernel }} ${{ matrix.target_arch }}) | ||
runs-on: ubuntu-24.04 | ||
needs: build-integration-test-binaries | ||
timeout-minutes: 10 | ||
strategy: | ||
matrix: | ||
kernel-version: | ||
- "4.19.314" | ||
- "5.4.276" | ||
- "5.10.217" | ||
- "5.15.159" | ||
- "6.1.91" | ||
- "6.6.31" | ||
- "6.8.10" | ||
- "6.9.1" | ||
include: | ||
# List of available kernels here: | ||
# https://github.com/cilium/ci-kernels/pkgs/container/ci-kernels/versions?filters%5Bversion_type%5D=tagged | ||
|
||
# AMD64 | ||
- { target_arch: amd64, kernel: 4.19.314 } | ||
- { target_arch: amd64, kernel: 5.4.276 } | ||
- { target_arch: amd64, kernel: 5.10.217 } | ||
- { target_arch: amd64, kernel: 5.15.159 } | ||
- { target_arch: amd64, kernel: 6.1.91 } | ||
- { target_arch: amd64, kernel: 6.6.31 } | ||
- { target_arch: amd64, kernel: 6.8.10 } | ||
- { target_arch: amd64, kernel: 6.9.1 } | ||
|
||
# ARM64 (NOTE: older ARM64 kernels are not available in Cilium repos) | ||
- { target_arch: arm64, kernel: 6.6.31 } | ||
- { target_arch: arm64, kernel: 6.8.4 } | ||
- { target_arch: arm64, kernel: 6.9.1 } | ||
steps: | ||
- name: Clone code | ||
uses: actions/checkout@v4 | ||
- name: Set up environment | ||
uses: ./.github/workflows/env | ||
- name: Install dependencies | ||
run: | | ||
sudo apt-get -y install qemu-system-x86 | ||
case "${{ matrix.target_arch }}" in | ||
amd64) sudo apt-get -y install qemu-system-x86;; | ||
arm64) sudo apt-get -y install qemu-system-arm;; | ||
*) echo >&2 "bug: bad arch selected"; exit 1;; | ||
esac | ||
go install github.com/florianl/[email protected] | ||
sudo mv ~/go/bin/bluebox /usr/local/bin/. | ||
- name: Fetch integration test binaries | ||
uses: actions/download-artifact@v4 | ||
with: { name: integration-test-binaries } | ||
- name: Fetch precompiled kernels | ||
with: { name: "integration-test-binaries-${{ matrix.target_arch }}" } | ||
- name: Fetch precompiled kernel | ||
run: | | ||
install -d ci-kernels | ||
echo "FROM ghcr.io/cilium/ci-kernels:${{ matrix.kernel-version }}" \ | ||
| docker buildx build --quiet --pull --output="ci-kernels" - | ||
mv ci-kernels/boot/ ci-kernels/${{ matrix.kernel-version }}/ | ||
- name: Test on kernel ${{ matrix.kernel-version }} | ||
echo "FROM ghcr.io/cilium/ci-kernels:${{ matrix.kernel }}" \ | ||
| docker buildx build --platform linux/${{ matrix.target_arch }} \ | ||
--quiet --pull --output="ci-kernels" - | ||
mv ci-kernels/boot/ ci-kernels/${{ matrix.kernel }}/ | ||
- name: Test on kernel ${{ matrix.kernel }} | ||
run: | | ||
chmod a+rx *.test | ||
support/run-tests.sh ${{ matrix.kernel-version }} | ||
case "${{ matrix.target_arch }}" in | ||
amd64) export QEMU_ARCH=x86_64;; | ||
arm64) export QEMU_ARCH=aarch64;; | ||
*) echo >&2 "bug: bad arch selected"; exit 1;; | ||
esac | ||
support/run-tests.sh ${{ matrix.kernel }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,11 @@ | ||
#!/usr/bin/env bash | ||
# Test the current package under a different kernel. | ||
# Requires qemu-system-x86_64 and bluebox to be installed. | ||
# Requires qemu-system-$QEMU_ARCH and bluebox to be installed. | ||
|
||
set -eu | ||
set -o pipefail | ||
|
||
qemu_arch="${QEMU_ARCH:-x86_64}" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Alternatively we could make this a positional argument to There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Env variable seems good to me. |
||
color_green=$'\033[32m' | ||
color_red=$'\033[31m' | ||
color_default=$'\033[39m' | ||
|
@@ -40,17 +41,31 @@ done < <(find . -name '*.test' -print0) | |
|
||
additionalQemuArgs="" | ||
|
||
supportKVM=$(grep -c -E 'vmx|svm' /proc/cpuinfo || echo "0") | ||
if [ "$supportKVM" -ne 0 ]; then | ||
supportKVM=$(grep -E 'vmx|svm' /proc/cpuinfo || true) | ||
if [ ! "$supportKVM" ] && [ "$qemu_arch" = "$(uname -m)" ]; then | ||
additionalQemuArgs="-enable-kvm" | ||
fi | ||
|
||
case "$qemu_arch" in | ||
x86_64) | ||
additionalQemuArgs+=" -append console=ttyS0" | ||
bb_args+=(-a amd64) | ||
;; | ||
aarch64) | ||
additionalQemuArgs+=" -machine virt -cpu max" | ||
bb_args+=(-a arm64) | ||
;; | ||
esac | ||
|
||
if [ "$qemu_arch" = "aarch64" ]; then | ||
additionalQemuArgs+=" -machine virt -cpu max" | ||
fi | ||
|
||
bluebox "${bb_args[@]}" || (echo "failed to generate initramfs"; exit 1) | ||
|
||
echo Testing on "${kernel_version}" | ||
$sudo qemu-system-x86_64 ${additionalQemuArgs} \ | ||
$sudo qemu-system-${qemu_arch} ${additionalQemuArgs} \ | ||
-nographic \ | ||
-append "console=ttyS0" \ | ||
-monitor none \ | ||
-serial file:"${output}/test.log" \ | ||
-no-user-config \ | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: Using
stable
as version for Go comes with the risk of breaking CI if there is an unexpected change in Go. E.g. there could be changes similar to golang/go#67401, that could break assumptions.Manually updating the Go version allows to make sure CI does not break on events, that are happening outside of the scope and control of this repo/maintainership.
Ideally, this action should receive some version information as argument. Using
$matrix.go
here assumes that in the executed jobmatrix.go
is defined and makes it hard to follow/review.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm merely mirroring the previous logic here, but I'd also be OK with hardcoding a particular Go version globally. IMO we should just have one global Go version for all CI workflows and adding it as an argument would force us to duplicate the version everywhere.
Yeah, that's why I opted to just hardcode it directly in the action. Also, all callers either passed "stable" already or forgot to set it entirely.