Initial Support for bare-metal QEMU-TDX runtime deployment #611
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
msanft
added
feature
msanft
force-pushed
the
msanft/kata-qemu-tdx-runtime
branch
2 times, most recently
from
f69b435 to
5ad9592
Compare
msanft
changed the title
msanft
changed the title
msanft
force-pushed
the
msanft/kata-qemu-tdx-runtime
branch
from
5ad9592 to
d2c9eb8
Compare
|
PR is pretty big, can we split this into smaller chunks? |
msanft
force-pushed
the
msanft/kata-qemu-tdx-runtime
branch
from
d2c9eb8 to
60fb9ee
Compare
As mentioned in the PR description, I split this into digestible commits already. What benefit do you see in splitting this up into multiple PRs instead of reviewing each commit? I think the changes are quite entangled, so without a merge queue or set-merging, this would be hard to split into chunks that make sense on their own, but not break main. |
katexochen
requested changes
Jun 26, 2024
packages/by-name/kata/contrast-node-installer-image/package.nix
Outdated
Show resolved
Hide resolved
msanft
force-pushed
the
msanft/kata-qemu-tdx-runtime
branch
from
60fb9ee to
b9c452f
Compare
I think we should keep the pod-role annotation unique, as it is - besides the container image URL - the only way to see which installer is used. |
We can add another annotation with the string value of what is now called flavour instead. |
This migrates the runtime-class-files to the new directory structure, as they only contain Microsoft-specific components and thus should not live in the shared space.
This also pulls out the node installer image to the new directory structure, as it also only holds Microsoft-specific components. A future refactoring should make this share the lowest layer (i.e. the node installer binary itself) between flavour-specific node installers.
This packages the fileset required by the bare-metal kata runtime. For now, they are versioned in a makeshift content-addressable fashion, but not by the actual launch digest (MRTD), as we cannot calculate it ahead-of-time yet. This way, we can still deploy multiple runtimes side-by-side, each having a fixed, but not precalculatable launch digest.
This packages the Contrast node installer for bare-metal into a container image containing the node installer as well as the required runtime files. This does not implement the actual flavoring of the node installer yet, which is to be done in a follow-up commit.
This enables the runtime in the actual Contrast binary to be configurable by specifying it as a separate input, which can be set to either Kata (for bare-metal) or microsoft (for AKS).
This adds a package that contains some common tooling for working with the different deployment flavours of Contrast.
This changes the node-installer in a way so that it accepts a flavour as it's argument, specifying where it's deployed. Furthermore, it adds support for installing the bare-metal QEMU TDX runtime components of Contrast, which will be done if the corresponding flavour is specified.
This adds conditional generation of the Kubernetes resources (i.e. the Contrast runtime) to the resourcegen tool.
Since the main consumer of the flavours is the node-installer, the package now lives here. It also has the benefit of not needing to rebuild the node-installer whenever Contrast changes, but the other way around (which should be the case way less often).
This adds a container push instruction for the Kata node installer.
This changes the default snapshotter to virtio-9p, which proved to be more stable than virtio-fs in testing.
Building a statically linked QEMU is non-trivial on NixOS, nor is patching it to run on a FHS system without a Nix store. Therefore, we just take a pinned QEMU from the current Debian package for now.
This adds support for K3s/RKE2 deployment via the node-installer. It now, depending on its environment, replaces the configuration files for the embedded containerd on K3s and RKE2 and restarts the respective units.
The version of the runtime-class-files is now inherited from the image version, and the debug attribute in the node-installer image is now inherited from the runtime-class-files.
This renames the "flavours" package to the more descriptive "platforms".
This uses OVMF code and variables from nixpkgs, allowing us to not rely on the QEMU-supplied firmware.
msanft
force-pushed
the
msanft/kata-qemu-tdx-runtime
branch
from
f149b22 to
1e7b15d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds initial support for deploying the Contrast runtime on bare-metal TDX machines with QEMU.
Notice that this does not yet contain full support for running Contrast on such platforms. Things that are left out as of now contain (non-comprehensive list):
Due to the size of this PR, I encourage reviewers to see the commit messages for a more detailed description of what each commit does, and thus to review on a per-commit basis.