[TDX] Added basic documentation to enable TDX in ChatQnA #1212
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JakubLedworowski
force-pushed
the
enable-intel-tdx-chatqna
branch
from
68149c6 to
e47902a
Compare
dcmiddle
reviewed
Dec 6, 2024
| ### Kubelet Configuration | ||
|
|
||
| To run a complex and heavy application like OPEA, the cluster administrator must increase the kubelet timeout for container creation, otherwise the pod creation may fail due to timeout `Context deadline exceeded`. | ||
| This is required because the container creation process can take a long time due to the size of pod images and the need to download the AI models. |
There was a problem hiding this comment.
Is this timeout change generally required for any k8s deployment? If so should this be added to the main k8s readme?
There was a problem hiding this comment.
This is generally required for all use cases where the Container creation takes long time. When TDX is involved, container creation time increases so much that it usually exceeds the default 2 minutes. It is described in k8s docs: https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/
There was a problem hiding this comment.
Is that just for peer pods or running CoCo on the host also often breaks 2 minutes?
| > [!NOTE] | ||
| > Running TDX-protected services requires the user to define the pod's resources request (cpu, memory). | ||
| > | ||
| > Due to lack of hotplugging feature in TDX, the assigned resources cannot be changed after the pod is scheduled and the resources will not be shared with any other pod. |
There was a problem hiding this comment.
check hotplugging (TEE-specific? or kata-specific?)
| > | ||
| > After kubelet restart, some of the internal pods from `kube-system` namespace might be reloaded automatically. | ||
|
|
||
| All kubelet configuration options can be found [here](https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/). |
Comment on lines
135
to
140
| ```bash | ||
| POD_NAME=$(kubectl get pods | grep 'chatqna-tgi' | awk '{print $1}') | ||
| kubectl get pod $POD_NAME -o jsonpath='{.spec.runtimeClassName}' | ||
| ``` | ||
|
|
||
| In the output you should see: |
There was a problem hiding this comment.
Just show that it is running
- added README_tdx.md - described steps to run ChatQnA using helm and GMC Signed-off-by: Jakub Ledworowski <[email protected]>
- Removed deployment option with helm - Added sample chatqna_tdx.yaml - Generalized description but left ChatQnA as an example Signed-off-by: Jakub Ledworowski <[email protected]>
Signed-off-by: Jakub Ledworowski <[email protected]>
Signed-off-by: Jakub Ledworowski <[email protected]>
Signed-off-by: Jakub Ledworowski <[email protected]>
JakubLedworowski
force-pushed
the
enable-intel-tdx-chatqna
branch
from
12e4c94 to
608e869
Compare
Signed-off-by: Jakub Ledworowski <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Confidential computing in AI in the cloud focuses on protecting sensitive data and computations from unauthorized access and tampering. It uses advanced security technologies, such as hardware-based isolation and encryption, to create secure environments where data and AI models can be processed safely. This ensures that even cloud service providers cannot access the data, providing a higher level of privacy and security. By leveraging confidential computing, organizations can confidently use AI in the cloud for tasks that involve sensitive information, such as healthcare data analysis or financial predictions, while complying with strict data protection regulations.
This change introduces the guide on protecting chosen microservices with Intel TDX technology:
README_tdx.mdchatqna_tdx.yamlthat has all microservices configured with TDX-protection and default settingsIssues
n/a
Type of change
List the type of change like below. Please delete options that are not relevant.
Dependencies
n/a
Tests
Manual tests with sample request enabling TDX in all ChatQnA services:
dataprep,embedding,llm,redis,reranking,retriever,tei,teirerank,tgi