feat: support china region deployment

[IcyKallen]

Fixes #

🤖 AI-Generated PR Description (Powered by Amazon Bedrock)

Description

This pull request introduces support for Amazon SageMaker models in the project's infrastructure and lambda functions. The changes include modifications to the CLI, API, model management, UI, and user constructs to accommodate SageMaker model integration. Additionally, new files have been added to handle SageMaker chat and embedding models within the LangChain integration.

The motivation behind this change is to leverage the power of Amazon SageMaker for deploying and serving machine learning models, enabling the project to benefit from scalable and efficient model hosting capabilities.

Dependencies required for this change include the AWS SDK for Python (Boto3) and the LangChain library.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

File Stats Summary

File number involved in this PR: 16, unfold to see the details:

The file changes summary is as follows:

Files	Changes	Change Summary
source/infrastructure/lib/shared/types.ts	2 added, 0 removed	The code changes add two new regions, CN_NORTH_1 and CN_NORTHWEST_1, to the SupportedRegion enum for supporting regions in China.
source/script/build.sh	5 added, 3 removed	This code change adds a conditional check for the deployment region before logging into the Amazon Elastic Container Registry (ECR) public repository, allowing for different login regions based on the deployment location.
source/infrastructure/lib/ui/ui-stack.ts	2 added, 1 removed	The code changes involve adding a deployRegion property to the UserConstruct, updating the CfnOutput to use userConstruct.userPoolId instead of userConstruct.userPool.userPoolId, and modifying the callbackUrls list.
source/infrastructure/package.json	3 added, 1 removed	The code changes involve updating dependencies, including adding new AWS SDK v3 packages, removing the old AWS SDK v2 package, and updating versions of existing dependencies.
source/lambda/model_management/model_management.py	66 added, 0 removed	The code changes introduce new functionality to list SageMaker endpoints based on an optional endpoint type filter, and define constants for endpoint types (LLM, EMBEDDING, MULTIMODAL).
source/lambda/online/common_logic/common_utils/constant.py	7 added, 7 removed	The code changes involve modifying string values to use double quotes consistently across various constant classes like IntentType, MKTUserType, HistoryType, LLMTaskType, MessageType, and ModelProvider.
source/lambda/authorizer/custom_authorizer.py	21 added, 6 removed	The code changes appear to be related to handling JWT token verification and claims extraction from the Authorization header or query string parameter. It includes formatting changes for better readability and splitting long lines into multiple lines.
source/lambda/online/common_logic/langchain_integration/models/chat_models/sagemaker_models.py	67 added, 0 removed	This code defines classes for interacting with SageMaker LLM models, including SageMakerVllmChatModel, SageMakerDeepSeekR1DistillLlama, and its variants for different model sizes (70B, 8B, and 32B).
source/lambda/online/common_logic/langchain_integration/models/chat_models/init.py	18 added, 7 removed	The code changes include adding support for SageMaker models by importing the sagemaker_models module, updating the MODEL_PROVIDER_LOAD_FN_MAP to include SageMaker, and making some formatting changes for better code readability.
source/lambda/online/common_logic/langchain_integration/models/embedding_models/sagemaker_embeddings.py	19 added, 24 removed	The code changes involve importing necessary modules, defining a SageMakerEmbeddingBaseModel class, and creating a create_model function to instantiate a SageMakerEndpointEmbeddings model with specified configurations and credentials.
source/infrastructure/lib/api/model-management.ts	3 added, 1 removed	This code change adds a new API resource "/endpoints" under "/management" in the API Gateway, and associates it with the existing Lambda function integration for model management operations.
source/infrastructure/lib/ui/ui-portal.ts	66 added, 66 removed	The code changes involve creating an S3 bucket for web assets, configuring an Origin Access Identity and CloudFront distribution using CfnDistribution, granting read permissions to CloudFront, and deploying web assets to the S3 bucket using BucketDeployment.
source/infrastructure/cli/magic-config.ts	15 added, 8 removed	The code changes involve updating AWS SDK imports and methods for getting AWS account ID and region, using the new AWS SDK V3 modules and methods. It also updates the regex pattern for validating the custom domain endpoint.
source/infrastructure/lib/user/user-construct.ts	33 added, 20 removed	The code changes include setting up Cognito resources for user authentication and authorization, with a conditional setup for China regions using custom OIDC resources, and creating an admin user and group in the user pool.
source/infrastructure/lib/model/model-construct.ts	4 added, 2 removed	The code changes involve setting the modelRegion property from the provided configuration, handling the image URL domain and ECR account for China regions, and adding an EventBridge rule to trigger a Lambda function.
source/lambda/online/common_logic/langchain_integration/models/embedding_models/init.py	13 added, 6 removed	The code changes include importing the typing module, reordering imports, using f-strings, adding type hints, and renaming a constant for the SageMaker provider.

🤖 AI-Generated PR Description (Powered by Amazon Bedrock)

Description

This pull request includes changes to integrate Amazon SageMaker models for text generation and embedding tasks within the application. The key changes are:

1. Added a new module source/lambda/online/common_logic/langchain_integration/models/chat_models/sagemaker_models.py to support Amazon SageMaker text generation models for chat functionality.
2. Modified source/lambda/online/common_logic/langchain_integration/models/embedding_models/__init__.py and source/lambda/online/common_logic/langchain_integration/models/embedding_models/sagemaker_embeddings.py to integrate Amazon SageMaker embedding models.
3. Updated the frontend components (source/portal/src/pages/chatbot/ChatBot.tsx, source/portal/src/pages/chatbotManagement/ChatbotDetail.tsx, source/portal/src/pages/customService/CustomerService.tsx, and source/portal/src/pages/intention/IntentionDetail.tsx) to support the new SageMaker model integration.
4. Modified various infrastructure components (source/infrastructure/cli/magic-config.ts, source/infrastructure/lib/api/model-management.ts, source/infrastructure/lib/model/model-construct.ts, source/infrastructure/lib/shared/types.ts, source/infrastructure/lib/ui/ui-portal.ts, source/infrastructure/lib/ui/ui-stack.ts, source/infrastructure/lib/user/user-construct.ts, and source/infrastructure/package.json) to accommodate the SageMaker model integration.
5. Updated the build script (source/script/build.sh) to include the new changes.

This change requires updating the application documentation to reflect the new SageMaker model integration and usage instructions.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

File Stats Summary

File number involved in this PR: 20, unfold to see the details:

The file changes summary is as follows:

Files	Changes	Change Summary
source/infrastructure/lib/api/model-management.ts	3 added, 1 removed	This code change adds a new API resource and method for managing model endpoints in the ModelApi class, allowing clients to retrieve the endpoints associated with a deployed model.
source/portal/src/pages/chatbot/ChatBot.tsx	3 added, 1 removed	The code changes involve adding an enteredTextLabel prop to the Select component, which allows customizing the label for the entered text value.
source/script/build.sh	5 added, 3 removed	This code change adds a conditional check to set the ECR login region based on the deploy region, skipping the ECR login for specific regions in China.
source/infrastructure/lib/shared/types.ts	2 added, 0 removed	The code change adds two new regions, CN_NORTH_1 and CN_NORTHWEST_1, to the SupportedRegion enum, likely for supporting China regions.
source/lambda/model_management/model_management.py	66 added, 0 removed	The code changes introduce a new endpoint to list all SageMaker endpoints with a specified endpoint type (LLM, embedding, or multimodal), along with additional helper classes and functions to support this new functionality.
source/infrastructure/lib/ui/ui-stack.ts	2 added, 1 removed	The code changes involve adding a deployRegion prop to the UserConstruct and updating the CfnOutput to directly reference userConstruct.userPoolId instead of userConstruct.userPool.userPoolId.
source/portal/src/pages/intention/IntentionDetail.tsx	0 added, 16 removed	The code changes remove the enableKeyboardNavigation prop from the table component, remove the wrapLinesPreference, stripedRowsPreference, contentDensityPreference, and contentDisplayPreference props, and modify the stickyColumnsPreference prop.
source/lambda/online/common_logic/langchain_integration/models/embedding_models/sagemaker_embeddings.py	19 added, 24 removed	The code changes involve refactoring the imports, removing unused imports, and making minor changes to logging statements and variable assignments. The core functionality of creating a SageMaker embedding model remains the same.
source/lambda/online/common_logic/common_utils/constant.py	7 added, 7 removed	The code changes include fixing string quotation inconsistencies, removing a commented-out line, and renaming a constant value from "SAGEMAKER_MULTIMODEL" to "SAGEMAKER" in the ModelProvider class.
source/infrastructure/package.json	3 added, 1 removed	This code change updates the project dependencies by adding three new AWS SDK packages for STS, credential providers, and shared INI file loader, while removing the older aws-sdk package.
source/lambda/authorizer/custom_authorizer.py	21 added, 6 removed	The code changes include reorganizing import statements, modifying the token extraction logic from headers for different API types (REST API and WebSocket API), constructing the public key using the RSAAlgorithm from the jwt library, and verifying the JWT token signature with additional options like audience, issuer, and verify_exp.
source/infrastructure/lib/ui/ui-portal.ts	66 added, 66 removed	The code changes include creating an S3 bucket for web assets, configuring an Origin Access Identity (OAI) and granting read permissions to CloudFront, creating a CloudFront distribution using CfnDistribution with custom error responses, and deploying web assets to the S3 bucket using BucketDeployment with CloudFront invalidation.
source/infrastructure/cli/magic-config.ts	15 added, 8 removed	The code changes include updating AWS SDK imports, using the new AWS SDK V3 clients (STSClient, GetCallerIdentityCommand) and credential providers (fromIni, loadSharedConfigFiles) to get AWS account and region information, and updating the regex pattern for validating the OpenSearch domain endpoint.
source/lambda/online/common_logic/langchain_integration/models/chat_models/init.py	18 added, 7 removed	The code changes involve adding support for importing and loading SageMaker models, updating the mapping of model providers to their respective module loading functions, and refactoring some code for better readability and formatting consistency.
source/infrastructure/lib/user/user-construct.ts	33 added, 20 removed	This code change adds a new property deployRegion to the UserProps interface, and conditionally sets up either Cognito resources (UserPool, UserPoolDomain, UserPoolClient, etc.) or custom OIDC resources based on whether the deployRegion starts with cn- (China region).
source/portal/src/pages/customService/CustomerService.tsx	6 added, 0 removed	The code changes add i18n string props to the TopNavigation component for accessibility labels and menu texts, likely for internationalization and localization purposes.
source/lambda/online/common_logic/langchain_integration/models/chat_models/sagemaker_models.py	67 added, 0 removed	This code defines classes for interacting with SageMaker LLM models, including DeepSeek R1 Distill Llama and DeepSeek R1 Distill Qwen models of different sizes, with configuration options like temperature and max tokens.
source/infrastructure/lib/model/model-construct.ts	4 added, 2 removed	The code changes update the modelRegion property to be initialized from the provided configuration, and set the modelImageUrlDomain and modelPublicEcrAccount based on whether the deployment region is in China or not.
source/portal/src/pages/chatbotManagement/ChatbotDetail.tsx	0 added, 16 removed	The code changes remove the enableKeyboardNavigation prop from the Table component, remove the wrapLinesPreference, stripedRowsPreference, contentDensityPreference, and contentDisplayPreference props from the Preferences component, and modify the stickyColumnsPreference prop.
source/lambda/online/common_logic/langchain_integration/models/embedding_models/init.py	13 added, 6 removed	The code changes include importing the typing module, reordering imports, using f-strings, adding type hints, and refactoring a dictionary mapping to use the ModelProvider enum values as keys.

🤖 AI-Generated PR Description (Powered by Amazon Bedrock)

Description

This pull request introduces a new feature that integrates Amazon SageMaker models for both chat and embedding capabilities into the existing infrastructure. The key changes include:

1. Addition of sagemaker_models.py to support SageMaker chat models.
2. Modifications to sagemaker_embeddings.py to incorporate SageMaker embedding models.
3. Updates to various components, such as model-construct.ts, ui-portal.ts, ui-stack.ts, and user-construct.ts, to accommodate the new SageMaker model integration.
4. Changes to the frontend components, including ChatBot.tsx, ChatbotDetail.tsx, CustomerService.tsx, and IntentionDetail.tsx, to reflect the new SageMaker model functionality.
5. Updates to the build script build.sh and package dependencies in package.json.

This change requires the installation of the necessary SageMaker dependencies and the provisioning of SageMaker resources (models, endpoints, etc.) for the integration to work correctly.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

File Stats Summary

File number involved in this PR: 20, unfold to see the details:

The file changes summary is as follows:

Files	Changes	Change Summary
source/infrastructure/lib/shared/types.ts	2 added, 0 removed	The code change adds two new regions, CN_NORTH_1 and CN_NORTHWEST_1, to the SupportedRegion enum.
source/portal/src/pages/customService/CustomerService.tsx	6 added, 0 removed	The code change adds internationalization (i18n) strings for various TopNavigation component elements, allowing for localized text display.
source/infrastructure/lib/model/model-construct.ts	4 added, 2 removed	The code changes modify the ModelConstruct class to dynamically set the deployment region using a configuration property, and handle the appropriate ECR account and image URL domain for China regions.
source/portal/src/pages/chatbot/ChatBot.tsx	3 added, 1 removed	The code changes involve adding an enteredTextLabel prop to the Select component, which allows customizing the label displayed for the entered text value in the dropdown.
source/infrastructure/lib/api/model-management.ts	3 added, 1 removed	This code change adds a new API resource and method for managing model endpoints, allowing retrieval of endpoint information via a GET request.
source/lambda/authorizer/custom_authorizer.py	21 added, 6 removed	The code changes involve refactoring the token extraction logic from the Authorization header, handling both "authorization" and "Authorization" cases, and using string formatting for better readability.
source/infrastructure/lib/ui/ui-stack.ts	2 added, 1 removed	This code change adds a deployRegion property to the UserConstruct and updates the CfnOutput to export the userPoolId directly from the UserConstruct instead of accessing it through the userPool property.
source/portal/src/pages/intention/IntentionDetail.tsx	0 added, 16 removed	The code changes remove the enableKeyboardNavigation prop from the table component, remove the wrapLinesPreference, stripedRowsPreference, contentDensityPreference, and contentDisplayPreference props, and modify the stickyColumnsPreference prop.
source/infrastructure/package.json	3 added, 1 removed	The code changes involve updating dependencies, including adding new AWS SDK packages for STS, credential providers, and shared INI file loader, while removing the old aws-sdk package.
source/script/build.sh	5 added, 3 removed	This code change adds a conditional check to set the ECR login region based on the deploy region, skipping the ECR login for specific regions, and removes a redundant ECR login command.
source/lambda/online/common_logic/langchain_integration/models/chat_models/init.py	18 added, 7 removed	The code changes involve adding support for importing SageMaker models, updating the model loading mechanism to handle SageMaker models, and improving code readability through formatting changes.
source/lambda/online/common_logic/langchain_integration/models/embedding_models/sagemaker_embeddings.py	19 added, 24 removed	The code changes involve refactoring the import statements, removing redundant imports, and updating the logger configuration. Additionally, it modifies the creation of the SageMaker embedding model by using environment variables for AWS credentials, handling optional model kwargs, and adding support for specifying a target model during model creation.
source/lambda/online/common_logic/langchain_integration/models/embedding_models/init.py	13 added, 6 removed	This code change adds type hinting using the typing module, imports the module at the beginning, renames a model provider constant, and improves code readability with better formatting and line breaks.
source/lambda/model_management/model_management.py	66 added, 0 removed	The code changes introduce new functionality to list SageMaker endpoints based on an optional endpoint type filter, including pagination support. It also adds constants for endpoint types (LLM, EMBEDDING, MULTIMODAL) and a new boto3 client for SageMaker.
source/infrastructure/cli/magic-config.ts	15 added, 8 removed	The code changes involve updating the AWS SDK imports and methods for retrieving the AWS account ID and region. The new imports use the modular @AWS-SDK packages, and the getAwsAccountAndRegion function has been updated to use the new SDK clients and methods. Additionally, a minor change was made to the regular expression validation for the custom domain endpoint.
source/infrastructure/lib/user/user-construct.ts	33 added, 20 removed	This code change introduces region-specific setup for OIDC resources in a UserConstruct class. It checks if the deployment region is a China region and sets up custom OIDC resources if true, otherwise it creates Cognito resources like UserPool, UserPoolDomain, UserPoolClient, and AdminUser.
source/infrastructure/lib/ui/ui-portal.ts	63 added, 67 removed	This code change sets up an S3 bucket for hosting web assets, creates a CloudFront distribution using the AWS CDK, and deploys the web assets to the S3 bucket using the AWS CDK's BucketDeployment construct, with the CloudFront distribution handling caching and content delivery.
source/lambda/online/common_logic/common_utils/constant.py	7 added, 7 removed	The code changes involve updating string values from single quotes to double quotes for consistency, renaming the SAGEMAKER_MULTIMODEL constant to SAGEMAKER in the ModelProvider class, and removing commented-out code lines.
source/portal/src/pages/chatbotManagement/ChatbotDetail.tsx	0 added, 16 removed	This code change removes the enableKeyboardNavigation prop from the table component, simplifies the contentDisplayPreference prop by removing unnecessary options, and removes the wrapLinesPreference, stripedRowsPreference, and contentDensityPreference props.
source/lambda/online/common_logic/langchain_integration/models/chat_models/sagemaker_models.py	67 added, 0 removed	This code introduces new classes for interacting with AWS SageMaker large language models, specifically DeepSeek R1 DistillLlama models of different sizes (70B, 8B) and DeepSeek R1 DistillQwen 32B model, with configuration options for model parameters and AWS credentials.