Sagemaker Invocation

[usen1364]

Hi,
I tried the promptnode with sagemaker invocation using model_kwargs={"aws_profile_name": "profile_name", "aws_region_name": "us-east-1"}
And it was successful. However, I am unclear on how I do the same inside an ecs container (fargate).

I noticed you can pass

aws_access_key_id
aws_secret_access_key
aws_session_token
aws_region_name

are these values available inside the container environment by default?
Or is there a better way to do this?
Thank you

[vblagoje]

We have yet to try that option @usen1364 Perhaps my colleague @ZanSara currently working on this feature, might give us more insights.

[usen1364]

Hi thanks for the reply,
I ended up doing something like this. and it works

/app/src/build_config.py

from boto3 import Session

session = Session()
credentials = session.get_credentials()
current_credentials = credentials.get_frozen_credentials()

MODEL_NAME = os.getenv("MODEL_NAME_OR_PATH")
AWS_REGION = os.getenv("AWS_REGION", "us-east-1")
ACCESS_KEY = current_credentials.access_key
ACCESS_SECRET = current_credentials.secret_key
SESSION_TOKEN = current_credentials.token

prompt_node = PromptNode(model_name_or_path=MODEL_NAME,
                         model_kwargs={"aws_access_key_id": ACCESS_KEY,
                                       "aws_secret_access_key": ACCESS_SECRET,
                                       "aws_session_token": SESSION_TOKEN,
                                       "aws_region_name": AWS_REGION
                                       },
                         default_prompt_template=rag_prompt, max_length=500,
                         )
pipe = Pipeline()
.....
pipe.save_to_yaml(Path("../pipeline/faq.yaml"))                

And then at the container start I run above script using docker as an entrypoint.sh
entrypoint.sh

#!/bin/sh
# entrypoint.sh

echo "Building pipeline config"

# Check if IN_AWS is set to "true"
if [ "$IN_AWS" = "true" ]; then
  # Run the Python file only if IN_AWS is true
  python3 /app/src/build_config.py
else
  echo "IN_AWS is not set to true. Skipping Python file execution."
fi

cd /app/rest_api &&
exec "$@"

And in the dockerfile

ENTRYPOINT ["/app/entrypoint.sh"]
CMD ["python3", "application.py"]

[usen1364]

This is not a good solution, the SESSION_TOKEN expires after 12 hours or so.