[FEATURE] Add Amazon SageMaker as a provider

[bobbywlindsey]

Feature Area

Documentation

Is your feature request related to a an existing bug? Please link it here.

NA

Describe the solution you'd like

• Include SageMaker as a provider for LLMs with documented examples
• Include SageMaker into the cli that bootstraps a new crewAI project

Describe alternatives you've considered

NA

Additional context

NA

Willingness to Contribute

Yes, I'd be happy to submit a pull request

[dgallitelli]

+1 to this request. I've done a quick test and here are my findings:

1. the CLI bootstrap has SageMaker as provider under "other" - select 10 first, then 31 for SageMaker
2. To use it with the agents.yaml config, here is how:

replace-with-agent-name:
  role: [...]
  goal: [...]
  backstory: [...]
  llm: sagemaker_chat/replace-with-endpoint-name 
  # alternatively, if the model does not support the Messages API:
  # llm: sagemaker/replace-with-endpoint-name

What I could not figure out was how to use the agents.yaml file to:

1. configure the AWS default region (had to go into the env vars manually)
2. configure the max number of tokens or any other model parameter

Looking forward to official Crew.AI documentation on this very common use case!

[bobbywlindsey]

+1 to this request. I've done a quick test and here are my findings:

1. the CLI bootstrap has SageMaker as provider under "other" - select 10 first, then 31 for SageMaker
2. To use it with the agents.yaml config, here is how:

replace-with-agent-name:
role: [...]
goal: [...]
backstory: [...]
llm: sagemaker_chat/replace-with-endpoint-name

alternatively, if the model does not support the Messages API:

llm: sagemaker/replace-with-endpoint-name

What I could not figure out was how to use the agents.yaml file to:

1. configure the AWS default region (had to go into the env vars manually)
2. configure the max number of tokens or any other model parameter

Looking forward to official Crew.AI documentation on this very common use case!

Good callouts 🔥 , I had similar findings as well. To configure model parameters, I found you can just update one of the Agents in the crew.py file like so:

Agent(
    config=self.agents_config['researcher'],
    verbose=True,
    llm=LLM(
        model="sagemaker/my-endpoint",
        temperature=0.6,
        top_p=0.9,
        details=True,
        max_new_tokens=1000,
        verbose=True
    )
)