Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On demand Support for InvokeModelWithResponseStream for anthropic.claude-3-5-sonnet-20241022-v2:0 fails while AWS support on demand #352

Open
AnujShahSifthub opened this issue Dec 3, 2024 · 1 comment

Comments

@AnujShahSifthub
Copy link

While invoking amazon bedrock using InvokeModelWithResponseStream with modelId as anthropic.claude-3-5-sonnet-20241022-v2:0, it returns:
raise error_class(parsed_response, operation_name)
| botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the InvokeModelWithResponseStream operation: Invocation of model ID anthropic.claude-3-5-sonnet-20241022-v2:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.
+------------------------------------

@terricain
Copy link
Owner

This looks like a configuration issue to do with the data you provided to the SDK call, not an issue with the library. I don't use the bedrock APIs so I won't be able to give you much more than that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants