Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add llm_drop_params setting to avoid AnthropicException #1239

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

nobu007
Copy link

@nobu007 nobu007 commented Apr 27, 2024

litellm.drop_params=True can avoid this error(#1240).
Error in chat: AnthropicException - anthropic does not support parameters
I think drop_params is good altanative for debugging or workaround of future errors.

Describe the changes you have made:

This option allow litellm.drop_params=True.

Reference any relevant issues (e.g. "Fixes #000"):

Related with #1240.

Pre-Submission Checklist (optional but appreciated):

  • I have included relevant documentation updates (stored in /docs)
  • I have read docs/CONTRIBUTING.md
  • I have read docs/ROADMAP.md

I found "Use Anthropic function calling" in ROADMAP.md.
Is #1240 just not implemaneted?

OS Tests (optional but appreciated):

  • Tested on Windows
  • Tested on MacOS
  • Tested on Linux

Test(works fine)

interpreter --model claude-3-haiku-20240307 -y --llm_drop_params

Note1

This error is probably resolved by this PR.

Error in chat: AnthropicException - {"type":"error","error":{"type":"invalid_request_error","message":"messages: roles must alternate between \"user\" and \"assistant\", but found multiple \"user\" roles in a row"}}

Note2

"litellm.modify_params" helps activate here code in litellm.
The first message is empty or non user, dummy message append in anthropic model.
It is not important, but helps for me.

@nobu007 nobu007 force-pushed the add_litellm_drop_params branch 3 times, most recently from e2818d7 to b920da7 Compare April 27, 2024 07:40
@nobu007 nobu007 marked this pull request as ready for review April 27, 2024 08:50
@KillianLucas
Copy link
Collaborator

Hey @nobu007, thanks for testing this with Anthropic and making this fix!

Should we just set litellm.drop_params = True? It seems like that could never be bad— if LiteLLM doesn't want to pass a param into the LLM, I feel like we should just let it silently drop it. Any side effects of that that you can see?

@nobu007
Copy link
Author

nobu007 commented May 12, 2024

@KillianLucas
Yes.
In my for 2 weeks try, it seems no side effects forme.
drop_params is used around here.

Btw, modify_params is also useful for Anthropic.
modify_params is used around here and here.

How about adding this also?

@nobu007 nobu007 force-pushed the add_litellm_drop_params branch 2 times, most recently from f4a9f2c to a9e26f2 Compare May 14, 2024 00:33
@nobu007 nobu007 force-pushed the add_litellm_drop_params branch 3 times, most recently from 151e866 to 49c4c3b Compare June 2, 2024 03:17
@nobu007
Copy link
Author

nobu007 commented Jun 2, 2024

@KillianLucas
I added "llm_modify_params" also.
Please check it!

nobu007 added 2 commits June 25, 2024 08:58
litellm.drop_params=True can avoid this error.
Error in chat: AnthropicException - anthropic does not support parameters
@nobu007 nobu007 force-pushed the add_litellm_drop_params branch from 49c4c3b to a19ebac Compare June 25, 2024 00:00
@@ -805,4 +805,45 @@ computer.import_computer_api: True
```

</CodeGroup>
````
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This ```` that added in this PR looks no meanings. So I delete it.
#1291

@nobu007
Copy link
Author

nobu007 commented Jun 25, 2024

@KillianLucas
I rebased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants