Amazing project - thanks for using LiteLLM ?

[ishaan-jaff]

Hi @gokdt thanks for using litellm here - why did you decide to use it ?

[gokdt]

Hi @ishaan-jaff, project is still WIP.

I decided to use it because litellm made it a lot easier to integrate LLMs. This gives me flexibility to switch between OpenAI, Azure OpenAI, and other LLM APIs which I use. All I need to do is change several environment variables and it's ready.

I am looking forward to have your feedback when the project is ready.

[krrishdholakia]

Hey @gokdt,

Great work! Looks like the litellm call was separated into a separate chai.py file. Why do that vs. calling it in main.py?

I'm assuming there's some additional functionality you planning on adding here. Curious what that is?

[gokdt]

Hi @krrishdholakia,

Thanks a lot! Very good timing, I just pushed 4186c16 which introduced function calling support. There will be better plugin management soon.

[krrishdholakia]

Interesting - what else are you planning on adding to chai.py?

[gokdt]

There will be some improvements on the function calling, on the cost savings side. I have a prototype for the "conditional function calling" and I will implement that there soon.

Also, reduced messaging context will be included which provides even more savings on the tokens. I am not sure if that one was built before.

[krrishdholakia]

oh - why is saving tokens/cost important for your use-case? @gokdt

[gokdt]

GPT-4 is expensive :)

It is one of the features that I thought would be nice to have. @krrishdholakia