ChatCompletionToolChoice and ChatCompletionToolChoiceOption are unnecessarily complex

[the-gigi]

The ChatCompletionToolChoice forces the model to call a specific tool. let's call it X. There are 3 options:

1. The tools list doesn't contain a tool called X
2. The tools list contains just the tool X
3. The tools list contains X and additional tools

Option 1 is invalid. There is no reason to allow it.
Option 2 is valid, but why specify the tool X twice? once in the list of tools and second time in the ChatCompletionToolChoice.
Option 3 is valid but wasteful/confusing. If the goal is to call X, why even pass additional tools?

I suggest to completely get rid of ChatCompletionToolChoice. The same effect can be accomplished by passing a tool list that contains just X and the ChatCompletionToolChoiceOption "required".

Now, let's consider the ChatCompletionToolChoiceOption. "auto" and "required" are necessary and useful. But, "none" doesn't provide any value. If you pass "none" it tells the model to avoid calling any tool. If this is the case, then just pass an empty list of tools.

If following both recommendations the complex specification for tool choice (can be either ChatCompletionToolChoice or ChatCompletionToolChoiceOption) will collapse to super simple single enum ChatCompletionToolChoiceOption with just two values: "auto" and "enum". That's it.

This will considerably simply the life of OpenAI API users and OpenAI client libraries (especially if using strongly typed languages).

Actually, we can take it further. If the Choice is only "auto" or "required". We don't even need an enum. We can specify everything with a simple boolean "toolRequired" field. If it's "true" then a tool call is required. If it's not true (missing or "false") then it's auto (model decides which tool to call if any).

[sashirestela]

If your scenario is to send a different list of functions in every request to Chat Completion, yeah, it could be kind of redundant to indicate some option or the specific function to call.

But there is another scenario where people send the same list of functions in every request, so, in this case, they would need the tool_choice parameter in its two flavors: as option to indicate none/auto/required or as object to force the model to call a specific function.

If they want to cover all the scenarios, I think it is fine to keep the tool_choice parameter as OpenAI has designed it.

[the-gigi]

Sending N functions every time over the network to a REST API when you know for sure that you want to force calling just one specific function (or even not call a function at all with "none") is a waste of bandwidth, memory, compute and most importantly - tokens. It can make sense if the model had something like session memory and you could maintain a list of functions and the in specific requests reference these functions by name or some short id returned when you initialized the session with a list of functions.

Note that libraries can support this scenario and allow the user to initialize a client with a list of functions and then when making a request choose between the current options (auto, none, required, specific function). Then the library when preparing the request can decide which functions actually to send. For example, no functions if "none", all functions if "auto"/"required" with toolRequired=false/true and exactly one function if a specific function was specified.

[sashirestela]

Note that libraries can support this scenario and allow the user to initialize a client with a list of functions and then when making a request choose between the current options (auto, none, required, specific function). Then the library when preparing the request can decide which functions actually to send. For example, no functions if "none", all functions if "auto"/"required" with toolRequired=false/true and exactly one function if a specific function was specified.

That could be a great extra feature for a library.
I'm going to evaluate how to add it to simple-openai

[sashirestela]

This is available in the last version 3.3.0 of simple-openai