You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Does SGLang support tool calling for the latest mistralai/Mistral-Small-24B-Instruct-2501? I'm asking because using this script, I can get correct tool call when the model is mistralai/Mistral-7B-Instruct-v0.3 but not when the model is mistralai/Mistral-Small-24B-Instruct-2501.
I'm running the server using python -m sglang.launch_server --model-path <model_tag> --tool-call-parser mistral --port 3000. For Mistral-7B-Instruct-v0.3 a function calling request will return results with tool_calls=[ChatCompletionMessageToolCall(id='0', function=Function(...) while for Mistral-Small-24B-Instruct-2501 the same request will return results with tool_calls=[].
If tool calling is not yet supported for Mistral Small 24B, could you please provide information about when this functionality might be implemented?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi all,
Does SGLang support tool calling for the latest
mistralai/Mistral-Small-24B-Instruct-2501
? I'm asking because using this script, I can get correct tool call when the model ismistralai/Mistral-7B-Instruct-v0.3
but not when the model ismistralai/Mistral-Small-24B-Instruct-2501
.I'm running the server using
python -m sglang.launch_server --model-path <model_tag> --tool-call-parser mistral --port 3000
. ForMistral-7B-Instruct-v0.3
a function calling request will return results withtool_calls=[ChatCompletionMessageToolCall(id='0', function=Function(...)
while forMistral-Small-24B-Instruct-2501
the same request will return results withtool_calls=[]
.If tool calling is not yet supported for Mistral Small 24B, could you please provide information about when this functionality might be implemented?
Thank you for your assistance.
Beta Was this translation helpful? Give feedback.
All reactions