[Roadmap]LocalAI access o1mini , o1

[wac81]

Description
I have a third-party openai provider and I use localai to access the third-party url and key.
however found LocalAI cannot access o1mini , o1.

Requirements
LocalAI can access o1mini , o1

[enricoros]

Hi @wac81, please add some details. As it stands this doesn't seem to be a Big-AGI issue as we support LocalAI as well as o1 models.
What's the third party provider and your configuration?

[wac81]

use third party provider url is https://api.myhispreadnlp.com/

can use gpt4o, but not o1mini,o1

try to access o1, get error:
Localai: Bad Request - Unsupported value: 'messages[0].role' does not support 'system' with this model. - [DEV_URL: https://api.myhispreadnlp.com/v1/chat/completions]

[wac81]

in other way , or i can setup proxy to access this thrid party url in configure to replace the url of openai itself? Is this feasible?

[enricoros]

@wac81 instead of a LocalAI service, can you set up an OpenAI service with a custom server URL? It may fix the problem.

The o1 and o1-preview models have breaking API changes that are not discoverable and require client tweaks, and we apply them in "openai" services, not localai services.

This is the case in the error above, as openai rejects the 'system' message in o1 models.

Lmk if this fixes it already.

[wac81]

Okay, I'll give it a try

[enricoros]

@wac81 please let me know if you need further support on this.