New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Including GBNF-grammar to constrain decoder? #81

Open

matthiaskrmr opened this issue Apr 16, 2024 · 1 comment

Labels

matthiaskrmr commented Apr 16, 2024

Hello!

In the V1.5.0 release of whisper.cpp it was mentioned that GBNF-grammar can be used to constrain the decoder in specific scenarios. ggernganov referred to the implementation of that grammar approach in llama.cpp: https://github.com/ggerganov/llama.cpp/blob/a6fc554e268634494f33b0de76f9dde650dd292f/grammars/README.md

Is it possible to use this approach in Unity-environments using this package and if yes, how so?

Thanks in advance!

Owner

Macoron commented Apr 16, 2024

Unfortunately, right now it's not supported by whisper.unity. It would require two components to work:

Prepare bindings for grammar parameters in whisper_full_params
Include an original GBNF parser or rewrite it in C#

I would be glad to merge that if anyone is interested in working on this feature.

Macoron added the enhancement label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment