Process for getting the latest llama.cpp version?

[MTStrothers]

So, there is a bug right now with llama-3 with the version of llama.cpp that runs in the current version of text-generation-webui. Information about the bug (something with the tokenizer) can be found here:

Just today a fix was applied to the main release of llama.cpp with this commit.

My question, is, what exactly is the process for this fixed version of llama.cpp to get into text-generation-webui? It seems that there are two things that need to happen:

-abetlen/llama-cpp-python/ needs to update based on the new version of ggerganov/llama.cpp
-oogabooga needs to update the dependency requirements to pull the new version of abetlen/llama-cpp-python/. I think currently it's not even pulling the latest version of abetlen/llama-cpp-python/

Am I missing anything here or do I just have to wait? Just wondering.

[strawberrymelonpanda]

Pretty much just have to wait, but I've had luck sometimes with building each step of the chain myself. (Build llama.cpp's latest commit, include that in llama-cpp-python, include that in Ooba...) but it's not straight-forward, especially on Windows.

The instructions are in the various repos. Just start from Llama.cpp and work up, replacing the version pulled in the downstream repo. Some of the better LLMs can probably help walk you through it.

Personally I would really like to see Ooba use Llama.cpp directly without llama-cpp-python in the middle.