Gorilla from UC Berkeley

[simonw]

Gorilla:

Gorilla is a LLM that can provide appropriate API calls. It is trained on three massive machine learning hub datasets: Torch Hub, TensorFlow Hub and HuggingFace.

It can generate code and explanations for working directly with those Python APIs.

They offer a free hosted version that you can try out, which imitates the OpenAI API. This means you can run llm against it using the OpenAI compatible models mechanism.

I got this working just now. First I ran this command to find the right folder:

dirname "$(llm logs path)"

On my machine that output:

/Users/simon/Library/Application Support/io.datasette.llm

So this should work for everyone:

cd ~/Library/Application\ Support/io.datasette.llm

Create a extra-openai-models.yaml file in that folder:

touch extra-openai-models.yaml

Now open it in your editor. I use code for VS Code:

code extra-openai-models.yaml

I added this block of YAML to that file:

- model_id: gorilla
  model_name: gorilla-7b-hf-v1
  api_base: "http://zanino.millennium.berkeley.edu:8000/v1"
  api_key: EMPTY

Now I can run prompts against that model like this:

llm -m gorilla 'Convert audio to text'

Output:

<<<domain>>>: Audio Automatic Speech Recognition
<<<api_call>>>: model = SpeechRecognitionModel('jonatasgrosman/wav2vec2-large-xlsr-53-english')
<<<api_provider>>>: Hugging Face Transformers
<<<explanation>>>:1. Import the required libraries and initialize the SpeechRecognitionModel with the pretrained model 'jonatasgrosman/wav2vec2-large-xlsr-53-english'.
2. Provide an audio file path and transcribe the audio using the model.<<<code>>>:

from huggingface_speech_to_text import SpeechRecognition

def load_model():
    model = SpeechRecognition("jonatasgrosman/wav2vec2-large-xlsr-53-english")
    return model

def process_data(audio_path, model):
    audio_data, sample_rate = librosa.load(audio_path, sr=16000)
    transcript = model.transcribe(audio_data)
    return transcript

audio_path = "your_audio_path.wav"

# Load the model
model = load_model()

# Process the data
transcript = process_data(audio_path, model)

print(transcript)

[ShishirPatil]

Thanks for starting this discussion @simonw Excited to see 🦍 Gorilla 🤝 llm integrations!