How to send image to vision model

[engdante]

Please tell me the correct way to send an image to the vision model.

this is my function:

def generate_image_description(image_path):
prompt = f"Describe the content of this image: {image_path}."
response = client.chat(model='llava-phi3:3.8b', messages=[
{
'role': 'user',
'content': prompt,
},
])
return response['message']['content']

[karanravindra]

Please refer to the definition of a "chat message" in the python code Message Type Dict.

The image can be passed in using the "images" key in your message dictionary. The "images" key is a sequence of "bytes" or "path-like str".

Here is an example:

import ollama

response = ollama.chat(
    model="moondream",
    messages=[
        {"role": "user", "content": "Describe the image", "images": ["./cat.jpeg"]}
    ],
)

print(response["message"]['role'])
print(response["message"]['content'])