You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am getting similar error. Here's what print(chunk) prints:
ChatCompletionResponse(completion_message=None, logprobs=None, error={'message': '500: Internal server error: An unexpected error occurred.'})
On the server side, i see this error:
INFO: ::1:45792 - "POST /inference/chat_completion HTTP/1.1" 200 OK
Traceback (most recent call last):
File "/home/user/src/llama-stack/llama_stack/distribution/server/server.py", line 206, in sse_generator
async for item in event_gen:
TypeError: 'async for' requires an object with __aiter__ method, got coroutine
data_url = data_url_from_image("dog.jpg")
print("The obtained data url is", data_url)
iterator = client.inference.chat_completion(
model=model,
messages=[
{
"role": "user",
"content": [
{ "image": { "uri": data_url } },
"Write a haiku describing the image"
]
}
],
stream=True
)
for chunk in iterator:
print(chunk.event.delta, end="", flush=True)
The text was updated successfully, but these errors were encountered: