Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: 'ChatCompletionResponse' object has no attribute 'event' #246

Open
AI-Aether opened this issue Oct 14, 2024 · 2 comments
Open
Labels
question Further information is requested

Comments

@AI-Aether
Copy link

AI-Aether commented Oct 14, 2024

data_url = data_url_from_image("dog.jpg")
print("The obtained data url is", data_url)
iterator = client.inference.chat_completion(
model=model,
messages=[
{
"role": "user",
"content": [
{ "image": { "uri": data_url } },
"Write a haiku describing the image"
]
}
],
stream=True
)

for chunk in iterator:
print(chunk.event.delta, end="", flush=True)

@yanxi0830
Copy link
Contributor

Could you print out on what chunk is? Do you have the output logs from Llama Stack server if you are starting up your Llama Stack distribution?

@yanxi0830 yanxi0830 added the question Further information is requested label Oct 15, 2024
@japsingh
Copy link

japsingh commented Oct 21, 2024

I am getting similar error. Here's what print(chunk) prints:
ChatCompletionResponse(completion_message=None, logprobs=None, error={'message': '500: Internal server error: An unexpected error occurred.'})

On the server side, i see this error:
INFO: ::1:45792 - "POST /inference/chat_completion HTTP/1.1" 200 OK
Traceback (most recent call last):
File "/home/user/src/llama-stack/llama_stack/distribution/server/server.py", line 206, in sse_generator
async for item in event_gen:
TypeError: 'async for' requires an object with __aiter__ method, got coroutine

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants