[Bug] On UI, user is not getting any response for VisualQnA after uploading image and submitting query

[mdfaheem-intel]

Priority

P2-High

OS type

Ubuntu

Hardware type

Gaudi2

Installation method

• Pull docker images from hub.docker.com
• Build docker images from source

Deploy method

• Docker compose
• Docker
• Kubernetes
• Helm

Running nodes

Single Node

What's the version?

NA

Description

UI is getting hanged and user is not getting ay response after uploading an image and submitting a valid query.

Reproduce steps

Once the application is deployed using Helm, access the UI as per below steps:
export port=$(kubectl get service visualqna-nginx --output='jsonpath={.spec.ports[0].nodePort}')
echo $port

http://:${port}

Now Upload an image and submit a valid query.
Actual Result: User is not getting any response in UI
Attached the logs errors.

Raw log

[2024-12-19 02:13:47,226] [    INFO] - Base service - CORS is enabled.
[2024-12-19 02:13:47,226] [    INFO] - Base service - Setting up HTTP server
[2024-12-19 02:13:47,227] [    INFO] - Base service - Uvicorn server setup on port 8888
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8888 (Press CTRL+C to quit)
[2024-12-19 02:13:47,239] [    INFO] - Base service - HTTP server setup successful
/home/user/visualqna.py:45: PydanticDeprecatedSince20: The `parse_obj` method is deprecated; use `model_validate` instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.10/migration/
  chat_request = ChatCompletionRequest.parse_obj(data)
INFO:     10.233.75.225:50402 - "POST /v1/visualqna HTTP/1.0" 200 OK
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 259, in __call__
    await wrap(partial(self.listen_for_disconnect, receive))
  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 255, in wrap
    await func()
  File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 232, in listen_for_disconnect
    message = await receive()
              ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 531, in receive
    await self.message_event.wait()
  File "/usr/local/lib/python3.11/asyncio/locks.py", line 213, in wait
    await fut
asyncio.exceptions.CancelledError: Cancelled by cancel scope 7f03314732d0

During handling of the above exception, another exception occurred:

  + Exception Group Traceback (most recent call last):
  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
  |     result = await app(  # type: ignore[func-returns-value]
  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
  |     return await self.app(scope, receive, send)
  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
  |     await super().__call__(scope, receive, send)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 113, in __call__
  |     await self.middleware_stack(scope, receive, send)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__
  |     raise exc
  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__
  |     await self.app(scope, receive, _send)
  |   File "/usr/local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 174, in __call__
  |     raise exc
  |   File "/usr/local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 172, in __call__
  |     await self.app(scope, receive, send_wrapper)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 93, in __call__
  |     await self.simple_response(scope, receive, send, request_headers=headers)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 144, in simple_response
  |     await self.app(scope, receive, send)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
  |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
  |     raise exc
  |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
  |     await app(scope, receive, sender)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 715, in __call__
  |     await self.middleware_stack(scope, receive, send)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 735, in app
  |     await route.handle(scope, receive, send)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle
  |     await self.app(scope, receive, send)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 76, in app
  |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
  |     raise exc
  |   File "/usr/local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
  |     await app(scope, receive, sender)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
  |     await response(scope, receive, send)
  |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 252, in __call__
  |     async with anyio.create_task_group() as task_group:
  |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 815, in __aexit__
  |     raise BaseExceptionGroup(
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 820, in generate
    |     yield from self.raw.stream(chunk_size, decode_content=True)
    |   File "/usr/local/lib/python3.11/site-packages/urllib3/response.py", line 1057, in stream
    |     yield from self.read_chunked(amt, decode_content=decode_content)
    |   File "/usr/local/lib/python3.11/site-packages/urllib3/response.py", line 1206, in read_chunked
    |     self._update_chunk_length()
    |   File "/usr/local/lib/python3.11/site-packages/urllib3/response.py", line 1136, in _update_chunk_length
    |     raise ProtocolError("Response ended prematurely") from None
    | urllib3.exceptions.ProtocolError: Response ended prematurely
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 255, in wrap
    |     await func()
    |   File "/usr/local/lib/python3.11/site-packages/starlette/responses.py", line 244, in stream_response
    |     async for chunk in self.body_iterator:
    |   File "/usr/local/lib/python3.11/site-packages/starlette/concurrency.py", line 62, in iterate_in_threadpool
    |     yield await anyio.to_thread.run_sync(_next, as_iterator)
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
    |     return await get_async_backend().run_sync_in_worker_thread(
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2505, in run_sync_in_worker_thread
    |     return await future
    |            ^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1005, in run
    |     result = context.run(func, *args)
    |              ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/local/lib/python3.11/site-packages/starlette/concurrency.py", line 51, in _next
    |     return next(iterator)
    |            ^^^^^^^^^^^^^^
    |   File "/home/user/GenAIComps/comps/cores/mega/orchestrator.py", line 217, in generate
    |     for chunk in response.iter_content(chunk_size=None):
    |   File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 822, in generate
    |     raise ChunkedEncodingError(e)
    | requests.exceptions.ChunkedEncodingError: Response ended prematurely