Methods astream and astream_event not working for ChatVertexAI

[tanukiki]

When using:

from langchain_google_vertexai import ChatVertexAI, VertexAI
llmodel = ChatVertexAI(model_name="gemini-2.0-flash", max_tokens=1024)

async for chunk in llmodel.astream('Say some interesting facts about chinchillas'):
    print(chunk, flush=True)

I get

TypeError: object ResponseIterator can't be used in 'await' expression

It works for base VertexAI model and for ChatGoogleGenerativeAI from langchain_google_genai but not for ChatVertexAI. The same with .astream_events

[tanukiki]

When using:

from langchain_google_vertexai import ChatVertexAI, VertexAI
llmodel = ChatVertexAI(model_name="gemini-2.0-flash", max_tokens=1024)

async for chunk in llmodel.astream('Say some interesting facts about chinchillas'):
print(chunk, flush=True)
I get

TypeError: object ResponseIterator can't be used in 'await' expression

It works for base VertexAI model and for ChatGoogleGenerativeAI from langchain_google_genai but not for ChatVertexAI. The same with .astream_events

Ok it turns out that this error only occurs if we use vertexai.init() before running the model:

from langchain_google_vertexai import ChatVertexAI, VertexAI
import vertexai

vertexai.init()
llmodel = ChatVertexAI(model_name="gemini-2.0-flash", max_tokens=1024)

async for chunk in llmodel.astream('Say some interesting facts about chinchillas'):
    print(chunk, flush=True)

It may be related to incorrect async rest credentials https://www.googlecloudcommunity.com/gc/AI-ML/Async-REST-credentials-for-Vertex-AI/m-p/839712

[SauravP97]

Looks like the error is being originated from here:

https://github.com/langchain-ai/langchain-google/blob/main/libs/vertexai/langchain_google_vertexai/chat_models.py#L639 #

i.e the last line in this code snippet.

 @retry_decorator
    async def _completion_with_retry_inner(
        generation_method: Callable, **kwargs: Any
    ) -> Any:
       return await generation_method(**kwargs)

I further looked at the generation_method. It seems this comes from the async_prediction_client.stream_generate_content.

Ref: https://github.com/langchain-ai/langchain-google/blob/main/libs/vertexai/langchain_google_vertexai/chat_models.py#L1699

response_iter = _acompletion_with_retry(
    self.async_prediction_client.stream_generate_content,
    max_retries=self.max_retries,
    request=request,
    is_gemini=True,
    **kwargs,
)

This is being created from this method async_prediction_client

Ref: https://github.com/langchain-ai/langchain-google/blob/main/libs/vertexai/langchain_google_vertexai/_base.py#L147C9-L147C32

    @property
    def async_prediction_client(self) -> v1beta1PredictionServiceAsyncClient:
        """Returns PredictionServiceClient."""
        if self.async_client is None:
            async_client_kwargs: dict[str, Any] = dict(
                client_options=self.client_options,
                client_info=get_client_info(module=self._user_agent),
                credentials=self.credentials,
            )

            if self.api_transport is not None:
                async_client_kwargs["transport"] = self.api_transport

            self.async_client = v1beta1PredictionServiceAsyncClient(
                **async_client_kwargs
            )

        return self.async_client

I am guessing the error is getting originated from the Vertex AI cloud library since the v1beta1PredictionServiceAsyncClient is a part of that library.

[lkuligin]

@tanukiki could you share a full trace, please? and is the error consistently reproducible on your side?