PROMPT CACHING with PromptTemplate

[yadgire7]

I could implement prompt caching using ChatPromptTemplate. I have a different concern. I have the exact same prompt which I used as a PromptTemplate and it worked as intended. Now, the performance has worsened when I use ChatPromptTemplate. Any comments or solution for this?
Thank you!

[dosubot[bot]]

Hey @yadgire7! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this issue together!

I found some related discussions and issues that might be helpful:

1. PromptTemplate with Prompt Caching for ChatAnthropic - This discussion is unsolved and might provide insights into similar caching concerns.

2. Using Claude prompt caching with ChatPromptTemplate - Another unsolved discussion that could be relevant.

3. Anthropic's prompt caching in langchain does not work with ChatPromptTemplate - An open issue without a solution yet.

Regarding performance issues, prompt caching in ChatPromptTemplate works by checking if a cache is available and using it to store and retrieve results for given prompts. If caching is enabled, it looks up the prompt in the cache and returns cached results if available. This mechanism is intended to reduce redundant computations ^[1].

If performance has worsened, it might be worth checking if the cache is being used effectively or if there are any specific configurations affecting performance.

To continue talking to Dosu, mention @dosu.

[yadgire7]

I have referred the mentioned discussions, but could not resolve the issue. I am wondering if LangChain can add the functionality of using prompt caching with PromptTemplate.

[DonghaeSuh]

I have referred the mentioned discussions, but could not resolve the issue. I am wondering if LangChain can add the functionality of using prompt caching with PromptTemplate.

I've posted a discussion on #29747 to enable the feature you've mentioned.