Program Chat Conversation Persistence? #192
-
When running a program multiple times, is there a way to retain or pass conversational memory across program instantiations or executions? I need to call the same program multiple times, having it continue an existing conversation, not seeing an example of this in the project notebooks (or perhaps I missed it). |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 1 reply
-
Figured out a way to do it... here's a working example below. The key is to track the conversations and keep passing the conversational history back into the program while it's waiting. import guidance llm = guidance.llms.OpenAI("gpt-3.5-turbo") prog = guidance( prompt = prog(convos=[], user_text = 'Who was first US President?') Output: |
Beta Was this translation helpful? Give feedback.
-
@rickbraddy-pharma - it's up to you what is best based on your application needs. For example, we have a chat application that creates context per request, so instead of having a long-running conversation in one thread, we do a linear execution of the prompt and dynamically fill in the chat context. Each new message is stored, and we build the chat history in context when the next request comes in. This enables you to build robust chat applications. However, this is just a chat example. The great thing about guidance is that you're completely in control and need to find the best way to accomplish your goal. |
Beta Was this translation helpful? Give feedback.
@rickbraddy-pharma - it's up to you what is best based on your application needs.
For example, we have a chat application that creates context per request, so instead of having a long-running conversation in one thread, we do a linear execution of the prompt and dynamically fill in the chat context. Each new message is stored, and we build the chat history in context when the next request comes in. This enables you to build robust chat applications. However, this is just a chat example. The great thing about guidance is that you're completely in control and need to find the best way to accomplish your goal.