Save user chat sessions and provide API's for session CRUD and message retrieval

[mbertrand]

What are the relevant tickets?

Closes https://github.com/mitodl/hq/issues/6536 (mostly)

Description (What does it do?)

• Adds a DjangoSaver checkpointer class for Langgraph, based on the redis reference implementation, along with associated data models; removes the mgmt command that previously set up the PostgresSaver checkpoint and tables. Also removes the short-term MemorySaver. Saving to the database is now the default.
• Adds a new UserChatSession model, creates it at the start of a new thread, with title based on initial user message (editable via API), and if created by anonymous user, updates it with the correct user when finally logged in.
• Uses cookies to track anonymous users' sessions (thread ids) and logged-in users' current session, per agent. When a formerly anonymous user logs in and asks another question, the thread ids in the cookie should be assigned to that user in the database.
• Adds API endpoints for:
  • listing chat sessions for the logged in user (../api/v0/chat_sessions/)
  • chat session details - update title, or delete (../api/v0/chat_sessions/<thread_id>/)
  • chat session messages - list all human and agent messages in chronological order, skipping any tool call messages; meant for backfilling UI messages when loading an old session (../api/v0/chat_sessions/<thread_id>/messages/)

How can this be tested?

Saving chat history

• Run docker compose up: if you've already run migrations before, check that they don't run again (the LiteLLM proxy service was previously wiping out the db on startup). The LiteLLM should also start up successfully.
• Change this line in frontend-demo/src/app/page.tsx to include clear_history: true:

const REQUEST_OPTS: AiChatProps["requestOpts"] = {
  apiUrl: `${process.env.NEXT_PUBLIC_MITOL_API_BASE_URL}/http/recommendation_agent/`,
  transformBody(messages) {
    const message = messages[messages.length - 1].content
    return { message, clear_history: true }
  },
  fetchOpts: {
    credentials: "include",
  },
}

• Go to http://ai.open.odl.local:8003 and send a few messages. Because of the change above, each message will be treated as a new "chat session" and the chatbot won't recall your previous messages.
• Check a couple things:
  • There should be a UserChatSession object for each of your messages, with title=your message (maybe truncated), and userid=None.
  • Check your cookies on the last response. You should see something like this, with an empty ResourceRecommendationBot_ai_thread_id cookie and a ResourceRecommendationBot_ai_threads_anonymous cookie with multiple comma-delimited ids:

• Go to http://ai.open.odl.local:8005/admin. Login through keycloak w/username [email protected] and password student. Then go back to the chat page and send another message.
• Check a couple things:
  • All the UserChatbotSession objects should now have a user (you) associated with them.
  • Check your cookies on the last response. You should see something like this, with ResourceRecommendationBot_ai_threads_anonymous cookie now being empty and ResourceRecommendationBot_ai_thread_id having 1 value.

• Go ahead and revert the change to the page, so history is not cleared on each request. Send a few messages. Check that the cookie value for remains the same now with each response.

Chat history endpoints

• Go to this url, you should see a list of all your chat sessions, from newest to oldest: http://ai.open.odl.local:8002/api/v0/chat_sessions
• If you try that in an incognito browser, you should get a 403. If you try as another user, you should get a 404
• Not the thread_id of the newest session, and go to http://ai.open.odl.local:8002/api/v0/chat_sessions/<threaad_id>. You should be able to edit the title.
• Now go to http://ai.open.odl.local:8002/api/v0/chat_sessions/<threaad_id>/messages. You should see the user and ai agent messages in chronological order.

DRF-spectacular schema

• Go to http://ai.open.odl.local:8002/api/v0/schema/swagger-ui/
• You should see the new API endpoints plus the chatbot endpoints here

Additional Context

• Although Langgraph has a PostgresSaver checkpointer, it is not ORM-friendly, so I created a new AsyncDjangoSaver based on the AsyncRedisSaver implementation described here: https://langchain-ai.github.io/langgraph/how-tos/persistence_redis/

TODO for a subsequent PR:

• The checkpointer creates a lot of rows for each chat (though not nearly as many as langrgaph's own PostgresSaver): one per message, including tool-related messages that the user never sees - so maybe we should consider offloading old (> x months) chats to S3, and import them back into the db if the user ever requests to retrieve them.