Long Term Memory extension

[wawawario2]

EDIT: As a quick heads up, the repo has been converted to a proper extension, so you no longer have to manage a fork of ooba's repo.

So I'm working on a long-term memory module. Right now, I'm using this UI as a means to field-test it and make improvements, but if there's any interest in merging this module directly into this repo, I can align some of my priorities accordingly (mainly adding some features necessary to make it usable for a general-purpose audience). It's currently a decoupled extension so it should have minimal, if any, impact on the rest of the repo. The current implementation is detailed in the readme. I'm open to any suggestions.

[kreolsky]

Wow! It's looks ineteresing and great feature for chat mode!

[USBhost]

@wawawario2 I have a question: So does LTM only work the next time you load chat? Can this function as additional context? Or the only way to use LTM is to keep reloading your character?

[wawawario2]

Yes, but you can override this behavior by pressing a button in the UI and force-load memories from disk to RAM. This was a deliberate design decision for a couple of reasons, but I'm open to changing this, especially if it will improve the user experience.

1. From experimentation I've seen that if we make long-term memory immediately visible to the chatbot, there can be a tendency to load very recently-stored memories, which would basically defeat the spirit of long-term memory, even if said memories are just outside the context window. That said, I can see a use case for this for much longer chat sessions.
2. It keeps the implementation simpler on the backend. That said, it's not super-hard to have this module support it.

It doesn't expand the size of the context window, but it does enable context to automatically transcend just the most recent 2048 (or however many) tokens by looking for semantically similar conversations in the past. It essentially tries to make "better" use of the limited window we have.

[USBhost]

1: To resolve this could we add message priority based on LTM age?
This would be a cool way to give the bot a little more memory. Kind of feels like Kobold world info but dynamic on what's being talked about.

Can you load character info into LTM and free up some context space?

[wawawario2]

I have some ideas in mind for how to implement a notion of priority, I can experiment with that. As for storing character traits into LTM that's actually a very good point you bring up, there's a lot of potential with having some "fixed" LTM storage that stores much more detailed character profile info that can be dynamically looked up as-needed.

[oobabooga]

By merging the extension here, you end up losing your ability to easily modify it. Since this is a very complex extension in active development, I encourage you to convert your repository into something that can be installed like this:

cd text-generation-webui/extensions
git clone https://github.com/wawawario2/long_term_memory

I have added a mention to your extension in the wiki: https://github.com/oobabooga/text-generation-webui/wiki/Extensions

Do you want me to merge the changes to modules/chat.py?

[wawawario2]

Sounds good and sure, if you merge those changes, that'll make this extension completely decoupled and make things super easy. Thank you for the mention!

[oobabooga]

Done here: fcda3f8

[tensiondriven]

@wawawario2 This is great; I haven't looked at the code yet, can you describe how it works?

Also, if you open up the issues queue on your repro, I'd gladly test and possibly contribute.

Lack of long term memory (be it through context, lora, softprompt, or some other voodoo) is the biggest thing holding these bots back from becoming parts of our lives (at least, for the very bleeding edge early adopters like us). I look forward to this! Let me know if I can help!

[wawawario2]

Good to hear! I plan to expand the descriptions of how things works under the hood in the readme at some point. I opened up issues and discussions.

[tensiondriven]

This is so great, this link describes how the LTM module works under the hood.

[bmoconno]

Yeah, this is very cool. I'm excited to see where this goes. Thanks for your efforts so far.

[oobabooga]

I think that this commit broke the extension: e722c24

The fix should be simple. I will also need to add a "continue" parameter to generate_chat_prompt later.

[BarfingLemurs]

I never realized this extension was the vector embedding storage system.
It seems to still be updated, great news.

Could I feed an entire github repository with it's various files, and query related lines using the extension? that would be magic, I never really played with chatgpt and its vector embedding storage so I don't really have a feel to its effectiveness for code.

[Felixqian4160]

This is a really great feature, I hope it can be quickly integrated into the code.

[BarfingLemurs]

it already is. just install it and all the features work.

[Felixqian4160]

thank you

[mykeehu]

I installed this extension and it worked fine for two days, but then my models started to go crazy. I don't know why, I switched to any model, got irrelevant responses, mixed up the persons in the responses, or the Chronos model put in a specific detail from LTM in one of the responses (with syntax!)
So now I turned it off, but is it possible some incompatibility with the new webui changes? It was quick, I was happy to use it, but I can't turn it on for now.

[wawawario2]

Hey all, as a heads up this extension is no longer in development. Anyone still using this extension should migrate. I really appreciate all the support over the past few months.

[TomLucidor]

What is the relationship and differences between RAG ("just read my files") vs long-term memory? Is it merely a coherence issue?

[pay-tracker]

Hey wawawario, I've really been meaning to get in contact with you to ask a simple question about your extension. I want to know if I edit long_term_memory.db with DataBase Browser for SQLite (as I have been able to), will the embeddings be automatically updated or will it still use the old embeddings? I want to retroactively change memories as we are trying to implement oobabooga in a small business setting with business reports as memories, and it would be much easier than re-submitting all of the memories in order to change the contents of one of them.