What just happend here? It simulates a human talking with himself :o

[Username69992872]

Can anyone explain what just happend? Looks like it simulates a human talking to himself ?

[mcmonkey4eva]

Are you using a custom trained model like Vicuna? Several models like that are hard-trained with the format of

Human: input
Assistant: output

Which means, because your name is set as You, you've broken the expected format the model demands. It can sometimes work within this anyway, but it was literally trained to expect a Human:, and since you're not providing one, it can end up generating one itself.

Your options to prevent this are

(A): use a model that isn't trained with such a strict format
(B): under parameters check the Stop at newlines option
(C): Change your name to Human as it expects. This both means that you'll be obeying the format rules, and it means that the webui can autodetect if the AI tries to generate on your behalf and will stop it.
(D): Set up a good character file (the template/placeholder 'Assistant' is just a placeholder, you're not meant to rely on it). LLMs will always try to follow the character file's example dialogue format, and that may suffice to override the model's format training -- and if you see it doing the format break, stop & regenerate (once it does it once in chat history, it'll repeat, so you have to make sure to cut it out of chat history)
If you don't have or don't want to use a proper character file, there are new default included character files that serve as close-enough templates for some of the common model variants (alpaca/vicuna/etc)

EDIT: (E): this PR: #903 adds custom stop words that will also serve as a solution here

---

Background info

LLMs, at their fundamental core design, just take some initial text, and try to predict what the next piece of text is. So if the input is a conversation, the next text is more conversation. They almost always want to generate multiple replies like that (except when they've been trained to output end-of-stream after specific reply lines). It's a software toolset that stops it, specifically by detecting 'stop words'. In the case of cai-chat mode, the default stop detector is if the AI generates a newline + your name + a : - ie, it's trying to generate a reply from you. This isn't able to stop the model from generating replies from other people as the code doesn't know to look for that. The stop at newlines options adapts the stop trigger to just, any newlines, which hardblocks any form of additional reply, but has the side effect that if the AI's message includes a newline for more-legitimate reasons, it'll be cut unnecessarily short.