Request: A termination/exit/shutdown command

[mirshalak]

Apologies if this already exists, but I was wondering if any explicit means of exiting exists within Kobold, and if not, if one could be added. Maybe I'm just being paranoid, but I know that rebooting a computer without cleanly dismounting can lead to orphaned filesystem inodes and potential corruption. For an analogous reason, I always feel a vague sense of unease when I just send SIGKILL to a Kobold process whenever I've finished using it.

If you could write a small handler function for SIGTERM and its' Windows equivalent, (maybe cleanly unloading the language model, if appropriate) it would be appreciated. Aside from anything else, in Windows I did notice variable loading times for Kobold if I was using it repeatedly without rebooting the system, so that could ensure that memory was being completely freed when Kobold terminates.

[LostRuins]

Memory definitely is completely freed on process exit - the reason why it starts faster again on the next run is probably something to do with caching. You can check in task manager - when the process exits VRAM should drop back to 0 and RAM should be back to where it was before.

KoboldCpp usually runs in readonly mode except if you output benchmark to file - so there should not be any risk of corruption as there is no writes to disk done at all.

Hmm, if you're running from the terminal sigterm should be correctly handled, there is code in the python script to cleanup and shutdown the process. Is it not working? For windows, close the terminal window should have the same effect.

[mirshalak]

OK, I'm probably just being paranoid then. Never mind. :)