Commercial Use of Text Generation WebUI for High-Concurrency Applications

[SpaceDandy13]

Hello everyone,

I'm currently exploring the feasibility of deploying a text generation WebUI for commercial use and have a couple of questions regarding its licensing and performance capabilities.

Licensing Concerns with AGPL-3.0: I plan to deploy my own model as an API service without modifying the original WebUI code. Given that the WebUI is licensed under AGPL-3.0, it should be ok?

Performance for High-Concurrency: The service needs to support over 100 concurrent users. Does the Text Generation WebUI have the capability to handle such a level of concurrency efficiently, or would I be better off directly deploying the model using a more specialized framework like VLLM? Additionally, how does its performance compare to using llama.cpp directly?

I'm new to large language models, any related information would be appreciated!

[ZiyaCu]

Hi,
Instead of asking to ooba, you can check the Gnu's FAQ section about gpl licenses (includes agpl3) . Here is a link:
https://www.gnu.org/licenses/gpl-faq.en.html#GPLRequireSourcePostedPublic
Hope this helps :)