Issue with Model "lmsys_vicuna-13b-v1.5" and Oobabooga (CPU)

[lucasuniverse]

Issue with Model "lmsys_vicuna-13b-v1.5" and Oobabooga

I'm in urgent need of your assistance. I'm trying to utilize the "lmsys_vicuna-13b-v1.5" model, and upon installing the Oobabooga web interface, I opted not to use the GPU, choosing the "N" option to use only the CPU. After a successful installation, I attempted to use the model. I managed to use the "Model loader=Transformers" option with the CPU selected. However, even though it functions, the performance is extremely slow. CPU usage seems to be almost negligible, with hardly any perceptible activity, and responses take a long time to arrive, being very sluggish.

Error When Attempting to Use "llama.cpp"

Subsequently, I tried to use "llama.cpp", selecting the "CPU" option, but encountered the following error:

arduin
Traceback (most recent call last):
File "C:\AI\webUI\oobabooga\text-generation-webui-main\modules\ui_model_menu.py", line 245, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
File "C:\AI\webUI\oobabooga\text-generation-webui-main\modules\models.py", line 87, in load_model
output = load_func_maploader
File "C:\AI\webUI\oobabooga\text-generation-webui-main\modules\models.py", line 247, in llamacpp_loader
model_file = list(Path(f'{shared.args.model_dir}/{model_name}').glob('*.gguf'))[0]
IndexError: list index out of range

Certainly! Here's your text organized and translated into English:

Issue with Model "lmsys_vicuna-13b-v1.5" and Oobabooga

I'm in urgent need of your assistance. I'm trying to utilize the "lmsys_vicuna-13b-v1.5" model, and upon installing the Oobabooga web interface, I opted not to use the GPU, choosing the "N" option to use only the CPU. After a successful installation, I attempted to use the model. I managed to use the "Model loader=Transformers" option with the CPU selected. However, even though it functions, the performance is extremely slow. CPU usage seems to be almost negligible, with hardly any perceptible activity, and responses take a long time to arrive, being very sluggish.

Error When Attempting to Use "llama.cpp"

Subsequently, I tried to use "llama.cpp", selecting the "CPU" option, but encountered the following error:

arduino
Copy code
Traceback (most recent call last):
File "C:\AI\webUI\oobabooga\text-generation-webui-main\modules\ui_model_menu.py", line 245, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
File "C:\AI\webUI\oobabooga\text-generation-webui-main\modules\models.py", line 87, in load_model
output = load_func_maploader
File "C:\AI\webUI\oobabooga\text-generation-webui-main\modules\models.py", line 247, in llamacpp_loader
model_file = list(Path(f'{shared.args.model_dir}/{model_name}').glob('*.gguf'))[0]
IndexError: list index out of range
Device Specifications

My device specifications are as follows:

Device Name: DESKTOP-KPBBH01
Processor: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz (2 processors)
Installed RAM: 128 GB
System Type: 64-bit operating system, x64-based processor
Edition: Windows 10 Pro
Version: 22H2
OS Build: 19045.4170
Experience: Windows Feature Experience Pack 1000.19054.1000.0