Can we add gptx/vicuna/alpaca models to the model download.py file?

[gsgoldma]

there's a lot of running around finding different models, it would be nice if they were all available from that list

[pcrii]

The script should download anything you find on hugging face press the copy model name button. Get something like this "anon8231489123/vicuna-13b-GPTQ-4bit-128g" to use with the script
#python download-model.py organization/model
python download-model.py anon8231489123/vicuna-13b-GPTQ-4bit-128g

This example had a mismatched file name though, the .safetensors file needs renamed to match the folder I also had to run it with --model-type llama

You can even feed download-model.py a Lora link and it'll detect that and place it where it needs to be. As long as it's on hugging face download-model.py should support it

Ps. LLaMA is a leak. It's supposed to be for researchers that have been approved by Meta. That's probably why it and it's derivatives are mostly a find it yourself kind of thing

[gsgoldma]

that's a switch for running the model, not for the download script

i see, thank you

but I still get the same error when using it with server.py as well.

"This example had a mismatched file name though, the .safetensors file needs renamed to match the folder I also had to run it with --model-type llama"

what did the safetensors file name need to be changed to?

[pcrii]

the folder needs to match the model this is what mine looks like now after fixing
G:\My Drive\text-generation-webui\models\vicuna-13b-GPTQ-4bit-128g\vicuna-13b-GPTQ-4bit-128g.safetensors
originally one of the names had GPTQ and other didnt. doesnt matter if you add QPTQ or remove it as long as folder name matches file name

[gsgoldma]

thanks for clarifying what that is about, pcrii. Another problem with this is, there seems to be .pt files, .bin files, and .safetensors. on top of that, some have the .bin as one file, and others use several to dozens of .bin files. is there any good way to understand which is for which and why?

[pcrii]

when the repository isnt ideal (like anon8231489123) use --text-only read the repo then manually choose and download the right model and put in its folder. in the case of vicuna rename the file as well

most models work well with the script maybe anon intentionally does it for the lulz :)
its all containers idk why they might prefer one over the other but i think sharded models (1 of 6) are easier for people to download and maybe a little easier for systems to load too, not sure

python download-model.py anon8231489123/vicuna-13b-GPTQ-4bit-128g --text-only
wget https://huggingface.co/anon8231489123/vicuna-13b-GPTQ-4bit-128g/resolve/main/vicuna-13b-4bit-128g.safetensors
mv vicuna-13b-4bit-128g.safetensors {model_dir}/vicuna-13b-GPTQ-4bit-128g/vicuna-13b-GPTQ-4bit-128g.safetensors

[OWKenobi]

--model-type llama is no longer required, as long as it has "vicuna" in the folder name.