1080ti enough? (11 GB GDDR5X memory)

[bonecast]

i get the not enough memory error on a 1080ti.
trying to run https://huggingface.co/anon8231489123/vicuna-13b-GPTQ-4bit-128g/tree/main
followed this tutorial: https://www.youtube.com/watch?v=cCQdzqAHcFk

[PurpleOm]

Done some more looking and with for bitsandbytes you will be able to run 8bit if you want with a 1080ti, but 4bit you need compute above 7. 1 or 7.6
My 3060 has this displayed "Highest compute capability among GPUs detected: 8.6" the 3060 has faster memory, the 1080ti mem is a bit slower

[GrandaddyShmax]

strange, i also have 1080ti and have no such issue.
running gpt4-x-alpaca-13b-native-4bit-128g
with the following --auto-devices --chat --wbits 4 --groupsize 128 --pre_layer 30

[PurpleOm]

i didnt use --pre_layer 30, but i tried it with my 1080ti in combination with my 3060 and it told me my gpu compute was not enough and i could only use 8bit. Took the 1080ti out and it worked.