How to Run Llama-7B 4-bit Model on CPU

[janpashashaik123]

Hi,

I am trying to run (inference) llama-7b 4-bit Model in my local Ubuntu system in CPU without GPU. But facing an error with quant_cuda (NameError: name 'quant_cuda' is not defined). Can the llama-7b 4-bit model be run on the CPU?