Converting model in order to use it with flexgen #242
Unanswered
Wolvensdale
asked this question in
Q&A
Replies: 1 comment 1 reply
-
You need to activate the conda environment first with |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi, ive been trying to run opt 6.7b models with textgen.
As the wiki stated it needed to be converted first before we could use it with these command :
python convert-to-flexgen.py models/opt-1.3b/
My question is, where exactly to put these command? im not a programmer, so im kinda confused.
I alredy put the command on cmd on the directory that have the convert-to-flexgen.py, but i got this error :
E:\Oobaboga\oobabooga\text-generation-webui>python convert-to-flexgen.py models/opt-6.7b
Traceback (most recent call last):
File "E:\Oobaboga\oobabooga\text-generation-webui\convert-to-flexgen.py", line 12, in
import torch
ModuleNotFoundError: No module named 'torch'
Does this mean i have to install torch?
But torch should have already installed in order for the web ui to run, right?
Beta Was this translation helpful? Give feedback.
All reactions