-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fail to run examples/offline.py , unable to download the model to reproduce #35
Comments
is there alternative way to provide the model except for downloading online directly |
You can use this website to download model:https://modelscope.cn/my/overview |
thanks , I have downloaded it manually through meata.com, and convert it into hf format using convert_llama_weights_to_hf.py, and got the files as following: root@d7b9ced7ced8:/workspace/DistServe# ls -lt ../lama2-7b-hf/ but when I started it with : it report another issue:
|
I encountered the same question before. I can tell you that is because distserver needs to convert the model into another form first. I changed the file distserve/downloader/downloader.py to solve this. You can replace the same part in this file. Here's my code:
|
but I don't have the .bin file in the folder , only got these files:
|
https://modelscope.cn/models/shakechen/Llama-2-7b-hf/files It has *.bin files, maybe download it again? |
Hi Robert, Is it necessary to download all the files ? it will take too much time to download all. |
I'm not sure, but according to the code, *.bin is necessary. You can use --model to specify the local dir, like
|
Hi Robert , |
I am able to run the examples/offline.py code, got following result :
but I'm still confused :
|
Hi mate, have you ever met this problem: (ParaWorker pid=1955026) Error: Peer-to-peer access is unsupported on this platform. I checked the P2P access, it should be supported actually... Thank you for any help! |
you can use below command to check your system if it's support P2P: Legend: X = Self |
Thank you for response, I checked this, seems that the framework cannot run
on 4090 machines, I ran it successfully on A100 machines.
…On Tue, 13 Aug 2024 at 02:54, William12github ***@***.***> wrote:
you can use below command to check the your if it's support P2P:
`nvidia-smi topo -p2p wr
GPU0 GPU1 GPU2 GPU3 GPU4 GPU5 GPU6 GPU7
GPU0 X OK OK OK OK OK OK OK
GPU1 OK X OK OK OK OK OK OK
GPU2 OK OK X OK OK OK OK OK
GPU3 OK OK OK X OK OK OK OK
GPU4 OK OK OK OK X OK OK OK
GPU5 OK OK OK OK OK X OK OK
GPU6 OK OK OK OK OK OK X OK
GPU7 OK OK OK OK OK OK OK X
Legend:
X = Self
OK = Status Ok
CNS = Chipset not supported
GNS = GPU not supported
TNS = Topology not supported
NS = Not supported
U = Unknown`
—
Reply to this email directly, view it on GitHub
<#35 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AUK4KHT3I7DBMZI3VJV62RTZRFRPFAVCNFSM6AAAAABMBIF2RSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBVGE4TKNBRGM>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Hi ,
I am trying to reproduce the result, but it's unable to download the llama2-7b-hf model as below logs printed,
`root@d7b9ced7ced8:/workspace/DistServe# python3 examples/offline.py
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_errors.py", line 304, in hf_raise_for_status
response.raise_for_status()
File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://huggingface.co/meta-llama/Llama-2-7b-hf/resolve/main/config.json
The above exception was the direct cause of the following exception:
Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-hf/resolve/main/config.json.
Your request to access model meta-llama/Llama-2-7b-hf has been rejected by the repo's authors.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/workspace/DistServe/examples/offline.py", line 32, in
model_config=ModelConfig(
File "/workspace/DistServe/distserve/config.py", line 177, in init
self.hf_config = self._get_hf_config()
File "/workspace/DistServe/distserve/config.py", line 192, in _get_hf_config
raise ValueError(
ValueError: Failed to load the model config, please check the model name or path: meta-llama/Llama-2-7b-hf`
, although I login into successfully huggingface-cli login, is there alternative way to acquire the model? thanks
Token has not been saved to git credential helper. Your token has been saved to /root/.cache/huggingface/token Login successful
The text was updated successfully, but these errors were encountered: