-
Notifications
You must be signed in to change notification settings - Fork 187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rocm(7900xtx) GPU fail #3
Comments
Hi, sorry now we are not able to support ROCm devices, since we are using Triton, which only support CUDA. |
Try triton git main branch, latest code supports RDNA3 and CDNA2+3 https://github.com/triton-lang/triton/blob/main/third_party/amd/backend/driver.py#L434 |
@ridgerchu Please add ROCM support for Triton, there's this official repo that supports Triton with ROCM https://github.com/ROCm/triton |
@radna0 I don't have an AMD GPU to test with, but if you do, maybe put in a PR to add ROCM support? |
Will gladly do! |
-Triton-nightly 3.0.0 might anyone be of help regarding this
|
I might be wrong but seems like a driver error...? It seems like it gets all the way to "instantiating the layers", but then it tries to run some sort of benchmark...and then, when it tries to run the kernel in the GPU, it crashes here:
Idk if the changes have been merged upsteam, but are you using the ROCm fork? |
There are 3 versions of Triton that I have tried: -Triton 2.1.0, which is the ROCM/triton repo/branch that you just mentioned @taylor-shift, there are errors regarding the launcher implementation, I think it has not been implemented until Triton 2.2 which is also the required version for the matmulfreellm package -Triton 2.3.1, the latest stable version, I get the same error as @Wintoplay |
@radna0 I only have triton 3 that come with pytorch for Rocm. Btw, I wonder if matmulfreellm can be implemented in Jax efficiently. |
I haven't used Jax, so I don't know, I ran the official rocm-triton docker image and then ran my test cases from there. Still there are some verified concerns from @ridgerchu saying that the current matmulfreellm repo can only do training but not inference, because they still would have to update the package to work with BitBlas, I also have check with the BitBlas team and they said there's a branch that works with HIP |
Thanks for your all attention on BitBLAS dudes :), we've update the citation in our repo:
|
Hey @LeiWang1999, thanks for your info! We will update the reference when next version of our paper released! |
The text was updated successfully, but these errors were encountered: