Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run tests and examples on CUDA #251

Merged
merged 11 commits into from
Jul 23, 2024
Merged

Run tests and examples on CUDA #251

merged 11 commits into from
Jul 23, 2024

Conversation

dacorvo
Copy link
Collaborator

@dacorvo dacorvo commented Jul 22, 2024 •
edited
Loading

What does this PR do?

This pull-request:

  • makes sure that unit tests are actually run on CUDA devices (they were previously skipped).
  • disables TinyGemm kernels when CUDA is < 12.1.
  • modifies the CI to run examples on both CUDA and CPU,
  • disables AWQ Kernels if output features are < 128,
  • adds text-to-image example to the CI.

@dacorvo dacorvo force-pushed the cuda_examples branch 5 times, most recently from 78a7c07 to b95ad45 Compare July 22, 2024 15:17
@dacorvo
fix(awq): disable AWQ kernel if output features < 128
6f4610e 
The GEMM kernel crashes if output features are less than 128. It is
unclear if the issue comes from:
- the kernel itself,
- a wrong packing/formatting of the inputs (padding might be required),
- a wrong evaluation of the kernel parameters.
As a workaround, both GEMM and GEMV kernels are disabled if the number
of output features is less than 128.
Eventually, only the unpacking code should remain in quanto, and the
AWQ kernels should be replaced by the latest MARLIN kernels that are
better maintained.
@dacorvo dacorvo changed the title Run examples on CUDA Run tests and examples on CUDA Jul 22, 2024
@dacorvo dacorvo force-pushed the cuda_examples branch 2 times, most recently from 001b031 to c8d3910 Compare July 22, 2024 16:22
@dacorvo
fix(tinygemm): requires CUDA >= 12.1
9278fc4
@dacorvo
refactor(library): rename ext module to extensions
e965f10
@dacorvo
test(library): add a test to verify extensions compilation
812193d
@dacorvo
ci: install package in editable mode
df43e37 
We need to install the package in editable mode, because otherwise
extension files are not properly installed.
This probably comes from a wrong behaviour of `setuptools_scm`, since
it is in charge of detecting package assets automatically.
@dacorvo
ci(cuda): bump pytorch version
febdf2c
@dacorvo
ci: rename examples worlflow
c117532
@dacorvo
ci(cuda): make CUDA device visible
104d7c1 
Without explicitly binding the CUDA device to the CI container, all
unit tests running on the CUDA device were previously skipped.
@dacorvo
ci: run Linux examples on CUDA
1c659eb
@dacorvo dacorvo marked this pull request as ready for review July 23, 2024 11:28
@dacorvo dacorvo merged commit f61768c into main Jul 23, 2024
13 checks passed
@dacorvo dacorvo deleted the cuda_examples branch July 23, 2024 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@dacorvo