Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Continuation] Merge EmbeddedLLM/vllm-rocm into vLLM main #1836

Merged
merged 63 commits into from
Dec 8, 2023
Merged

[Continuation] Merge EmbeddedLLM/vllm-rocm into vLLM main #1836

merged 63 commits into from
Dec 8, 2023

Commits on Oct 10, 2023

  1. port dtype_float16.cuh and cache_kernels.cu

    @pcmoritz
    pcmoritz committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    43af310 View commit details
    Browse the repository at this point in the history

  2. port dtype_bfloat16.cuh

    @pcmoritz
    pcmoritz committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    cc81866 View commit details
    Browse the repository at this point in the history

  3. port attention_utils.cuh

    @pcmoritz
    pcmoritz committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    475b5e2 View commit details
    Browse the repository at this point in the history

  4. port more kernels

    @pcmoritz
    pcmoritz committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    ddc496c View commit details
    Browse the repository at this point in the history

  5. fix typo

    @pcmoritz
    pcmoritz committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    5eaa7a1 View commit details
    Browse the repository at this point in the history

  6. add cuda_compat.h

    @pcmoritz
    pcmoritz committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    f7273c6 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2023

  1. Merge branch 'main' into port-to-rocm

    @pcmoritz
    pcmoritz authored Oct 16, 2023
    Configuration menu
    Copy the full SHA
    99c3be7 View commit details
    Browse the repository at this point in the history

  2. sync branches

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    f8093dc View commit details
    Browse the repository at this point in the history

  3. update

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    41df689 View commit details
    Browse the repository at this point in the history

  4. update

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    93be9c5 View commit details
    Browse the repository at this point in the history

  5. fixes

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    d96fa3c View commit details
    Browse the repository at this point in the history

  6. cleanup

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    421365b View commit details
    Browse the repository at this point in the history

  7. update

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    06b800e View commit details
    Browse the repository at this point in the history

  8. update

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    2312beb View commit details
    Browse the repository at this point in the history

  9. update

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    2958b39 View commit details
    Browse the repository at this point in the history

  10. fmt

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    3f89734 View commit details
    Browse the repository at this point in the history

  11. cleanup

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    5397a57 View commit details
    Browse the repository at this point in the history

  12. refactor

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    90e02d2 View commit details
    Browse the repository at this point in the history

  13. update

    @pcmoritz
    pcmoritz committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    a420202 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2023

  1. Merge branch 'main' into port-to-rocm

    @pcmoritz
    pcmoritz authored Oct 17, 2023
    Configuration menu
    Copy the full SHA
    b072182 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. detecting rocm and adding flag for compiling

    @iAmir97 @pcmoritz
    iAmir97 authored and pcmoritz committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    2d1e435 View commit details
    Browse the repository at this point in the history

  2. using asm volatile instead of hip api

    @iAmir97 @pcmoritz
    iAmir97 authored and pcmoritz committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    e231b79 View commit details
    Browse the repository at this point in the history

  3. using asm volatile for type casting of f16

    @iAmir97 @pcmoritz
    iAmir97 authored and pcmoritz committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    31bb335 View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2023

  1. Hipifying csrc file to accomodate rocm builds

    @kliuae
    kliuae committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    b027d06 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2023

  1. Checked CUDA ROCm Compatibility (#15) 

    Features

* Auto-code path selection
* support llama2
* support squeezellm rocm
* add documentation amd-installation.rst. Describing how to setup vllm ROCm version
* format.sh all the code
* add base amd.Dockerfile

---------

Co-authored-by: tjtanaa <[email protected]>
Co-authored-by: kuanfu <[email protected]>
    @tjtanaa @kliuae
    3 people authored Nov 29, 2023
    Configuration menu
    Copy the full SHA
    9a1781c View commit details
    Browse the repository at this point in the history

  2. merged with latest upstream

    @kliuae
    kliuae committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    0f67117 View commit details
    Browse the repository at this point in the history

  3. format code

    @kliuae
    kliuae committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    7dbf2d4 View commit details
    Browse the repository at this point in the history

  4. downgrade torch requirement in toml to torch 2.0.1 to accommodate ROC… 

    …m support
    @kliuae
    kliuae committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    52ffcf0 View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2023

  1. Merged changes from vllm main

    @kliuae
    kliuae committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    27f0513 View commit details
    Browse the repository at this point in the history

  2. Merged with changes in vllm main

    @kliuae
    kliuae committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    5cce649 View commit details
    Browse the repository at this point in the history

  3. Updated Dockerfile, rocm installation guide and setuppy

    @kliuae
    kliuae committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    16d3ccc View commit details
    Browse the repository at this point in the history

Commits on Dec 2, 2023

  1. Updated amd installation guide and dockerfile

    @kliuae
    kliuae committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    d764f9d View commit details
    Browse the repository at this point in the history

  2. Added num_gpus for ray init in ROCm

    @kliuae
    kliuae committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    e798632 View commit details
    Browse the repository at this point in the history

  3. Synced torch version with vllm main in pyproject.toml

    @kliuae
    kliuae committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    0e8129f View commit details
    Browse the repository at this point in the history

  4. Format code

    @kliuae
    kliuae committed Dec 2, 2023
    Configuration menu
    Copy the full SHA
    2b3821b View commit details
    Browse the repository at this point in the history

Commits on Dec 4, 2023

  1. Merge branch 'main' into vllm-cuda-rocm-dev

    @kliuae
    kliuae committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    0c8795a View commit details
    Browse the repository at this point in the history

  2. Updated dockerfile.rocm and requirements-rocm.txt

    @kliuae
    kliuae committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    5793f30 View commit details
    Browse the repository at this point in the history

  3. Disable mistral for ROCm

    @kliuae
    kliuae committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    b172cdd View commit details
    Browse the repository at this point in the history

  4. Format code

    @kliuae
    kliuae committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    9cd5b18 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2023

  1. Revert to cuda kernels

    @kliuae
    kliuae committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    b86f88a View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'pcmoritz/port-to-rocm' 

    Co-authored-by: Philipp Moritz <[email protected]>
Co-authored-by: Amir Balwel <[email protected]>
    @kliuae @pcmoritz @iAmir97
    3 people committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    9727ab4 View commit details
    Browse the repository at this point in the history

  3. Port latest kernels to ROCm

    @kliuae
    kliuae committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    c4aa2af View commit details
    Browse the repository at this point in the history

  4. Update readme

    @kliuae
    kliuae committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    f8c304e View commit details
    Browse the repository at this point in the history

  5. Cleaned up kernel code

    @kliuae
    kliuae committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    e608c30 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2023

  1. Added wrapper for setting devFuncAttributeMaxDynamicSharedMemorySize

    @kliuae
    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    951e225 View commit details
    Browse the repository at this point in the history

  2. Added wrapper for setting devFuncAttributeMaxDynamicSharedMemorySize

    @kliuae
    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    25f9a97 View commit details
    Browse the repository at this point in the history

  3. Updated ROCm warp size

    @kliuae
    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    e984ada View commit details
    Browse the repository at this point in the history

  4. Format code

    @kliuae
    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    cc1195f View commit details
    Browse the repository at this point in the history

  5. Check hip from wrapper

    @kliuae
    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    f92980e View commit details
    Browse the repository at this point in the history

  6. Format code

    @kliuae
    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    66b4aa1 View commit details
    Browse the repository at this point in the history

  7. Enable support for mistral models

    @kliuae
    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    4a0ecb8 View commit details
    Browse the repository at this point in the history

  8. Fixed hip device attribute

    @kliuae
    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    acf51a8 View commit details
    Browse the repository at this point in the history

  9. Format code

    @kliuae
    kliuae committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    4a52977 View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2023

  1. Restored awq file

    @kliuae
    kliuae committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    23a987a View commit details
    Browse the repository at this point in the history

  2. Format code

    @kliuae
    kliuae committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    8787a4e View commit details
    Browse the repository at this point in the history

  3. Merge latest vllm main

    @kliuae
    kliuae committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    5911131 View commit details
    Browse the repository at this point in the history

  4. Updated rocm dockerfile

    @kliuae
    kliuae committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    9fa8075 View commit details
    Browse the repository at this point in the history

  5. Update amd installation guide

    @kliuae
    kliuae committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    81e052d View commit details
    Browse the repository at this point in the history

  6. Update vLLM Documentations (#18) 

    Update the vLLM installation procedures on AMD platform.
Update vLLM documentations.
    @tjtanaa
    tjtanaa authored Dec 7, 2023
    Configuration menu
    Copy the full SHA
    fb8ac26 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2023

  1. Updated setup.py, vllm/utils.py and amd-installation doc

    @kliuae
    kliuae committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    98f5487 View commit details
    Browse the repository at this point in the history

  2. Updated setup.py

    @kliuae
    kliuae committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    d90187a View commit details
    Browse the repository at this point in the history

  3. Format code

    @kliuae
    kliuae committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    c840531 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into vllm-cuda-rocm-mod

    @kliuae
    kliuae committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    9dba1d8 View commit details
    Browse the repository at this point in the history