Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Hardware][CPU] Support chunked-prefill and prefix-caching on CPU #10355

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

[Hardware][CPU] Support chunked-prefill and prefix-caching on CPU #10355

wants to merge 7 commits into from
+529 −365

Commits on Nov 15, 2024

  1. support chunked prefill 

    Signed-off-by: jiang1.li <[email protected]>
    @bigPYJ1151
    bigPYJ1151 committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    5f99876 View commit details
    Browse the repository at this point in the history

  2. rebase 

    Signed-off-by: jiang1.li <[email protected]>
    @bigPYJ1151
    bigPYJ1151 committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    ba2575b View commit details
    Browse the repository at this point in the history

  3. fix test 

    Signed-off-by: jiang1.li <[email protected]>
    @bigPYJ1151
    bigPYJ1151 committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    5980981 View commit details
    Browse the repository at this point in the history

  4. support prefix cache 

    Signed-off-by: jiang1.li <[email protected]>
    @bigPYJ1151
    bigPYJ1151 committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    c45b967 View commit details
    Browse the repository at this point in the history

  5. add tests 

    Signed-off-by: jiang1.li <[email protected]>
    @bigPYJ1151
    bigPYJ1151 committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    46be06a View commit details
    Browse the repository at this point in the history

  6. update doc 

    Signed-off-by: jiang1.li <[email protected]>
    @bigPYJ1151
    bigPYJ1151 committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    d33a175 View commit details
    Browse the repository at this point in the history

  7. fix multi modal chunked prefill 

    Signed-off-by: jiang1.li <[email protected]>
    @bigPYJ1151
    bigPYJ1151 committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    96cafe5 View commit details
    Browse the repository at this point in the history