Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DO NOT MERGE] Rope decomposition by Sergey #27554

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

[DO NOT MERGE] Rope decomposition by Sergey #27554

wants to merge 5 commits into from
+292 −0

Commits on Nov 11, 2024

  1. Draft of GQA decomposition: half/half RoPE, SDPA. Not implemented: re… 

    …al GQA, inplace KV cache, not balanced batch, rope_interleaved etc.
    @slyalin
    slyalin committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    9269453 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2024

  1. Group broadcast is implemented via commonly used UBR-pattern

    @slyalin
    slyalin committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    7eb07a6 View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2024

  1. Correct handling of kv-cache position range (with limitations and ass… 

    …umptions mentioned in the code).
    @slyalin
    slyalin committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    6452b3d View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2024

  1. Fix warning

    @slyalin
    slyalin committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    033ab9d View commit details
    Browse the repository at this point in the history

  2. Another attempt to fix the warning

    @slyalin
    slyalin committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    557a1ba View commit details
    Browse the repository at this point in the history