Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[linalgExt::Attention] add AttentionOP boolean useExp2 to choose either exp or exp2 #19820

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

[linalgExt::Attention] add AttentionOP boolean useExp2 to choose either exp or exp2 #19820

wants to merge 4 commits into from

Conversation

ita9naiwa
Copy link
Member

@ita9naiwa ita9naiwa commented Jan 27, 2025 •
edited
Loading

#18932

  • Add useExp attribute to use exp_e or exp_2
    @Groverkss I created separated PR here,

@ita9naiwa
add exp
76a805b 
Signed-off-by: Hyunsung Lee <[email protected]>
@ita9naiwa
lint
f3ce0d5 
Signed-off-by: Hyunsung Lee <[email protected]>
@ita9naiwa ita9naiwa force-pushed the ita9naiwa/att-logexp branch from 8c25391 to f3ce0d5 Compare January 27, 2025 12:25
@hanhanW hanhanW requested a review from Groverkss January 28, 2025 07:18
@ita9naiwa
Copy link
Member Author

@Groverkss Hi, any updates?

Groverkss
Groverkss reviewed Feb 24, 2025
View reviewed changes
Copy link
Contributor

@Groverkss Groverkss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good mostly. Can you also change the torch lowering to set this flag and verify if this flag actually gets propagated until the end?

@ita9naiwa
add Attention TMTensorToLinalgExt Lowering
fb114fa 
Signed-off-by: Hyunsung Lee <[email protected]>
@ita9naiwa
lint
a1f9be1 
Signed-off-by: Hyunsung Lee <[email protected]>
@ita9naiwa ita9naiwa force-pushed the ita9naiwa/att-logexp branch from 8703ce5 to a1f9be1 Compare February 25, 2025 00:03
@ita9naiwa
Copy link
Member Author

Hi, I added fix and tests to see it's propagated from TMTensor to LinalgExt.

btw, I don't see any that uses use_exp2 or something like that from torch.aten.scaled_dot_attention?

@ita9naiwa ita9naiwa requested a review from Groverkss February 25, 2025 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hanhanW hanhanW Awaiting requested review from hanhanW hanhanW is a code owner

@MaheshRavishankar MaheshRavishankar Awaiting requested review from MaheshRavishankar MaheshRavishankar is a code owner

@Groverkss Groverkss Awaiting requested review from Groverkss

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ita9naiwa @Groverkss