Refactoring: turn the attribute _return_attention_scores
into an argument
#20803
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have a small refactoring for the class
MultiHeadAttention
👾I propose to replace the private attribute
_return_attention_scores
with an extra argument in_compute_attention
.This would make subclassing more straightforward and avoid cases where the attention scores are
None
.This issue shows in
CachedMultiHeadAttention
on KerasHub.The rationale is explained in more details in #20802