Fix (examples/llm): Fix infinite loop in LLM entrypoint with WikiText2 #1044
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix for issue #1036. When fuse_sequences is False, and the seqlen is greater than 842 (maximum sequence length in WikiText2), the condition in line 131 of optimum/amd/brevitas/data_utils.py always evaluates to True, resulting in an infinite loop (see commit d0d06070225804dc6fecde00911ae38b53080123 in optimum-amd).
Consequently, we opted for enabling passing fuse_sequences as an argument and warning the user, when fuse_sequences=False, than an overly large seqlen can result in the infinite loop indicated in issue #1036.