Skip to content

Commit

Permalink
[bugfix] fix RuntimeError on apc (#648)
Browse files Browse the repository at this point in the history
This PR fixes a bug that results in the following RuntimeError when APC
is enabled.
```
ERROR 12-19 02:30:05 engine.py:140]   File "/workspace/vllm/worker/hpu_model_runner.py", line 854, in _prepare_prompt
ERROR 12-19 02:30:05 engine.py:140]     if prefix_block_list_tensor:
ERROR 12-19 02:30:05 engine.py:140] RuntimeError: Boolean value of Tensor with more than one value is ambiguous
```
  • Loading branch information
kkimmk authored Jan 7, 2025
1 parent 9d6917f commit 5d582b5
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion vllm/worker/hpu_model_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -974,7 +974,7 @@ def _prepare_prompt(
# Note: num_prefill_tokens is calculated using the length of
# input_tokens after padding.
num_prefill_tokens = input_tokens_tensor.numel()
if prefix_block_list_tensor:
if prefix_block_list_tensor is not None:
prefix_block_list_tensor = prefix_block_list_tensor.to(
self.device, non_blocking=True)
input_tokens_tensor = input_tokens_tensor.to( # type: ignore
Expand Down

0 comments on commit 5d582b5

Please sign in to comment.