Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Commit

Permalink
Lower unstructured sparsity threshold to 40% (#100)
Browse files Browse the repository at this point in the history
SUMMARY:
"please provide a brief summary"

TEST PLAN:
"please outline how the changes were tested"
  • Loading branch information
mgoin authored Mar 7, 2024
1 parent 3e9ba20 commit 8d617e5
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions vllm/model_executor/layers/parameters/lazy_compressed.py
Original file line number Diff line number Diff line change
Expand Up @@ -120,8 +120,8 @@ def compress(self) -> None:
sparsity = 1 - (torch.count_nonzero(self.uncompressed_data).item()
/ numpy.prod(self.shape))

# Only compress if we have sufficient sparsity (>=45%)
if sparsity < 0.45:
# Only compress if we have sufficient sparsity (>=40%)
if sparsity < 0.4:
logger.warning(
f"Called compress() on tensor of shape {self.shape} but only has "
f"{sparsity:.2}% sparsity, skipping compression")
Expand Down

0 comments on commit 8d617e5

Please sign in to comment.