Skip to content

Commit

Permalink
fix(ray): fix gpu resource > 1 (#91)
Browse files Browse the repository at this point in the history
Because

- GPU resource quantities > 1 must be whole numbers.

This commit

- apply math.ceil to vram usage ratio
  • Loading branch information
heiruwu authored Jan 26, 2024
1 parent 2ea4737 commit 9ac7241
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 8 deletions.
5 changes: 3 additions & 2 deletions instill/helpers/ray_config.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import math
import os
from typing import Callable, Optional

Expand Down Expand Up @@ -54,13 +55,13 @@ def _determine_vram_usage(self, model_path: str, vram: str):
MINIMUM_VRAM_RESERVE,
1.1 * os.path.getsize(model_path) / (1024 * 1024 * 1024),
)
return min_vram_usage / float(vram)
return math.ceil(min_vram_usage / float(vram))
if os.path.isdir(model_path):
min_vram_usage = max(
MINIMUM_VRAM_RESERVE,
1.1 * get_dir_size(model_path) / (1024 * 1024 * 1024),
)
return min_vram_usage / float(vram)
return math.ceil(min_vram_usage / float(vram))
raise ModelPathException

def _determine_ram_usage(self, model_path: str):
Expand Down
11 changes: 5 additions & 6 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 9ac7241

Please sign in to comment.