ImportError: Using bitsandbytes 8-bit quantization requires Accelerate

[nicolai256]

(qloravenv) C:\deepdream-test\llm_qlora-main>python train.py C:\deepdream-test\llm_qlora-main\configs\mistralaiMistral-7B-v0.1.yaml
Load base model
Traceback (most recent call last):
File "C:\deepdream-test\llm_qlora-main\train.py", line 25, in
trainer.load_base_model()
File "C:\deepdream-test\llm_qlora-main\QloraTrainer.py", line 37, in load_base_model
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map={"":0})
File "C:\deepdream-test\llm_qlora-main\qloravenv\lib\site-packages\transformers\models\auto\auto_factory.py", line 561, in from_pretrained
return model_class.from_pretrained(
File "C:\deepdream-test\llm_qlora-main\qloravenv\lib\site-packages\transformers\modeling_utils.py", line 3024, in from_pretrained
hf_quantizer.validate_environment(
File "C:\deepdream-test\llm_qlora-main\qloravenv\lib\site-packages\transformers\quantizers\quantizer_bnb_4bit.py", line 62, in validate_environment
raise ImportError(
ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes

I did follow the instructions of the import error but no difference..

[nicolai256]

I also tried testing if accelerate works and it does..

I am running on a windows 10 machine with a 3090

(qloravenv) C:\deepdream-test\llm_qlora-main>accelerate test

Running:  accelerate-launch C:\deepdream-test\llm_qlora-main\qloravenv\lib\site-packages\accelerate\test_utils\scripts\test_script.py
stderr: The following values were not passed to `accelerate launch` and had defaults used instead:
stderr:         `--num_processes` was set to a value of `0`
stderr:         `--num_machines` was set to a value of `1`
stderr:         `--mixed_precision` was set to a value of `'no'`
stderr:         `--dynamo_backend` was set to a value of `'no'`
stderr: To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
stdout: **Initialization**
stdout: Testing, testing. 1, 2, 3.
stdout: Distributed environment: NO
stdout: Num processes: 1
stdout: Process index: 0
stdout: Local process index: 0
stdout: Device: cpu
stdout:
stdout: Mixed precision type: no
stdout:
stdout:
stdout: **Test process execution**
stdout:
stdout: **Test split between processes as a list**
stdout:
stdout: **Test split between processes as a dict**
stdout:
stdout: **Test split between processes as a tensor**
stdout:
stdout: **Test random number generator synchronization**
stdout: All rng are properly synched.
stdout:
stdout: **DataLoader integration test**
stdout: 0 tensor([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
stdout:         18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31]) <class 'accelerate.data_loader.DataLoaderShard'>
stdout: Non-shuffled dataloader passing.
stdout: Shuffled dataloader passing.
stdout: Non-shuffled central dataloader passing.
stdout: Shuffled central dataloader passing.
stdout:
stdout: **Training integration test**
stdout: Model dtype: torch.float32, torch.float32. Input dtype: torch.float32
stdout: Model dtype: torch.float32, torch.float32. Input dtype: torch.float32
stdout: Training yielded the same results on one CPU or distributed setup with no batch split.
stdout: Model dtype: torch.float32, torch.float32. Input dtype: torch.float32
stdout: Training yielded the same results on one CPU or distributes setup with batch split.
stdout: BF16 training check.
stdout: Model dtype: torch.float32, torch.float32. Input dtype: torch.float32
stdout: Model dtype: torch.float32, torch.float32. Input dtype: torch.float32
stdout: Model dtype: torch.float32, torch.float32. Input dtype: torch.float32
stdout: Training yielded the same results on one CPU or distributed setup with no batch split.
stdout: Model dtype: torch.float32, torch.float32. Input dtype: torch.float32
stdout: Training yielded the same results on one CPU or distributes setup with batch split.
stdout: BF16 training check.
stdout: Model dtype: torch.float32, torch.float32. Input dtype: torch.float32
stdout:
stdout: **Breakpoint trigger test**
Test is a success! You are ready for your distributed training!

[georgesung]

In python if you try import accelerate does that work?