You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
You only need to modify the GPU configuration in the DeepSpeed launch scripts for pretraining and finetuning. For example, change deepspeed --include localhost:4,5,6,7 in pretrain.sh to deepspeed --include localhost:0,1,2,3,4,5,6,7.
Hi, what do I need to change the code if I want to parallelize the computation with 8 gpu's
The text was updated successfully, but these errors were encountered: