Finetune Qwen2-VL-7B-Instruct

[kiennt120]

Anyone successful finetune Qwen2-VL-7B-Instruct. I finetuned with 2 card A00 and got error CUDA OOM. Can you share which resources recommend to finetune it?
Thank you!

[zhangfaen]

reduce batch_size or decrease numbers of this part in util/vision_util.py

IMAGE_FACTOR = 28
MIN_PIXELS = 4 * 28 * 28
MAX_PIXELS = 16384 * 28 * 28
MAX_RATIO = 200

VIDEO_MIN_PIXELS = 128 * 28 * 28
VIDEO_MAX_PIXELS = 768 / 4 * 28 * 28
VIDEO_TOTAL_PIXELS = 24576 / 4 * 28 * 28
FRAME_FACTOR = 2
FPS_MIN_FRAMES = 4
FPS_MAX_FRAMES = 768 / 4

[zhangfaen]

or try

replace from torch.optim import AdamW with from torch.optim import SGD

[xyjigsaw]

It seems that 7B models will encounter OOM issues.