Resolving Accuracy Issues in Chinese Output for Florence-2 Fine-Tuned with LoRA

[Dqiqi123]

Using the llava-instruct-chinese dataset, the image encoder weights are frozen, and the language part of Florence-2 is fine-tuned using the LoRA method. While performing the "CAPTION" task, the model is capable of outputting in Chinese, but the accuracy of the answers is zero. How can this issue be resolved?