Fine-Tuning on a Small Dataset

[said702]

First of all, thank you for your excellent work and for providing this repository. It is a great resource!

I am currently trying to fine-tune the English model using a small dataset with 200 samples to improve its performance on specific words that are not generated correctly. However, after fine-tuning the model for a few epochs, the generated images are almost completely black and unrecognizable.

Is this issue caused by using too few samples?

Thank you in advance for your guidance!

[dailenson]

Hi, Thanks for your attention! Before fine-tuning, you could try to turn down the learning rate by 10 times.

[said702]

Thank you very much for the quick response! One more question, please: The English One-DM model that you kindly released was trained on both the CVL and IAM datasets, correct?