PPO stage of multi-node training

[tianbuwei]

Thank you very much for your work. I need to consult you about a problem in the process of PPO training of 8B model. Showing Evaluation responses not received in the log indicates that the training was incorrect. This sentence sometimes shows, sometimes does not show, this is normal?

[vwxyzjn]

Yeah this is expected. We only run evaluations --num_evals times, so in other iterations we jus say evaluation response is not received and not log them.