Whether an artificial reset of the environment is required?

[njchen2002]

When you are updating the parameters, you collect the data first. However, I noticed while running the code that if an environment didn't succeed the last time when it collected data, it didn't start from the initial state the next time. Neither the action history queue nor the observation history queue are empty, would it make more sense to reinitialize them, or does this setup serve some other purpose.