Reinforcement Learning, stable-baselines

[jgerstmayr]

Discussion on how to use exudyn's artificialIntelligence module.

[jgerstmayr]

I am currently working on the implementation of the state2InitialValues() class method. Since I have formulated my mobile robot with redundant coordinates, among other things, I want to make sure that I have understood the concept correctly. Are the return values of State2InitialValues(), namely [initialValues, initialValues_t], only “manually” calculable - similar to the example at https://github.com/jgerstmayr/EXUDYN/blob/master/docs/RST/Examples/openAIgymTriplePendulum.rst?

Furthermore, I would like to confirm whether my assumption is correct that initialValues must have the same size as the vector mbs.systemData.GetODE2Coordinates()?

[jgerstmayr]

There 2 things to know:

1. you can select which states are available to the RL-method, they do not need to represent the full state of your system in mbs.systemData.GetODE2Coordinates(). However, you need to make sure that enough information is provided, in particular relevant positions as well as velocities.
2. The function State2InitialValues() needs to set the state of the multibody system accordingly. You can do that fully consistently as in the openAIgymTriplePendulum.rst example, or you can also set some values to zero (like some initial velocities). Whatever is not initialized from the state will also not be randomized and may not be trained accordingly by the method.