Radam instead of adam

[JRMeyer]

>>> caucheteux
[August 21, 2019, 9:15am]

Hello, slash
just a topic to discuss the use of Rectifier Adam (RAdam) instead of
Adam in DeepSpeech.

If I trust this
link
using Radam might improve results.

Did this method will be implemented or is it not in the current
reflexion ?

[This is an archived TTS discussion thread from discourse.mozilla.org/t/radam-instead-of-adam]

[JRMeyer]

>>> lissyx
[August 22, 2019, 8:07am]



> Did this method will be implemented or is it not in the current
> reflexion ?

As far as I can tell, it's not on our radar.

[Archived Post]

[JRMeyer]

>>> kdavis
[August 22, 2019, 9:14am]

Is an RAdam implementation in TensorFlow?

[Archived Post]

[JRMeyer]

>>> kdavis
[August 22, 2019, 9:50am]

Looks like it's not there yet, issue
422

[Archived Post]

[JRMeyer]

>>> carlfm01
[August 22, 2019, 10:36pm]

This looks testable:
https://github.com/CyberZHG/keras-radam#tensorflow-without-keras

[Archived Post]

[JRMeyer]

>>> caucheteux
[August 23, 2019, 9:31am]

It might be interesting to test it, don't know if there will be
improvement

[Archived Post]