Hyperparameter tuning grid search in deepspeech

[JRMeyer]

>>> agarwalaashish20
[June 5, 2019, 7:04pm]

Hello Team! I am training a German Speech model using DeepSpeech and
struggling to get the best set of Hyperparameters. I referred to the
documented Hyperparameters in the DeepSpeech releases but the model is
not producing great results (WER 30). On increasing the number of epochs
to more than 10, the model is overfitting i.e. Training loss less than
5% but Validation loss more than 100%. I am wondering if there is a way
to tune the Hyperparameters using Grid Search or any other way in
DeepSpeech?

I am using approximately 300 hours of Dataset, with following
parameters:

[This is an archived TTS discussion thread from discourse.mozilla.org/t/hyperparameter-tuning-grid-search-in-deepspeech]

[JRMeyer]

>>> kdavis
[June 6, 2019, 9:11am]

Independent of hyperparameters, 300 hours alone is not sufficient to
train a model capable of understanding unrestricted speech. For English
we use more than 10 times that amount of speech and that's still not
enough.

That said if 300 hours is all you have, I'd try fine-tuning the 0.4.1
English model with your German data and with the standard character
substitutions ä= slash >ae, ü= slash >ue, ö= slash >oe, and ß= slash >ss to fix the fact that
the alphabets differ.

As for the hyperparameters, first try this fine-tuning with the ones you
are using.

[Archived Post]

[JRMeyer]

>>> lissyx
[June 6, 2019, 9:00am]

To
complement that answer, I can confirm that with a somewhat lower amount
of audio, around 235 hours of french, including Common Voice released
data, and fine-tuning on top of english with compatible alphabet, even
though WER / CER of test set are not really awesome, actual fiel usage
under proper condition gives decent enough result (speak slow,
articulate, etc).

[Archived Post]

[JRMeyer]

>>> lissyx
[June 6, 2019, 9:03am]

If you need
https://github.com/Common-Voice/commonvoice-fr/blob/master/DeepSpeech/CONTRIBUTING.md
and current WIP PR:
common-voice/commonvoice-fr#44

[Archived Post]