Training model for fluently spoken language

[JRMeyer]

>>> Jendker
[August 9, 2019, 3:30pm]

Hello everyone,

I am training the model on 'read aloud' data and the test results for
similar data are quite good (around 0.2 WER). The problem is, that this
model should be used for spoken, more carelessly spoken language and
then the quality of transcription is significantly worse.

Would you have any ideas on how to modify the 'read aloud' dataset to
train the model for spoken language? Would something like making the
train audio files faster or adding some noise to them make any sense?

[This is an archived TTS discussion thread from discourse.mozilla.org/t/training-model-for-fluently-spoken-language]

[JRMeyer]

>>> lissyx
[August 9, 2019, 10:02am]



> 'read aloud'

What is this dataset ? In general, you should share much more
informations, we have no idea how much data you have, how your training
and your inference are performed, there are multiple factors that could
explain poor results.

[Archived Post]

[JRMeyer]

>>> Jendker
[August 9, 2019, 12:29pm]

Thank you for your answer!

Currently I am trying to train a model for German language, spoken over
phone.

Datasets: slash
Currently I am using publicly available datasets:

(http://www.repository.voxforge1.org/downloads/de/Trunk/Audio/Main/16kHz_16bit/)
(http://ltdata1.informatik.uni-hamburg.de/kaldi_tuda_de/german-speechdata-package-v3.tar.gz)
(https://www2.informatik.uni-hamburg.de/nats/pub/SWC/SWC_German.tar)
(http://goofy.zamia.org/zamia-speech/corpora/forschergeist/)

Altogether it is about 500 hours of the training data, but unfortunately
from all here only the very small dataset 'Forscher' is spoken in more
natural manner, similarly to actual conversations, and only this data
correspond closely to the actual use case.

Preprocessing before training: slash
Because the phone conversation data for which I am going to do the
inference have 8kHz sampling rate I first downsample all the training
data to 8kHz with a-law encoding (to make it more similar to target
data), and than again to 16 kHz for training. Upsampling to 16 kHz is
done in the same way as in client.py:

sox <input file> --bits 16 --channels 1 --rate 16000 --encoding signed-integer --endian little --compression 0.0 --no-dither <output_file>

The training parameters: slash

Inference: slash
Inference is done with client.py file. I have adjusted the
BEAM_WIDTH = 1024 parameter, I am not sure if I need to change
N_FEATURES and N_CONTEXT though. slash
As mentioned earlier, the data on which I am doing inference are the
phone data, so they have 8 kHz sampling rate. I allow the upsampling of
this 8 kHz data to 16 kHz to be done by client.py, because the
training data conversion is done in the same way.

The inference results are not that good as the results of testing, and
that's the current problem.

[Archived Post]

[JRMeyer]

>>> carlfm01
[August 10, 2019, 2:29am]

Hello [ slash Jendker](
500h is not enough to start the training from scratch, maybe try
transfer learning? I'm getting the best result with transfer learning
using 500h of Spanish.

You can start reading here: slash


V0.5.1 Model, right branch, method and
discussion](https://discourse.mozilla.org/t/non-native-english-with-transfer-learning-from-v0-5-1-model-right-branch-method-and-discussion/42101/2)
drop-close='true'
.box}

> It would be great that you join forces:
> https://discourse.mozilla.org/c/voice/fr
> https://github.com/Common-Voice/commonvoice-fr/issues As much as I
> can tell, they are deprecated. No, what is the AMI dataset ? Common
> Voice, as you can see on the french training data shared earlier, is
> not yet enough.

[Archived Post]

[JRMeyer]

>>> Jendker
[August 10, 2019, 10:12pm]

Thanks for the link, it looks very promising! I'll try that out soon.

[Archived Post]