Poor results from wavernn generation

[morphr5466]

hi
I get eval.wav by training tacotron2,it sounds good,

and I training wavernn 215k with gta,generate a new wav use batched,It doesn't sound as good as your audio.
It's a little noisy and husky,What do I need to do to make it sound as good as you show it?

[morphr5466]

audio.zip

[begeekmyfriend]

At least 500K steps. What is your tacotron2 training epochs?

[morphr5466]

130k,My dataset contains 50,000 audio, about 25 hours, and training is slow.

[begeekmyfriend]

Maybe you need some better GPU like 1080Ti or 2080Ti

[morphr5466]

audio.zip
The 800k result doesn't sound great either.

[begeekmyfriend]

You might try 'O1' level on this line

[begeekmyfriend]

By the way, would you please try the latest commit 55ff380 that it might achieve better alignment as this issue mentioned #22

[morphr5466]

hello,sir I'm having problem training the tacotron model until I don't use the “--load-mel-from-disk”
File "train.py", line 405, in
main()
File "train.py", line 291, in main
for i, batch in enumerate(train_loader):
File "/home/ubuntu/workspace/fujingchuan/RTVCm/venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 819, in next
return self._process_data(data)
File "/home/ubuntu/workspace/fujingchuan/RTVCm/venv/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data
data.reraise()
File "/home/ubuntu/workspace/fujingchuan/RTVCm/venv/lib/python3.6/site-packages/torch/_utils.py", line 385, in reraise
raise self.exc_type(msg)
AssertionError: Caught AssertionError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/ubuntu/workspace/fujingchuan/RTVCm/venv/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/ubuntu/workspace/fujingchuan/RTVCm/venv/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/ubuntu/workspace/fujingchuan/RTVCm/venv/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/kv-data/alg/fujingchuan/taco2+wavernn/kdxf/kedaxunfei_taco2_fenci/t2-BIAOBEI-6.8新/tacotron2/data_function.py", line 83, in geitem
group = [self.get_mel_text_pair(i, self.metadatas[i][self.offsets[i]]) for i in range(self.speaker_num)]
File "/kv-data/alg/fujingchuan/taco2+wavernn/kdxf/kedaxunfei_taco2_fenci/t2-BIAOBEI-6.8新/tacotron2/data_function.py", line 83, in
group = [self.get_mel_text_pair(i, self.metadatas[i][self.offsets[i]]) for i in range(self.speaker_num)]
File "/kv-data/alg/fujingchuan/taco2+wavernn/kdxf/kedaxunfei_taco2_fenci/t2-BIAOBEI-6.8新/tacotron2/data_function.py", line 63, in get_el_text_pair
mel = self.get_mel(mel_path)
File "/kv-data/alg/fujingchuan/taco2+wavernn/kdxf/kedaxunfei_taco2_fenci/t2-BIAOBEI-6.8新/tacotron2/data_function.py", line 75, in get_el
melspec.size(0), self.stft.n_mel_channels))
AssertionError: Mel dimension mismatch: given 64000, expected 80

[morphr5466]

and when I trained wavernn I ran into the following problem, at first I thought it was a data problem, but after I made shuffle=False the error went away.It's a strange phenomenon.
| Epoch: 3/913 (296/868) | Loss: 2.4442 | 3.0 steps/s | Step: 9k | /pytorch/aten/src/THCUNN/SpatialClassNLLCriterion.cu:104: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T *, T *, T *, long *, T *, int, int, int, int, int, long) [with T = float, AccumT = float]: block: [27,0,0], thread: [796,0,0] Assertion t >= 0 && t < n_classes failed.
Traceback (most recent call last):
File "train_wavernn.py", line 152, in
voc_train_loop(voc_model, loss_func, optimizer, train_set, test_set, init_lr, final_lr, total_steps)
File "train_wavernn.py", line 66, in voc_train_loop
scaled_loss.backward()
File "/home/ubuntu/workspace/fujingchuan/RTVCm/venv/lib/python3.6/site-packages/torch/tensor.py", line 150, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/ubuntu/workspace/fujingchuan/RTVCm/venv/lib/python3.6/site-packages/torch/autograd/init.py", line 99, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

[leijue222]

@morphr5466 Hi, have you taken any measures to improve the bad results of wavernn?