Replies: 19 comments
-
|
>>> eggonlea |
Beta Was this translation helpful? Give feedback.
-
|
>>> lissyx |
Beta Was this translation helpful? Give feedback.
-
|
>>> lissyx |
Beta Was this translation helpful? Give feedback.
-
|
>>> lissyx |
Beta Was this translation helpful? Give feedback.
-
|
>>> reuben |
Beta Was this translation helpful? Give feedback.
-
|
>>> eggonlea |
Beta Was this translation helpful? Give feedback.
-
|
>>> lissyx |
Beta Was this translation helpful? Give feedback.
-
|
>>> lissyx |
Beta Was this translation helpful? Give feedback.
-
|
>>> eggonlea |
Beta Was this translation helpful? Give feedback.
-
|
>>> lissyx |
Beta Was this translation helpful? Give feedback.
-
|
>>> lissyx |
Beta Was this translation helpful? Give feedback.
-
|
>>> eggonlea |
Beta Was this translation helpful? Give feedback.
-
|
>>> eggonlea |
Beta Was this translation helpful? Give feedback.
-
|
>>> lissyx |
Beta Was this translation helpful? Give feedback.
-
|
>>> lissyx |
Beta Was this translation helpful? Give feedback.
-
|
>>> reuben |
Beta Was this translation helpful? Give feedback.
-
|
>>> eggonlea |
Beta Was this translation helpful? Give feedback.
-
|
>>> reuben |
Beta Was this translation helpful? Give feedback.
-
|
>>> lissyx |
Beta Was this translation helpful? Give feedback.
-
>>> eggonlea
[May 19, 2019, 8:10pm]
Hello,
Background: DS 0.4.1 works fine. And now I'm switching to Master (aka
alpha8). I build the Mozilla Tensorflow r1.13 and native_client
successfully. And I can also train LBS with the new code.
The problem is, I cannot run the new deepspeech command utility to
decode wav file.
NOTE 1: I tried both of output_graph.pbmm & output_graph.pb. So I can
basically rule out convert_graphdef_memmapped_format. slash
NOTE 2: I tried both of local deepspeech (which I built) and the one
downloaded from TaskCluster (by util/taskcluster.py). So I can basically
rule out deepspeech native_client build issue.
So the only thing suspicious is the exported model. As the runtime
parameters changed a lot from 0.4.1 to alpha8, I might set something
wrong.
Training script
EPOCH=${1:-1}
TRAIN_FILES= slash
/srv/corpus/librivox/librivox-train-clean-100.csv, slash
/srv/corpus/librivox/librivox-train-clean-360.csv, slash
/srv/corpus/librivox/librivox-train-other-500.csv
DEV_FILES= slash
/srv/corpus/librivox/librivox-dev-clean.csv
TEST_FILES= slash
/srv/corpus/librivox/librivox-test-clean.csv
CACHE_PATH= slash
~/ds/cache/
time python -u ./DeepSpeech.py slash
--checkpoint_dir ~/ds/checkpoint slash
--summary_dir ~/ds/summary slash
--train_files ${TRAIN_FILES} slash
--dev_files ${DEV_FILES} slash
--test_files ${TEST_FILES} slash
--feature_cache ${CACHE_PATH} slash
--epochs ${EPOCH} slash
--n_hidden 2048 slash
--learning_rate 0.0001 slash
--dropout_rate 0.2 slash
--train_batch_size 24 slash
--dev_batch_size 48 slash
--test_batch-size 48 slash
--display_step 0 slash
--validation_step 1 slash
--log_level 1 slash
--summary_secs 60
time python -u ./DeepSpeech.py --checkpoint_dir ~/ds/checkpoint --n_hidden 2048 --nouse_seq_length --export_dir ~/ds/models
time python -u ./DeepSpeech.py --checkpoint_dir ~/ds/checkpoint --n_hidden 2048 --nouse_seq_length --export_tflite --export_dir
/ds/models/ds/models/output_graph.pbmmtime
/vobs/Mozilla/DeepSpeech/native_client/convert_graphdef_memmapped_format --in_graph=/ds/models/output_graph.pb --out_graph=With my local deepspeech
./native_client/deepspeech --model models/output_graph.pbmm --alphabet models/alphabet.txt --lm models/lm.binary --trie models/trie --audio LDC93S1.wav
TensorFlow: v1.13.1-10-g3e0cc5374d
DeepSpeech: v0.5.0-alpha.8-14-g033d0d6
2019-05-18 21:34:05.617736: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2019-05-18 21:34:05.769112: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:998] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-05-18 21:34:05.769891: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1433] Found device 0 with properties:
name: GeForce GTX 1080 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.6705
pciBusID: 0000:02:00.0
totalMemory: 10.89GiB freeMemory: 10.44GiB
2019-05-18 21:34:05.769910: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1512] Adding visible gpu devices: 0
2019-05-18 21:34:06.099880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:984] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-05-18 21:34:06.099923: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990] 0
2019-05-18 21:34:06.099927: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] 0: N
2019-05-18 21:34:06.100428: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10104 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:02:00.0, compute capability: 6.1)
2019-05-18 21:34:06.106328: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: 'UnwrapDatasetVariant' device_type: 'CPU'') for unknown op: UnwrapDatasetVariant
2019-05-18 21:34:06.106345: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: 'WrapDatasetVariant' device_type: 'GPU' host_memory_arg: 'input_handle' host_memory_arg: 'output_handle'') for unknown op: WrapDatasetVariant
2019-05-18 21:34:06.106351: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: 'WrapDatasetVariant' device_type: 'CPU'') for unknown op: WrapDatasetVariant
2019-05-18 21:34:06.106496: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: 'UnwrapDatasetVariant' device_type: 'GPU' host_memory_arg: 'input_handle' host_memory_arg: 'output_handle'') for unknown op: UnwrapDatasetVariant
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
With the downloaded deepspeech
./download/deepspeech --model models/output_graph.pbmm --alphabet models/alphabet.txt --lm models/lm.binary --trie models/trie --audio LDC93S1.wav
TensorFlow: v1.13.1-10-g3e0cc53
DeepSpeech: v0.5.0-alpha.8-2-gdf5bb31
2019-05-18 21:34:06.901687: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-05-18 21:34:06.909320: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: 'UnwrapDatasetVariant' device_type: 'CPU'') for unknown op: UnwrapDatasetVariant
2019-05-18 21:34:06.909346: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: 'WrapDatasetVariant' device_type: 'GPU' host_memory_arg: 'input_handle' host_memory_arg: 'output_handle'') for unknown op: WrapDatasetVariant
2019-05-18 21:34:06.909353: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: 'WrapDatasetVariant' device_type: 'CPU'') for unknown op: WrapDatasetVariant
2019-05-18 21:34:06.909413: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: 'UnwrapDatasetVariant' device_type: 'GPU' host_memory_arg: 'input_handle' host_memory_arg: 'output_handle'') for unknown op: UnwrapDatasetVariant
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
Error running session: Invalid argument: Tensor input_lengths:0, specified in either feed_devices or fetch_devices was not found in the Graph
[This is an archived TTS discussion thread from discourse.mozilla.org/t/fixed-error-with-master-alpha8-unknown-op-unwrapdatasetvariant-wrapdatasetvariant]
Beta Was this translation helpful? Give feedback.
All reactions