-
Notifications
You must be signed in to change notification settings - Fork 37
InferenceCommandLines
Vito Scaletta edited this page Feb 15, 2024
·
6 revisions
Sync mode:
python inference_mxnet_sync_mode.py --model_name mobilenetv2_1.0 --input ./data \
--input_name data --input_shape 3 3 224 224 \
--norm --mean 0.485 0.456 0.406 --std 0.229 0.224 0.225 \
--batch_size 3 --labels labels/image_net_labels.json \
--task classification --output_names outputAsync mode:
python inference_mxnet_async_mode.py --model_name mobilenetv2_1.0 --input ./data \
--input_name data --input_shape 3 3 224 224 \
--norm --mean 0.485 0.456 0.406 --std 0.229 0.224 0.225 \
--batch_size 3 --labels labels/image_net_labels.json \
--task classification --output_names outputInfer models stored in the savedmodel-format:
python inference_tensorflow.py --model ./public/densenet-121-tf/densenet-121.savedmodel/ \
--input ./data/ --batch_size 3 --task classification \
--output_names predictions --labels labels/image_net_synset.txt \
--input_shape 224 224 3 --mean 123.68 116.78 103.94 \
--input_scale 58.395 57.12 57.375python inference_tensorflow.py --model ./public/googlenet-v4-tf/saved_model/ \
--input ./data/ --batch_size 3 --task classification \
--output_names InceptionV4/Logits/Predictions \
--labels labels/image_net_synset_first_class_base.txt \
--input_shape 299 299 3 --mean 127.5 127.5 127.5 \
--input_scale 127.7 127.5 127.5python inference_tensorflow.py --model ./public/efficientnet-b0/efficientnet-b0/saved_model/ \
--input ./data/ --batch_size 4 --task classification \
--output_names logits --labels labels/image_net_synset.txt \
--input_shape 224 224 3 --mean 0.485 0.456 0.406 \
--input_scale 0.229 0.224 0.225Infer models stored in the pb-format:
python inference_tensorflow.py --model ./public/googlenet-v4-tf/inception_v4.frozen.pb \
--input ./data/ --batch_size 3 --task classification \
--input_name "input:0" --output_names InceptionV4/Logits/Predictions \
--labels labels/image_net_synset_first_class_base.txt \
--input_shape 299 299 3 --mean 127.5 127.5 127.5 \
--input_scale 127.7 127.5 127.5Infer models stored in the meta-format:
python inference_tensorflow.py --model ./public/efficientnet-b0/efficientnet-b0/model.ckpt.meta \
--input ./data/ --batch_size 4 --task classification \
--output_names logits --labels labels/image_net_synset.txt \
--input_shape 224 224 3 --input_name "sub:0" \
--mean 0.485 0.456 0.406 --input_scale 0.229 0.224 0.225python inference_tensorflowlite.py --model ./public/densenet-121-tf/densenet-121.tflite \
--input ./data/ --batch_size 3 --task classification \
--output_names output --labels labels/image_net_synset.txt \
--mean [123.675,116.28,103.53] --input_scale [58.395,57.12,57.375]Infer model converted form the TensorFlow to TensorFlow Lite format:
# Please, set correct paths to convert model from TF to TF lite format
python tflite_converter.py --model-path ./public/mobilenet-v1-1.0-224-tf/mobilenet_v1_1.0_224_frozen.pb \
--input-names input --output-names MobilenetV1/Predictions/Reshape_1 \
--source-framework tf
python inference_tensorflowlite.py --model ./public/mobilenet-v1-1.0-224-tf/mobilenet_v1_1.0_224.tflite \
--input ./data/ --batch_size 3 \
--task classification --output_names output \
--labels labels/image_net_synset_first_class_base.txt \
--mean [123.675,116.28,103.53] --input_scale [58.395,57.12,57.375]Infer models stored in the pth-format:
python inference_pytorch.py --model_name resnet50 --weights ./public/resnet-50-pytorch/resnet50-19c8e357.pth \
--input ./data/ --labels labels/image_net_labels.json --batch_size 4 \
--input_names data --input_shapes data[4,3,224,224] -ni 1 \
--task classification --mean [123.675,116.28,103.53] \
--input_scale [58.395,57.12,57.375]Infer models stored in the rknn-format:
python inference_rknn.py -bch rknn_benchmark -m ghostfacenet_arcface.rknn \
-i wider_face -d NPU \
--shape [1,112,112,3] --layout [NHWC] --dtype [U8]Infer model stored in the PyTorch-format:
python inference_tvm_pytorch.py --model_name resnet50 --weights ./public/resnet-50-pytorch/resnet50-19c8e357.pth \
--input ./data/ILSVRC2012_val_00000023.JPEG --labels labels/image_net_synset.txt \
--batch_size 1 --input_name data --input_shape 1 3 224 224 -ni 1 \
--task classification --norm --mean 0.485 0.456 0.406 --std 0.229 0.224 0.225 \
--opt_level 0 --layout NCHWInfer model stored in the Caffe-format:
python inference_tvm_caffe.py --task classification -is 1 3 224 224 -m ./public/googlenet-v1/googlenet-v1.prototxt \
-w ./public/googlenet-v1/googlenet-v1.caffemodel -i ./data/ --mean 0.408 0.459 0.482 \
-b 4 -l labels/image_net_synset.txt --layout NCHW --not_softmax --channel_swap 2 1 0Infer model stored in the ONNX-format:
# Please, set correct paths converting from TF to ONNX format
python -m tf2onnx.convert --saved-model densenet-121.savedmodel/ --output densenet-121-tf.onnx
python inference_tvm_onnx.py --model ./public/densenet-121-tf/densenet-121-tf.onnx --task classification \
--input_shape 1 224 224 3 --input ./data/ --batch_size 4 \
--labels labels/image_net_synset.txt --mean 123.68 116.78 103.94
--std 58.395 57.12 57.375 --layout NHWC --not_softmax --channel_swap 0 1 2 \
-ni 1 --output_names outputInfer models converted to the TVM-format:
# Please, set correct paths converting from ONNX to TVM format
python onnx_to_tvm_converter.py -mn densenet-121-tf -is 4 224 224 3 \
-m ./public/densenet-121-tf/densenet-121-tf.onnx \
-o ./public/densenet-121-tf/tvm_bs4
python inference_tvm.py -mn densenet-121-tf -m ./public/densenet-121-tf/tvm_bs4/densenet-121-tf.json \
-w ./public/densenet-121-tf/tvm_bs4/densenet-121-tf.params -i ./data/ \
-b 4 -l labels/image_net_synset.txt -is 4 224 224 3 --not_softmax \
-t classification --channel_swap 0 1 2 --layout NHWC --mean 123.68 116.78 103.94 \
--std 58.395 57.12 57.375 --input_name input_1Infer models tuned using Apache TVM:
# Please, set correct paths tuning the model
python tvm_meta_schedule.py -m ./public/densenet-121-tf/tvm_bs4/densenet-121-tf.json \
-p ./public/densenet-121-tf/tvm_bs4/densenet-121-tf.params \
-t "llvm -mcpu=core-avx2 --num-cores=6" -n 64 --max_trials_per_task 4 \
-o ./public/densenet-121-tf/tvm_bs4/densenet-121-tf.so
python inference_tvm.py -mn densenet-121-tf -m ./public/densenet-121-tf/tvm_bs4/densenet-121-tf.so \
-w ./public/densenet-121-tf/tvm_bs4/densenet-121-tf.params \
-i ./data/ -b 4 -l labels/image_net_synset.txt -is 4 224 224 3 --not_softmax \
-t classification --channel_swap 0 1 2 --layout NHWC --mean 123.68 116.78 103.94 \
--std 58.395 57.12 57.375 --input_name input_1