Yolo-NAS ONNX invalid dimensions for input

[weitheng]

I am using Nvidia GPU, but want to run Yolo-Nas ONNX model. I have built and downloaded the model with pre-trained weights using the Google Collab link included in the guide. https://colab.research.google.com/github/blakeblackshear/frigate/blob/dev/notebooks/YOLO_NAS_Pretrained_Export.ipynb
I have changed my frigate configs to this:

detectors:
  onnx_0:
    type: onnx
  onnx_1:
    type: onnx
model:
  model_type: yolonas
  width: 320 # <--- should match whatever was set in notebook
  height: 320 # <--- should match whatever was set in notebook
  input_pixel_format: bgr
  path: /config/yolonas/yolo_nas_s.onnx
  labelmap_path: /labelmap/coco-80.txt

But getting error of invalid dimensions even though they are set like so input_image_shape=(320,320) in Google Collab :

2024-10-30 22:23:05.892731243  Process detector:onnx_1:
2024-10-30 22:23:05.892737177  Traceback (most recent call last):
2024-10-30 22:23:05.892739136    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-10-30 22:23:05.892740706      self.run()
2024-10-30 22:23:05.892742375    File "/opt/frigate/frigate/util/process.py", line 41, in run_wrapper
2024-10-30 22:23:05.892745341      return run(*args, **kwargs)
2024-10-30 22:23:05.892764665    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-10-30 22:23:05.892766251      self._target(*self._args, **self._kwargs)
2024-10-30 22:23:05.892768307    File "/opt/frigate/frigate/object_detection.py", line 144, in run_detector
2024-10-30 22:23:05.892769752      detections = object_detector.detect_raw(input_frame)
2024-10-30 22:23:05.892771205    File "/opt/frigate/frigate/object_detection.py", line 94, in detect_raw
2024-10-30 22:23:05.892793945      return self.detect_api.detect_raw(tensor_input=tensor_input)
2024-10-30 22:23:05.892795710    File "/opt/frigate/frigate/detectors/plugins/onnx.py", line 59, in detect_raw
2024-10-30 22:23:05.892797325      tensor_output = self.model.run(None, {model_input_name: tensor_input})
2024-10-30 22:23:05.892798997    File "/usr/local/lib/python3.9/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run
2024-10-30 22:23:05.892808785      return self._sess.run(output_names, input_feed, run_options)
2024-10-30 22:23:05.892810953  onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: input for the following indices
2024-10-30 22:23:05.892812374   index: 1 Got: 320 Expected: 3
2024-10-30 22:23:05.892813585   index: 3 Got: 3 Expected: 320
2024-10-30 22:23:05.892814876   Please fix either the inputs/outputs or the model.
2024-10-30 22:23:06.473410625  22:23:06.473 WRN [rtsp] error="streams: wrong login" stream=Roborock
2024-10-30 22:23:06.629322079  Process detector:onnx_0:
2024-10-30 22:23:06.629330541  Traceback (most recent call last):
2024-10-30 22:23:06.629332581    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-10-30 22:23:06.629334298      self.run()
2024-10-30 22:23:06.629336129    File "/opt/frigate/frigate/util/process.py", line 41, in run_wrapper
2024-10-30 22:23:06.629337812      return run(*args, **kwargs)
2024-10-30 22:23:06.629339608    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-10-30 22:23:06.629341364      self._target(*self._args, **self._kwargs)
2024-10-30 22:23:06.629343177    File "/opt/frigate/frigate/object_detection.py", line 144, in run_detector
2024-10-30 22:23:06.629344941      detections = object_detector.detect_raw(input_frame)
2024-10-30 22:23:06.629346557    File "/opt/frigate/frigate/object_detection.py", line 94, in detect_raw
2024-10-30 22:23:06.629348125      return self.detect_api.detect_raw(tensor_input=tensor_input)
2024-10-30 22:23:06.629349794    File "/opt/frigate/frigate/detectors/plugins/onnx.py", line 59, in detect_raw
2024-10-30 22:23:06.629351405      tensor_output = self.model.run(None, {model_input_name: tensor_input})
2024-10-30 22:23:06.629353241    File "/usr/local/lib/python3.9/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run
2024-10-30 22:23:06.629354788      return self._sess.run(output_names, input_feed, run_options)
2024-10-30 22:23:06.629356848  onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: input for the following indices
2024-10-30 22:23:06.629358226   index: 1 Got: 320 Expected: 3
2024-10-30 22:23:06.629359586   index: 3 Got: 3 Expected: 320
2024-10-30 22:23:06.629361066   Please fix either the inputs/outputs or the model.

I am not sure what went wrong. Appreciate any help! Thank you

[NickM-27]

you have to define the input_tensor

[weitheng]

Ahh, that's it. Thanks! But now I am getting a different error:

2024-10-30 22:45:56.176708927  Process detector:onnx_0:
2024-10-30 22:45:56.176715873  Traceback (most recent call last):
2024-10-30 22:45:56.176717730    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-10-30 22:45:56.176719211      self.run()
2024-10-30 22:45:56.176720855    File "/opt/frigate/frigate/util/process.py", line 41, in run_wrapper
2024-10-30 22:45:56.176722309      return run(*args, **kwargs)
2024-10-30 22:45:56.176723951    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-10-30 22:45:56.176725468      self._target(*self._args, **self._kwargs)
2024-10-30 22:45:56.176727089    File "/opt/frigate/frigate/object_detection.py", line 144, in run_detector
2024-10-30 22:45:56.176728612      detections = object_detector.detect_raw(input_frame)
2024-10-30 22:45:56.176730182    File "/opt/frigate/frigate/object_detection.py", line 94, in detect_raw
2024-10-30 22:45:56.176731708      return self.detect_api.detect_raw(tensor_input=tensor_input)
2024-10-30 22:45:56.176733333    File "/opt/frigate/frigate/detectors/plugins/onnx.py", line 83, in detect_raw
2024-10-30 22:45:56.176734655      raise Exception(
2024-10-30 22:45:56.176736327  Exception: ssd is currently not supported for rocm. See the docs for more info on supported models.

[NickM-27]

looks like the model type is not set correctly

[weitheng]

Hmm I don't understand. Isn't it

model:
  model_type: yolonas

?

[NickM-27]

what is the full config? seems like something is not right

[weitheng]

Sorry my bad, I deleted that line. It is working now! Thanks for the help :)