A working configuration on old jetson nano

[Blu-Eagle]

Finally, a working configuration.... the thing that bothers me is that any time I run docker-compose it has to recreate the yolo model.

anyway this is it:

docker-compose.yml

version: '2.4'
services:
  frigate:
    container_name: frigate
    image: madsciencetist/frigate-jetson:jetpack4.6.1   # this docker image
    privileged: true
    shm_size: "164mb"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/beppe/frigate/:/config
      - /home/beppe/frigate/storage:/media/frigate
      - /home/beppe/frigate/trt-models:/trt-models
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 500000000
    ports:
      - "5000:5000"
      - "18554:8554" # RTSP feeds
      - "18555:8555/tcp" # WebRTC over tcp
      - "18555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "xxx"
    runtime: nvidia
    environment:
      YOLO_MODELS: "yolov7-320" 

config.yml

mqtt:
  host: mqtt.server.com
cameras:
  cucina:
    ffmpeg:
      hwaccel_args: preset-jetson-h264
      inputs:
        - path: rtsp://login:[email protected]:88/videoSub
          roles:
            - detect

detectors:
  tensorrt:
    type: tensorrt
    model:
      path: /config/model_cache/tensorrt/yolov7-320.trt

model:
  input_tensor: nchw                              # yolo models are nchw
  width: 320                                      # must match model size
  height: 320                                     # must match model size

any suggestion or improvement is well accepted :)

[klutzzykitty]

Hello @Blu-Eagle
That's good that you've shared how you got it working on Jetson nano !

I guess it would add more meaning to this if you can share your Jetson nano configs as in Jetpack verion etc
and some details on how you've gotten the model conversion working. (basically a brief overall procedure that's required to get it up and going).

That would make this an excellent reference on frigate discussion board as currently people have to jump across
multiple discussions to understand how to set up things on Jetson.

[klutzzykitty]

@Blu-Eagle Awesome details. That actually helped.
A minor change would be that - /home/beppe/frigate/trt-models:/trt-models isn't required anymore isn't it ?

I've got it working on my Jetson as well - Nano, 4GB, jetpack 4.6.0
Also used jetpack 4.6.1,4.6.4.

Have you tried stopping your frigate container and starting it again a few times?
I've been seeing the container behavior change upon a few restarts.
Wanted to see if others were encountering it or if you can restart it a few times and see the debug window.
Here's a link to the observations: #6458 (comment)

PS: Wouldn't want to create a new issue for it's discussions as i'm the only one seeing it so far.
Thanks : )

[NickM-27]

Using tmpfs for /tmp/cache is highly recommended

[klutzzykitty]

Oops, typo. I've edited the comment.
Was referencing the line above - type: tmpfs --> mounting trt-models to container as that's not required anymore.

[NickM-27]

oh yes, that should not be used with the community jetson build / tesnorrt build

[wohali]

Updated version now that model has its own block:

mqtt:
  host: mqtt.server.com
  enabled: true

ffmpeg:
  hwaccel_args: preset-jetson-h264

cameras:
  cucina:
    ffmpeg:
      hwaccel_args: preset-jetson-h264
      inputs:
        - path: rtsp://login:[email protected]:88/videoSub
          roles:
            - detect

detectors:
  tensorrt:
    type: tensorrt

model:
  path: /config/model_cache/tensorrt/yolov7-320.trt
  input_tensor: nchw                              # yolo models are nchw
  width: 320                                      # must match model size
  height: 320                                     # must match model size

This should suppress the complaint about not being able to customize anything other than the detector type. I think it also sets the defaults for ffmpeg use everywhere, so they don't have to be supplied on a per-camera basis.

[Sharkflyee]

i followed above steps and it gets stuck at below log.. tried multiple times with restart as jetson nano 4gb becomes unresponsive

023-11-19 12:44:19.857848345 12:44:19.857 INF [webrtc] listen addr=0.0.0.0:8555/tcp
frigate | 2023-11-19 12:44:19.859514856 12:44:19.859 INF [srtp] listen addr=0.0.0.0:8443
frigate | 2023-11-19 12:44:26.850161433 [INFO] Starting go2rtc healthcheck service...
frigate |
frigate | Generating yolov7-320.trt. This may take a few minutes.
frigate |
frigate | /usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:172: DeprecationWarning: Use build_serialized_network instead.
frigate | engine = builder.build_engine(network, config)

[NickM-27]

Doesn't look like it's stuck it can just take some time

[beatmag]

Can anyone share the inference times and what models you are using for Jetson nano 4gb?
Interested to see what kind of performance is achievable.

[ayenz]

I followed steps above but got an error

2024-02-01 18:26:02.434574020 18:26:02.434 INF [api] listen addr=0.0.0.0:1984
2024-02-01 18:26:02.435080073 18:26:02.435 INF [rtsp] listen addr=0.0.0.0:8554
2024-02-01 18:26:02.435419091 18:26:02.435 INF [webrtc] listen addr=0.0.0.0:8555/tcp
2024-02-01 18:26:02.436111502 18:26:02.436 INF [srtp] listen addr=0.0.0.0:8443
Traceback (most recent call last):
File "/usr/local/src/tensorrt_demos/yolo/yolo_to_onnx.py", line 1065, in
main()
File "/usr/local/src/tensorrt_demos/yolo/yolo_to_onnx.py", line 1050, in main
yolo_model_def = builder.build_onnx_graph(
File "/usr/local/src/tensorrt_demos/yolo/yolo_to_onnx.py", line 584, in build_onnx_graph
initializer_layer, inputs_layer = weight_loader.load_conv_weights(
File "/usr/local/src/tensorrt_demos/yolo/yolo_to_onnx.py", line 458, in load_conv_weights
conv_init, conv_input = self._create_param_tensors(
File "/usr/local/src/tensorrt_demos/yolo/yolo_to_onnx.py", line 486, in _create_param_tensors
param_name, param_data, param_data_shape = self._load_one_param_type(
File "/usr/local/src/tensorrt_demos/yolo/yolo_to_onnx.py", line 514, in _load_one_param_type
param_data = np.ndarray(
TypeError: buffer is too small for requested array
s6-rc: warning: unable to start service trt-model-prepare: command exited 1
2024-02-01 18:26:11.256527417 [INFO] Starting go2rtc healthcheck service...

My memory still have a 2GB++, what could it be wrong?

[nguyentrangia]

Thanks @Blu-Eagle for the guide. I successfully installed in the first shot.
Is the inference speed ~100ms for 1 cam and ~120ms for 5 camera normal? And the VAINFO got error even though it is showing %GPU usage normal?
I have another Frigate on NUC7 i3, OpenVino (no Coral), 11 cameras with the same detection parameters as above. But the inference speed is only ~12ms!