[Detector Support]: 'TensorRtDetector' object has no attribute 'outputs' | Setting up a custom yolov7 model.

[IxcheI]

Describe the problem you are having

Hi, I'm trying to get my own yolov7 model going but I keep getting the following error log no matter how I export and convert it. I'm pretty beginner level so please try to make it digestible for me.

Here is how I exported the model in yolov7:

python export.py --weights yolov7x_custom_best.pt --grid --include-nms
python ./tensorrt-python/export.py -o yolov7x_custom_best.onnx -e yolov7x_custom_best-nms.trt -p fp16
or
trtexec --onnx=yolov7x_custom_best.onnx --saveEngine=yolov7x_custom_best-nms.trt --fp16

I've been trying and snooping for other people that might have a similar issues but at this point my headache is so bad I just have to ask directly.

I'm completely lost at this point. I used the base yolov7x_training model and trained 30 epochs on a custom dataset from my footage. The detection with the yolov7 repo worked perfectly but it refuses to work in frigate and I have absolutely no idea what everyone does differently to not have these issues.
Is the problem that I used frames from the HD stream as training set? Is it because of the png format of my dataset? Is the base model I used already wrong and I need to train one from scratch? Maybe its just my training or export settings and I haven't found the magic combination yet? I don't even know where to start :/

Version

0.15.0-3c591ad

Frigate config file

mqtt:
  enabled: true
  host: mqtt5
  port: 1883

go2rtc:
  streams:
    back:
      - rtsp://stream

review:
  alerts:
    labels:
      #- person
      - cat
      - fox
      - bird
      - dog
  detections:
    labels:
      - person

cameras:
  back:
    enabled: true
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
        - path: 
            rtsp://stream
          roles:
            - detect
        - path: 
            rtsp:stream
          roles:
            - record
    detect:
      width: 1280
      height: 720
      # width: 640
      # height: 480
      enabled: true
      # fps: 10
      max_disappeared: 100
      stationary:
        # threshold: 100
        max_frames:
          default: 3000
      annotation_offset: -600
    objects:
      track:
        - person
        - dog
        - fox
        - cat
        - bird
      filters:
        cat:
          min_score: 0.2
          threshold: 0.4
        bird:
          min_score: 0.2
          threshold: 0.4
        dog:
          min_score: 0.2
          threshold: 0.4
    snapshots:
      enabled: true
    record:
      enabled: true
    motion:
      mask: 0.695,0.046,0.97,0.045,0.969,0.095,0.695,0.095
      threshold: 38
      contour_area: 10
      improve_contrast: true

    zones: {}

record:
  retain:
    days: 3
    mode: motion
  alerts:
    retain:
      days: 20
  detections:
    retain:
      days: 7
  sync_recordings: false

snapshots:
  enabled: true
  clean_copy: false
  timestamp: true
  bounding_box: true
  crop: false
  retain:
    default: 20
  quality: 90


detectors:
  tensorrt:
    type: tensorrt
    device: 0

model:
  path: /trt-models/yolov7x_custom_best-nms.trt
  input_tensor: nchw
  input_pixel_format: rgb
  width: 640
  height: 640


version: 0.15-0

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:3c591ad-tensorrt
    #image: ghcr.io/blakeblackshear/frigate:stable-tensorrt
    shm_size: "256mb"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              #device_ids: ['0'] # this is only needed when using multiple GPUs
              count: all # number of GPUs
              capabilities: [gpu]
    volumes:
      - \\wsl$$\Ubuntu\etc\localtime:/etc/localtime:ro
      - C:\Users\USER\frigate\config:/config
      - C:\Users\USER\frigate\clips:/media/frigate
      - C:\Users\USER\frigate\trt-models:/trt-models:ro
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "8971:8971"
      - "5000:5000" # Internal unauthenticated access. Expose carefully.
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      - YOLO_MODELS=yolov7-320,yolov7x-640

  frigate-notify:
    image: ghcr.io/0x2142/frigate-notify:latest
    environment:
      TZ: Etc/UTC
    volumes:
      - C:\Users\USER\frigate\frigate_notify\config.yml:/app/config.yml
    restart: unless-stopped

  ntfy:
    image: binwiederhier/ntfy
    restart: unless-stopped
    environment:
      NTFY_BASE_URL: http://192.168.178.66
      NTFY_CACHE_FILE: /var/lib/ntfy/cache.db
      NTFY_AUTH_FILE: /var/lib/ntfy/auth.db
      NTFY_AUTH_DEFAULT_ACCESS: read-write
      NTFY_BEHIND_PROXY: true
      NTFY_ATTACHMENT_CACHE_DIR: /var/lib/ntfy/attachments
      NTFY_ENABLE_LOGIN: false
    volumes:
      - C:\Users\USER\frigate\ntfy:/var/lib/ntfy
    ports:
      - 80:80
    command: serve

  mosquitto:
    image: eclipse-mosquitto
    container_name: mqtt5
    restart: unless-stopped
    environment:
      PUID: 1883
      PGID: 1883
    ports:
      - "1883:1883"
      - "9001:9001"
    volumes:
      - C:\Users\USER\frigate\mosquitto\config:/mosquitto/config:rw
      - C:\Users\USER\frigate\mosquitto\data:/mosquitto/data:rw
      - C:\Users\USER\frigate\mosquitto\log:/mosquitto/log:rw

# docker-compose up -d

Relevant Frigate log output

2024-10-27 21:30:23 s6-rc: info: service s6rc-fdholder: starting
2024-10-27 21:30:23 s6-rc: info: service s6rc-oneshot-runner: starting
2024-10-27 21:30:23 s6-rc: info: service s6rc-oneshot-runner successfully started
2024-10-27 21:30:23 s6-rc: info: service fix-attrs: starting
2024-10-27 21:30:23 s6-rc: info: service s6rc-fdholder successfully started
2024-10-27 21:30:23 s6-rc: info: service fix-attrs successfully started
2024-10-27 21:30:23 s6-rc: info: service legacy-cont-init: starting
2024-10-27 21:30:23 s6-rc: info: service legacy-cont-init successfully started
2024-10-27 21:30:23 s6-rc: info: service trt-model-prepare: starting
2024-10-27 21:30:23 s6-rc: info: service log-prepare: starting
2024-10-27 21:30:23 s6-rc: info: service log-prepare successfully started
2024-10-27 21:30:23 s6-rc: info: service nginx-log: starting
2024-10-27 21:30:23 s6-rc: info: service go2rtc-log: starting
2024-10-27 21:30:23 s6-rc: info: service frigate-log: starting
2024-10-27 21:30:23 s6-rc: info: service certsync-log: starting
2024-10-27 21:30:23 s6-rc: info: service nginx-log successfully started
2024-10-27 21:30:23 s6-rc: info: service go2rtc-log successfully started
2024-10-27 21:30:23 s6-rc: info: service go2rtc: starting
2024-10-27 21:30:23 s6-rc: info: service frigate-log successfully started
2024-10-27 21:30:23 s6-rc: info: service certsync-log successfully started
2024-10-27 21:30:23 s6-rc: info: service go2rtc successfully started
2024-10-27 21:30:23 s6-rc: info: service go2rtc-healthcheck: starting
2024-10-27 21:30:23 s6-rc: info: service go2rtc-healthcheck successfully started
2024-10-27 21:30:23 2024-10-27 21:30:23.225648811  [INFO] Preparing new go2rtc config...
2024-10-27 21:30:23 No models to convert.
2024-10-27 21:30:23 s6-rc: info: service trt-model-prepare successfully started
2024-10-27 21:30:23 s6-rc: info: service frigate: starting
2024-10-27 21:30:23 s6-rc: info: service frigate successfully started
2024-10-27 21:30:23 s6-rc: info: service nginx: starting
2024-10-27 21:30:23 2024-10-27 21:30:23.230930923  [INFO] Preparing Frigate...
2024-10-27 21:30:23 2024-10-27 21:30:23.231588173  [INFO] Starting NGINX...
2024-10-27 21:30:23 2024-10-27 21:30:23.237086028  [INFO] Starting Frigate...
2024-10-27 21:30:23 s6-rc: info: service nginx successfully started
2024-10-27 21:30:23 s6-rc: info: service certsync: starting
2024-10-27 21:30:23 s6-rc: info: service certsync successfully started
2024-10-27 21:30:23 s6-rc: info: service legacy-services: starting
2024-10-27 21:30:23 2024-10-27 21:30:23.316300138  [INFO] Starting certsync...
2024-10-27 21:30:23 s6-rc: info: service legacy-services successfully started
2024-10-27 21:30:23 2024-10-27 21:30:23.368986695  [INFO] Starting go2rtc...
2024-10-27 21:30:23 2024-10-27 21:30:23.399130515  127.0.0.1 - - [27/Oct/2024:21:30:23 +0100] "" 400 0 "-" "-" "-"
2024-10-27 21:30:23 2024-10-27 21:30:23.425970987  21:30:23.425 INF go2rtc platform=linux/amd64 revision=b2399f3 version=1.9.2
2024-10-27 21:30:23 2024-10-27 21:30:23.425973500  21:30:23.425 INF config path=/dev/shm/go2rtc.yaml
2024-10-27 21:30:23 2024-10-27 21:30:23.426132478  21:30:23.426 INF [rtsp] listen addr=:8554
2024-10-27 21:30:23 2024-10-27 21:30:23.426195663  21:30:23.426 INF [webrtc] listen addr=:8555/tcp
2024-10-27 21:30:23 2024-10-27 21:30:23.426441646  21:30:23.426 INF [api] listen addr=:1984
2024-10-27 21:30:24 2024-10-27 21:30:24.293299310  [2024-10-27 21:30:24] frigate.util.config            INFO    : Checking if frigate config needs migration...
2024-10-27 21:30:24 2024-10-27 21:30:24.304613892  [2024-10-27 21:30:24] frigate.util.config            INFO    : frigate config does not need migration...
2024-10-27 21:30:24 2024-10-27 21:30:24.825840295  [2024-10-27 21:30:24] frigate.util.services          INFO    : Automatically detected nvidia hwaccel for video decoding
2024-10-27 21:30:25 2024-10-27 21:30:25.978645253  2024/10/27 21:30:25 [error] 170#170: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /ws HTTP/1.1", subrequest: "/auth", upstream: "http://127.0.0.1:5001/auth", host: "localhost:5000"
2024-10-27 21:30:25 2024-10-27 21:30:25.978649560  2024/10/27 21:30:25 [error] 170#170: *2 auth request unexpected status: 502 while sending to client, client: 172.18.0.1, server: , request: "GET /ws HTTP/1.1", host: "localhost:5000"
2024-10-27 21:30:25 2024-10-27 21:30:25.978678884  172.18.0.1 - - [27/Oct/2024:21:30:25 +0100] "GET /ws HTTP/1.1" 500 579 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36" "-"
2024-10-27 21:30:26 2024-10-27 21:30:26.009971627  [2024-10-27 21:30:26] frigate.app                    INFO    : Starting Frigate (0.15.0-3c591ad)
2024-10-27 21:30:26 2024-10-27 21:30:26.026785640  [2024-10-27 21:30:26] peewee_migrate.logs            INFO    : Starting migrations
2024-10-27 21:30:26 2024-10-27 21:30:26.027318324  [2024-10-27 21:30:26] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-10-27 21:30:26 2024-10-27 21:30:26.043765080  [2024-10-27 21:30:26] frigate.app                    INFO    : Recording process started: 389
2024-10-27 21:30:26 2024-10-27 21:30:26.043789792  [2024-10-27 21:30:26] frigate.app                    INFO    : Review process started: 391
2024-10-27 21:30:26 2024-10-27 21:30:26.044858054  [2024-10-27 21:30:26] frigate.app                    INFO    : go2rtc process pid: 102
2024-10-27 21:30:26 2024-10-27 21:30:26.060722748  [2024-10-27 21:30:26] frigate.app                    INFO    : Output process started: 423
2024-10-27 21:30:26 2024-10-27 21:30:26.074644294  [2024-10-27 21:30:26] detector.tensorrt              INFO    : Starting detection process: 406
2024-10-27 21:30:26 2024-10-27 21:30:26.160061806  [2024-10-27 21:30:26] frigate.app                    INFO    : Camera processor started for back: 440
2024-10-27 21:30:26 2024-10-27 21:30:26.160103806  [2024-10-27 21:30:26] frigate.app                    INFO    : Capture process started for back: 441
2024-10-27 21:30:26 2024-10-27 21:30:26.262924516  [2024-10-27 21:30:26] frigate.api.fastapi_app        INFO    : Starting FastAPI app
2024-10-27 21:30:26 2024-10-27 21:30:26.515919945  [2024-10-27 21:30:26] frigate.detectors.plugins.tensorrt INFO    : Loaded engine size: 140 MiB
2024-10-27 21:30:26 2024-10-27 21:30:26.561132924  [2024-10-27 21:30:26] frigate.detectors.plugins.tensorrt ERROR   : 1: [stdArchiveReader.cpp::StdArchiveReader::32] Error Code 1: Serialization (Serialization assertion magicTagRead == kMAGIC_TAG failed.Magic tag does not match)
2024-10-27 21:30:26 2024-10-27 21:30:26.565414467  [2024-10-27 21:30:26] frigate.detectors.plugins.tensorrt ERROR   : 4: [runtime.cpp::deserializeCudaEngine::66] Error Code 4: Internal Error (Engine deserialization failed.)
2024-10-27 21:30:26 2024-10-27 21:30:26.566557083  Process detector:tensorrt:
2024-10-27 21:30:26 2024-10-27 21:30:26.566558334  Traceback (most recent call last):
2024-10-27 21:30:26 2024-10-27 21:30:26.566558980    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-10-27 21:30:26 2024-10-27 21:30:26.566559390      self.run()
2024-10-27 21:30:26 2024-10-27 21:30:26.566559810    File "/opt/frigate/frigate/util/process.py", line 41, in run_wrapper
2024-10-27 21:30:26 2024-10-27 21:30:26.566561748      return run(*args, **kwargs)
2024-10-27 21:30:26 2024-10-27 21:30:26.566562538    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-10-27 21:30:26 2024-10-27 21:30:26.566562999      self._target(*self._args, **self._kwargs)
2024-10-27 21:30:26 2024-10-27 21:30:26.566570782    File "/opt/frigate/frigate/object_detection.py", line 109, in run_detector
2024-10-27 21:30:26 2024-10-27 21:30:26.566571243      object_detector = LocalObjectDetector(detector_config=detector_config)
2024-10-27 21:30:26 2024-10-27 21:30:26.566571643    File "/opt/frigate/frigate/object_detection.py", line 61, in __init__
2024-10-27 21:30:26 2024-10-27 21:30:26.566572012      self.detect_api = create_detector(detector_config)
2024-10-27 21:30:26 2024-10-27 21:30:26.566578451    File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
2024-10-27 21:30:26 2024-10-27 21:30:26.566578841      return api(detector_config)
2024-10-27 21:30:26 2024-10-27 21:30:26.566581855    File "/opt/frigate/frigate/detectors/plugins/tensorrt.py", line 244, in __init__
2024-10-27 21:30:26 2024-10-27 21:30:26.566582276      self.input_shape = self._get_input_shape()
2024-10-27 21:30:26 2024-10-27 21:30:26.566582727    File "/opt/frigate/frigate/detectors/plugins/tensorrt.py", line 118, in _get_input_shape
2024-10-27 21:30:26 2024-10-27 21:30:26.566583035      binding = self.engine[0]
2024-10-27 21:30:26 2024-10-27 21:30:26.566583393  TypeError: 'NoneType' object is not subscriptable
2024-10-27 21:30:26 2024-10-27 21:30:26.566583834  Exception ignored in: <function TensorRtDetector.__del__ at 0x7f84f80b04c0>
2024-10-27 21:30:26 2024-10-27 21:30:26.566584183  Traceback (most recent call last):
2024-10-27 21:30:26 2024-10-27 21:30:26.566584552    File "/opt/frigate/frigate/detectors/plugins/tensorrt.py", line 262, in __del__
2024-10-27 21:30:26 2024-10-27 21:30:26.566588325      if self.outputs is not None:
2024-10-27 21:30:26 2024-10-27 21:30:26.566588746  AttributeError: 'TensorRtDetector' object has no attribute 'outputs'

Install method

Windows WSL2

Object Detector

TensorRT

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

Yes I'm running an experimental version and not the last stable build. However I don't think that's the cause of this problem right?
Right now I can't go back to stable due to config validation errors.

[NickM-27]

I'm not super familiar with trt mirrors specifically so I personally won't be able to help too much. You'd likely have better luck exporting as onnx format

[IxcheI]

Then I don't really understand why the output values are so different in frigate vs the onnxruntime inference I'm doing aside from frigate.
There is nothing in the model converting to int. Here is how I'm running the model outside of frigate and the output is exactly as expected:

import onnxruntime as ort
import cv2
import numpy as np

cuda = True
w = "yolov7x_custom_best.onnx"

providers = ['CUDAExecutionProvider', 'CPUExecutionProvider'] if cuda else ['CPUExecutionProvider']
session = ort.InferenceSession(w, providers=providers)

# Image processing (copied from yolov7 repo)
img = cv2.imread('inference/images/img.png')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

image = img.copy()
image, ratio, dwdh = letterbox(image, auto=False) # letterbox is from yolov7 (defined above in my jupiter notebook)
image = image.transpose((2, 0, 1))
image = np.expand_dims(image, 0)
image = np.ascontiguousarray(image)

im = image.astype(np.float32)
im /= 255

outname = [i.name for i in session.get_outputs()]
inname = [i.name for i in session.get_inputs()]

inp = {inname[0]:im}

# ONNX inference
outputs = session.run(outname, inp)[0]
print(outputs)

And here the output:

[
[  0.         268.96875    316.09375    333.03125    395.40625   15.           0.9305551]
[  0.         101.84375    295.375      186.90625    470.625   15.           0.84457195]
]

Model output in Netron

[NickM-27]

Can you send that model then so I can try it?

[NickM-27]

looks like the /= 255 is required

[IxcheI]

Ayyyy you're right, it works now! I will run it some time to test but so far the debug + test footage works well.

Solution:
Changed line to tensor_output = self.model.run(None, {model_input_name: tensor_input.astype(np.float32)/255})
And line to (_, x_min, y_min, x_max, y_max, class_id, confidence) = prediction