Use of RGB sensor on D455

[HasanTheSyrian]

Hello, I noticed there is an "RGB sensor" on the D455. What does it do exactly, and how does it affect the depth calculation? On the datasheet there is only a few sentences about it which didn't satisfy me.

Could I use it for object detection, where the object is only 1 color in an isolated environment (the whole scene contains the object with the color I'm looking for).

[MartyG-RealSense]

Hi @HasanTheSyrian The RGB sensor is essentially an ordinary video webcam. It is not involved in the depth calculation.

Yes, the RGB camera can be used for object detection. The RealSense SDK provides a C++ example program for doing so.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv/dnn

Another popular method of object detection with RealSense is to analyze the RGB image with a tool called YOLO. An example of doing so with D455 can be found at the link below.

https://github.com/Paulraj916/RealSense-Depth-Sensing-Object-Detection-with-YOLOv5

Typically, object detection applications are "trained" to recognize particular types of objects and isolate them from the background of the RGB image.

[HasanTheSyrian]

I'm using Yolo for my project, but I'm wondering if I can use the depth/RGB data as is. My use case is unclear, I know, I will post another discussion about it.

I'm wondering now if I can use the depth data (No ML) to detect an object because the object is the only thing that will be in the scene.