Problem reading multiple cameras

[nathanieltagg]

---

Required Info
Camera Model	R457 (GMSL connectors)
Firmware Version	5.13.1.53
Operating System & Version	Ubuntu 20.04 (Jetpack 5.0.2)
Kernel Version (Linux Only)	5.10.104
Platform	NVIDIA Jetson
SDK Version	R/254 }
Language	C++
Segment	Robot

Issue Description

I'm having trouble reading out multiple cameras. Single camera readouts work fine, but if I start reading out a second camera in another thread I get errors such as:

terminate called after throwing an instance of 'rs2::invalid_value_error'
  what():
Failed to resolve the request:
        Format: BGR8, width: 1280, height: 720

Into:
        Formats:
         YUYV

Similar errors happen if I run examples/multicam/rs-multicam, so I don't think it's my code.
(The multicam error:

RealSense error calling rs2_pipeline_start_with_config(pipe:0xaaaaed972180, config:0xaaaaedda8ad0):

Failed to resolve the request:
        Format: Z16, width: 640, height: 480

Into:
        Formats:
         Z16
```)

There are 20 GB of free memory, so it's not an actual memory issue.

Any ideas on how to debug this?

[MartyG-RealSense]

Hi @nathanieltagg Firmware support for D457 was not officially added until firmware 5.14.0.0, the version after 5.13. The 5.14 firmware was designed to be used with librealsense 2.53.1.

If you are using librealsense 2.54.1 then firmware 5.15.0.2 should be used with it.

[nathanieltagg]

I have now updated all the camera firmware to 5.15.00.02. The same behavior is seen when running rs-multicam either from the current R/2542 branch or the current develop branch running rs-multicam.

Any other suggestions for things to try?

[MartyG-RealSense]

Are you able to add a second D457 camera in the RealSense Viewer by going to the Add Source option at the top of the Viewer's options side-panel? If the second D457 is accessible then it should be selectable from a list that appears after clicking Add Source.

[nathanieltagg]

Hi @MartyG-RealSense, sorry to take so long to get back to you. Yes, using the develop branch, I can indeed read two cameras simultaneously with the viewer code! But with the same release, the rs-multicam example fails with the error above.

[nathanieltagg]

Update: However, the realsense viewer doesn't always work. I just tried again, this time changing RGB resolution, and now I'm seeing it fail sometimes. However, I can't seem to find a reliable sequence that illustrates the problem: sometimes it works and sometimes it doesn't.

Failure dialog again claims an image conversion error, consistent with the error reported above.

[MartyG-RealSense]

I will consult with my Intel RealSense colleagues about your GMSL multicam issue. Thanks very much for your patience!

[MartyG-RealSense]

My colleagues advised me that the rs-multicam example should be able to work with D457 on a GMSL connection and should start both D457 cameras with a total of four streams (a pair of depth + RGB from each camera).

They also noted that it was unusual that both cameras were only sometimes able to be listed in the RealSense Viewer and thought that it might be related to the computer hardware and MIPI-GMSL adapter card that you are using.

The ideal match for the RealSense MIPI driver is an Nvidia Jetson AGX Xavier computing board and Intel's own brand of deserializer board.

https://github.com/IntelRealSense/realsense_mipi_platform_driver

IntelRealSense/realsense_mipi_platform_driver#166

What computer and deserializer are you using, please? If you are using a computing device other than AGX Xavier and a non-Intel deserializer board of your own choice then the MIPI driver must be converted (ported) to work with the particular computer and deserializer that you are using. Intel are unale to assist with the driver porting process, unfortunately.

The MIPI driver and deseriaizer component are not required when using D457 in USB mode instead of GMSL.

https://support.intelrealsense.com/hc/en-us/community/posts/14840675121043/comments/14843232242835

[MartyG-RealSense]

Hi @nathanieltagg Do you require further assistance with this case, please? Thanks!

[nathanieltagg]

The computer is a ConnectTech Anvil Orin AGX. Is it possible to affirmatively confirm that the issue is with the serializer or not? Looking at different RS realeases appears to affect results (although it is difficult to resolve problems)

[MartyG-RealSense]

As mentioned at #12184 (comment) Connect Tech will be the most appropriate support channel for using D457 with their D457-compatible Jetson hardware, which does not require the purchase of a separate deserializer component.

[MartyG-RealSense]

Hi @nathanieltagg Bearing in mind the information in the comment above, do you require further assistance with this case please? Thanks!

[dmipx]

Hi.
Recently we released multi MIPI camera support for LRS SDK on development branch.
You will need to apply latest udev-rules, from source scripts, it will enumerate mipi cameras and assign reasonable names for SDK.

[MartyG-RealSense]

Case closed due to no further comments received.