RealSense MIPI platform driver for Leopard LI-JXAV-MIPI-ADPT-4CAM, LI-GMSL2-IPX-DESER on AGX Xavier

[sammychentw]

Hi,
I am trying to use Intel D457 camera on AGX Xavier.

My hardware environment are:

• Nvidia Jetson AGX Xavier EVB * 1
• Leopard LI-JXAV-MIPI-ADPT-4CAM board * 1
• Leopard LI-GMSL2-IPX-DESER board * 1
• Leopard FAW-1233-03 cable *1
• Intel RealSense D457 camera * 1
• GMSL FAKA cable * 1

1. git clone the Intel D457 MIPI driver for Jetson AGX Xavier:
   $ git clone https://github.com/IntelRealSense/realsense_mipi_platform_driver

2. setup_workspace, apply_patches, build_all_deb on PC.
   $ ./setup_workspace.sh 5.0.2
   $ ./apply_patches.sh apply 5.0.2
   $ ./build_all_deb.sh 5.0.2

3. Flash JetPack 5.0.2 to the Jetson AGX Xavier EVB

4. Install Intel D457 mipi driver by .deb on AGX Xavier.
   $ sudo dpkg -i ./linux-image-5.10.104-d457_5.10.104-d457-1_arm64.deb
   $ sudo dpkg -i ./linux-headers-5.10.104-d457_5.10.104-d457-1_arm64.deb
   $ sudo dpkg -i ./linux-libc-dev_5.10.104-d457-1_arm64.deb
   $ sudo dpkg -i ./linux-image-5.10.104-d457-dbg_5.10.104-d457-1_arm64.deb

5. Edit /boot/extlinux/extlinux.conf on AGX Xavier.
   LINUX /boot/Image-5.10.104-d457
   FDT /usr/lib/linux-image-5.10.104-d457/tegra194-p2888-0001-p2822-0000.dtb

6. Create /etc/modules-load.d/d4xx.conf on AGX Xavier.
   $ echo "d4xx" | sudo tee /etc/modules-load.d/d4xx.conf

7. Install Intel Real Sense Viewer tool
   $ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE || sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE
   sudo add-apt-repository "deb https://librealsense.intel.com/Debian/apt-repo $(lsb_release -cs) main" -u
   $ apt-get update
   $ sudo apt-get -y install librealsense2-utils
   $ sudo apt-get -y install librealsense2-dev
   $ sudo apt-get -y install librealsense2-dbg
   $ reboot

8. But we can't see /dev/video0, /dev/video1, /dev/video2, /dev/video3 after reboot on AGX Xavier, and realsense-viewer also can't see any video stream.

My question is :
May I use this GitHub Intel D457 mipi driver for Leopard LI-JXAV-MIPI-ADPT-4CAM + FAW-1233-03 + LI-GMSL2-IPX-DESER + Intel RealSense D457 camera + AGX Xavier?

[dmipx]

The MIPI-GMSL adapter boards have different hardware configuration for I2C bus. We didn't had these Leopad boards to try with, but i'm usre they are incompatible with our mipi-gmsl board. You will need to reconfigure device-tree to match I2C configuration.

[sammychentw]

May I to know which MIPI-GMSL adapter board could support the realsense_mipi_platform_driver?
Where could I buy this MIPI-GMSL adpter board? which model number?

[sammychentw]

The MIPI-GMSL adapter boards have different hardware configuration for I2C bus. We didn't had these Leopad boards to try with, but i'm usre they are incompatible with our mipi-gmsl board. You will need to reconfigure device-tree to match I2C configuration.

The I2C switch is TCA9546(0x70) on Leopard LI-JXAV-MIPI-ADPT-4CAM + LI-GMSL2-IPX-DESER,
so I had change I2C switch bus number from TCA9548(0x72) to TCA9546(0x70) in tegra194-camera-d4xx.dtsi.
It could only see /dev/video0, video1, but it can't see any video in realsense-viewer tool.

There are four i2c device in tegra194-camera-d4xx.dtsi:

1. i2c@0: max9296@48, max9295_prim@0x40, d4m0(d4m@10/cam-type="Depth"/1280x720)
2. i2c@1: d4m1(d4m@10/cam-type="RGB"/1920x1080)
3. i2c@2: d4m2(d4m@10/cam-type="Y8"/1280x720)
4. i2c@3: d4m3(d4m@10/cam-type="IMU"/640x480)

I check the kernel message, and found it only d4m0 initial sucesss, and d4m1, d4m2, d4m3 initial fail.

May I to know any device tree setting I need to reconfigure?
Does the schematic of LI-JXAV-MIPI-ADPT-4CAM + LI-GMSL2-IPX-DESER need to any design for D457 device tree (tegra194-camera-d4xx.dtsi)?

[dmipx]

We use i2c switch to "emulate" dedicated camera for each stream.
This was one of decisions at the time to get it work.
The driver expects to see (assuming i2c-mux address base on 30):
Depth at address 30-0010, RGB at 33-0010, Y8 at 34-0010 and IMU at 35-0010
That requires i2c-mux ports to be shorted together.
This should be resolved in next development cycle.

You can see detection on kernel log
Try with
dmesg | grep d4xx

[sammychentw]

• Does the mipi driver only expect the I2C-mux address static at Depth 30-0010, RGB 33-0010, Y8 34-0010, IMU 35-0010?
  or It could be Depth 30-0010, RGB 31-0010, Y8 32-0010, IMU 33-0010?

• That requires i2c-mux ports to be shorted together.
  Does your mean the schematic of TCA9546 I2C output channel 0~3 of Leopard board need to shorted together in hardware?

• May I confirm your intel mipi-gmsl board is below D4 GMSL de-serializer board?
  Intel RealSense Vision Processor D4 GMSL/FAKRA De-serializer Board
  https://store.intelrealsense.com/buy-intel-realsense-des457.html

• I attached my kernel log.
  kernel_log.txt

[dmipx]

1. The i2c addresses can be different, it will be detected as soon you configure dts.
2. Yes. You need to short together SDA of 4 ports and SCL of same 4 ports, configure dtsi to these specific ports.
3. Yes, you can use that board, it uses one max9296 with 2 cameras so you can get only 4 streams simultaneously. You will need to use "dev" branch for this adapter.
4. Thanks.

[sammychentw]

About question.3, If I use the Intel GMSL/FAKRA De-serializer Board, and Does it need also to rework for short together SDA of 4 ports and SCL of same 4 ports, too?

Intel RealSense Vision Processor D4 GMSL/FAKRA De-serializer Board

[dmipx]

No, it is ready for use.