ClientTimeout when streaming OS1-128 LiDAR and camera images

[chadrs2]

Describe the bug
I am trying to stream LiDAR scans as well as images from an OAK-D camera at the same time. Both sensors are connected over a PoE Ethernet switch and my laptop is connected to the switch. If I try to stream either sensor alone, then no errors show up. When I try to stream both sensors, then the ouster sensor fails giving me the following error:

Traceback (most recent call last):
  File "/home/chadrs2/HetMAS/camera_lidar_calibration_files/lidar_camera_calibration/collect_lidar_calib_data.py", line 60, in <module>
    ouster_thread("10.42.0.89", data_dir)
  File "/home/chadrs2/HetMAS/camera_lidar_calibration_files/lidar_camera_calibration/collect_lidar_calib_data.py", line 34, in ouster_thread
    for i, scan in enumerate(lidar_scans):
  File "/home/chadrs2/miniconda3/envs/camli/lib/python3.9/site-packages/ouster/sdk/client/scan_source_adapter.py", line 77, in _stream_iter
    for ls in source:
  File "/home/chadrs2/miniconda3/envs/camli/lib/python3.9/site-packages/ouster/sdk/client/multi.py", line 45, in collate_scans
    for idx, m in source:
  File "/home/chadrs2/miniconda3/envs/camli/lib/python3.9/site-packages/ouster/sdk/sensor/sensor_scan_source.py", line 173, in _scans_iter
    raise ClientTimeout(f"No valid scans received within {self._timeout/1e9}s from sensor "
ouster.sdk.client.core.ClientTimeout: No valid scans received within 1.0s from sensor 10.42.0.89 using udp destination 10.42.0.1 on port 7502.

The camera continues to stream in data even after the LiDAR has failed and stopped.

I found that the LiDAR didn't crash when I decreased the camera frame rate down to 1 FPS. This shows me that for some reason there's a data overload happening. I'm using a Gigabit ethernet switch and the lights indicate that all sensors and computer are running at gigabit speed.

This is my simple LiDAR code:

import cv2
import numpy as np
from datetime import datetime, timezone

from ouster.sdk import open_source
from ouster.sdk.client import ChanField, destagger, XYZLut

# Function to convert LiDAR signal to an image
def convert_pc2img(scan, meta_data, do_destagger=True):
    if do_destagger:
        img_mono = destagger(meta_data, scan.field(ChanField.SIGNAL)).astype(np.float32)
    else:
        img_mono = scan.field(ChanField.SIGNAL).astype(np.float32)
    img_mono_normalized = np.uint8(((img_mono - np.min(img_mono)) / (np.max(img_mono) - np.min(img_mono))) * 255)
    img = np.repeat(img_mono_normalized[..., np.newaxis], 3, axis=-1)
    return img

def ouster_thread(source, data_dir):
    lidar_scans = open_source(source, sensor_idx=0, cycle=False)
    scan_cnt = 0
    for i, scan in enumerate(lidar_scans):
        if i == 0:
            continue
        # Convert scan to image and XYZ point cloud
        img_pc = convert_pc2img(scan, lidar_scans.metadata)

        timestamp_ns = scan.timestamp[-1]  # Timestamp in nanoseconds (grab the last point in the scan)
        lidar_time = datetime.fromtimestamp(timestamp_ns / 1e9, timezone.utc)

        print(f"LiDAR: Timestamp {lidar_time}")

        # Save LiDAR Intensity Image
        cv2.imwrite(f"{data_dir}/lidar/lidar_img_{lidar_time.timestamp()}.jpg", img_pc)
        
        scan_cnt += 1

        if scan_cnt > 100:
            break

# Main Program
if __name__ == "__main__":
    data_dir = '<save_directory>/data/'
    
    ouster_thread("10.42.0.89", data_dir)

    print("Finished processing LiDAR and Camera!")

Platform:

• Ouster Sensor? OS-1
• Ouster Firmware Version? [e.g. 2.3, 2.4, ..]
• Programming Language? Python 3.9.21
• Operating System? Ubuntu (Linux)
• Machine Architecture? x64

[matthew-lidar]

I suspect you are maxing out the bandwidth of your ethernet connection. Ethernet cameras can very easily max out the link all on their own and Ouster sensors use up to about 350 megabits depending on the beam configuration and profile settings.

Can you use iftop to check the network throughput while running both?

[chadrs2]

Using iftop, my LiDAR has a throughput of about 125 Mb and my camera is about 121 Mb