Radar Perception Architecture Design

[scepter914]

Summary

We would like to initiate the discussion for perception architecture with radars for Autoware.
This proposal follows the Perception architecture discussion.
As described in the radar sensing proposal, radar output can be divided into two types; scan and objects.
According to these output types, We suggest two sensor fusion types for radar output:

• Radar fusion in the object layer in the tracking module
• Radar fusion in the ros-perception/radar_msgs/msg/RadarScan.msg layer in the detection module

Perception pipeline with radar

This figure can be found at https://github.com/scepter914/autoware-radar-architecture-proposal/blob/main/perception/figure/perception_pipeline.drawio.svg

For more detail, please see radar perception design.

Discussion

• Radar perception pipeline
• Document location
  • We see autoware-documentation, but it is not clear where to put the architecture design document.
  • After discussions and if maintainers can instruct where to put the document, We will send a PR for it.

[xmfcx]

@xmfcx provide the place in the https://github.com/autowarefoundation/autoware-documentation/

[plane-li]

I agree with you to add a function module to remove the noise of raw data before fusion.

[scepter914]

I read your comment back now, I guess that I leaped in logic.

a function module to handle the radar data using the RCS

You mean that this module handle radar object using RCS of radar pointcloud to improve detection result (my comment)?
or that the message of radar objects need to have RCS information?

[plane-li]

"The message of radar objects need to have RCS information"

[scepter914]

@plane-li

From now Autoware architecture, I think objects information should handle in confidence (probability) 0.0 - 1.0 value.
Considering Classification of DetectedObjects, which is used mainly for LiDAR detection, now Autoware architecture handle not intensity from LiDAR, but detection confidence.

But I agree with your suggestion, so I suggest to divide for another discussion, "how objects have points information including LiDAR data, radar data, and segmentation result".

[plane-li]

I think it is hard to define a message including LiDAR data, radar data, and segmentation result.
Sometimes, the object cannot be detected by the Lidar and Radar at the same time.Can the message contain lidar_detect_flag and radar_detect_flag?

[storrrrrrrrm]

which camera-3d-detection model are you going to use

[scepter914]

Actually we don't have any plan to use what camera-3d-detection model.
Because the scope of this proposal is whole architecture for sensing and perception, I think that if we don't decide which camera-3d-detection model is used in detail here, it's a not problem.
(But I agree with your interest, and if you need I suggest that you send another discussion. Maybe the discussion will be active.)

[scepter914]

As first prototype, we already integrate to autoware.universe and start to maintain the packages.
And new design start to be discussed at https://github.com/orgs/autowarefoundation/discussions/5264.
So I close this discussion and if you want to ask questions, please make a new issue.