The Autoware Foundation
Privately Owned Vehicle Work Group Meeting - 2025/02/17 - Slot 2 #5790
m-zain-khawaja
started this conversation in
Working group meetings
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Agenda
Discussion
@m-zain-khawaja :
The fourth and final phase of Scene3D training using both simulator and real-world data is complete. In this phase, a dedicated DepthContext block has been introduced in order to allow the network to more accurately extract global image features and their relationships for background scene elements such as trees and vegetation - which were not being accurately captured in the prior network version in which the SceneContext block pre-trained for SceneSeg was utilized.
Furthermore, it was found that the MUAD dataset had some anomalous data samples with objects were very close to the virtual camera in the simulated environment, and at times, the sim-camera was clipping into the vehicle in front. These images were filtered out from the MUAD dataset using a minimum depth filter of 1m - resulting in much more stable training results for samples from this dataset.
An important addition to the network in this fourth phase is the introduction of a scale factor whereby the predicted network depth is scaled by the scale factor to achieve the final depth as follow:
metric_inverse_depth = scale_factor * predicted_network_depth_mapThe scale_factor is derived from the viewing angle of the camera and is designed to help the network better predict scene depth given prior information about metric scene scale, as different FOV cameras can view the same scene resulting in drastically different images and therefore incorrect metric depth estimates.
For example, the same scene looks very different when captured from a narrow angle camera (left) vs a wide-angle camera (right)
This version of the network has achieved the best performance on validation data with sharp and clear boundaries for both foreground and background objects, including vegetation and poles/infrastructure as seen in snapshots below.
mAE for the metric inverse depth map:
EgoPath Dataset Curation Update
@TranHuuNhatHuy - awaiting update.
@m-zain-khawaja to review and merge PR by @docjag for ROADWorks dataset
Dataset curation tracking
Ego Path Network Design
Literature review and network design discussion topics: @m-zain-khawaja will design a baseline version of the EgoPath network which can be used by other contributors to make tweaks, experiments and adjustments. @m-zain-khawaja will also share the processed datasets for EgoPath with open-source contributors so that they can begin data downloading ahead of network training.
EgoLanes Dataset Curation Update
@aadarshkt has begun dataset curation of the BDD100K dataset the EgoLanes Network. He has completed saving of the binary lane masks as well as the visualizations of the lanes. He has also been able to classify the ego left and ego right lanes using the lane 'anchors', however, there are a few scenarios where the classification is not correct, and he is investigating these scenarios. He is currently working on applying the correct scale to the images as per guidance by @sarun-hub and has completed the JSON format changes as per guidance by @devang-marvania
Attendees
Zoom Meeting Video Recording
Video Meeting Link
Please contact the work group lead (@m-zain-khawaja) to request access to a recording of this meeting.
Beta Was this translation helpful? Give feedback.
All reactions