feat(tier4_perception_launch): add image segmentation based pointcloud filter #7225

badai-nguyen · 2024-06-03T01:29:46Z

Description

This is to add image semantic segmentation based pointcloud filter into perception pipeline to sold the issue Add a new pointcloud filter based on image semantic segmentation #5594
The default mode is still disable. Make sure the tensorrt_yolox output semantic segmentation mask image is output by merging feat(tensorrt yolox): inference and publish mask image from yolox model with semantic segmentation header #5553 and fix: add download url of yolox with semantic segmentation header autoware#4012 when the filter is enable.

Releated Link

TIER IV INTERNAL LINK

Tests performed

Not applicable.

Effects on system behavior

Not applicable.

Interface changes

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

I've confirmed the contribution guidelines.
The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

The PR follows the pull request guidelines.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

…d filter Signed-off-by: badai-nguyen <[email protected]>

Signed-off-by: badai-nguyen <[email protected]>

codecov · 2024-06-24T03:15:49Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 2.64%. Comparing base (507e3f4) to head (18047c0).
Report is 115 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #7225       +/-   ##
==========================================
- Coverage   14.84%   2.64%   -12.21%     
==========================================
  Files        1999     165     -1834     
  Lines      139163   10099   -129064     
  Branches    43716    1579    -42137     
==========================================
- Hits        20661     267    -20394     
+ Misses      95731    9776    -85955     
+ Partials    22771      56    -22715

Flag	Coverage Δ
differential	`2.64% <ø> (?)`
total	`?`

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…_launch

github-actions · 2024-07-05T03:26:02Z

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

You've checked our contribution guidelines.
Your PR follows our pull request guidelines.
All required CI checks pass before marking the PR ready for review.

Signed-off-by: badai-nguyen <[email protected]>

technolojin

LGTM

Signed-off-by: badai-nguyen <[email protected]>

YoshiRi

LGTM

launch/tier4_perception_launch/package.xml

This reverts commit 5adfef6.

yukkysaito

LGTM

…d filter (autowarefoundation#7225) * feat(tier4_perception_launch): add image segmentation based pointcloud filter Signed-off-by: badai-nguyen <[email protected]> * chore: typo Signed-off-by: badai-nguyen <[email protected]> * fix: detection launch Signed-off-by: badai-nguyen <[email protected]> * chore: add maintainer Signed-off-by: badai-nguyen <[email protected]> * Revert "chore: add maintainer" This reverts commit 5adfef6. --------- Signed-off-by: badai-nguyen <[email protected]> Signed-off-by: palas21 <[email protected]>

…d filter (autowarefoundation#7225) * feat(tier4_perception_launch): add image segmentation based pointcloud filter Signed-off-by: badai-nguyen <[email protected]> * chore: typo Signed-off-by: badai-nguyen <[email protected]> * fix: detection launch Signed-off-by: badai-nguyen <[email protected]> * chore: add maintainer Signed-off-by: badai-nguyen <[email protected]> * Revert "chore: add maintainer" This reverts commit 5adfef6. --------- Signed-off-by: badai-nguyen <[email protected]>

…d filter (#7225) * feat(tier4_perception_launch): add image segmentation based pointcloud filter Signed-off-by: badai-nguyen <[email protected]> * chore: typo Signed-off-by: badai-nguyen <[email protected]> * fix: detection launch Signed-off-by: badai-nguyen <[email protected]> * chore: add maintainer Signed-off-by: badai-nguyen <[email protected]> * Revert "chore: add maintainer" This reverts commit 5adfef6. --------- Signed-off-by: badai-nguyen <[email protected]>

…d filter (autowarefoundation#7225) * feat(tier4_perception_launch): add image segmentation based pointcloud filter Signed-off-by: badai-nguyen <[email protected]> * chore: typo Signed-off-by: badai-nguyen <[email protected]> * fix: detection launch Signed-off-by: badai-nguyen <[email protected]> * chore: add maintainer Signed-off-by: badai-nguyen <[email protected]> * Revert "chore: add maintainer" This reverts commit 5adfef6. --------- Signed-off-by: badai-nguyen <[email protected]>

….29.0 (#1526) * feat(tensorrt yolox): inference and publish mask image from yolox model with semantic segmentation header (autowarefoundation#5553) * add segmentation model Signed-off-by: badai-nguyen <[email protected]> fix: add multitask for segment Signed-off-by: badai-nguyen <[email protected]> * fix: publish mask Signed-off-by: badai-nguyen <[email protected]> * feat: publish colorized mask Signed-off-by: badai-nguyen <[email protected]> * fix: resize yolox mask Signed-off-by: badai-nguyen <[email protected]> * fix: add memory allocate operations Signed-off-by: Manato HIRABAYASHI <[email protected]> * refactor: remove underscore for a local variable Signed-off-by: Manato HIRABAYASHI <[email protected]> * chore: add condition to check the number of subscriber for newly added topics Signed-off-by: Manato HIRABAYASHI <[email protected]> * chore: pre-commit Signed-off-by: badai-nguyen <[email protected]> * fix: add roi overlapping segment Signed-off-by: badai-nguyen <[email protected]> * fix: roi overlay segment Signed-off-by: badai-nguyen <[email protected]> * chore: refactor Signed-off-by: badai-nguyen <[email protected]> * docs: update readme Signed-off-by: badai-nguyen <[email protected]> * fix: update model name Signed-off-by: badai-nguyen <[email protected]> * fix: add utils into tensorrt_yolox Signed-off-by: badai-nguyen <[email protected]> * fix: launch file Signed-off-by: badai-nguyen <[email protected]> * chore: remove unnecessary depend Signed-off-by: badai-nguyen <[email protected]> * chore: fix yaml file Signed-off-by: badai-nguyen <[email protected]> * chore: remove duplicated param in launch Signed-off-by: badai-nguyen <[email protected]> * fix: semantic class Signed-off-by: badai-nguyen <[email protected]> * docs: update readme Signed-off-by: badai-nguyen <[email protected]> * chore: update default param Signed-off-by: badai-nguyen <[email protected]> * fix: add processing time topic Signed-off-by: badai-nguyen <[email protected]> * chore: typo Signed-off-by: badai-nguyen <[email protected]> * style(pre-commit): autofix * chore: fix cspell error Signed-off-by: badai-nguyen <[email protected]> * fix: yolox default param Signed-off-by: badai-nguyen <[email protected]> * chore: rename debug topics Signed-off-by: badai-nguyen <[email protected]> * chore: rename debug topics Signed-off-by: badai-nguyen <[email protected]> * docs: update model description Signed-off-by: badai-nguyen <[email protected]> * style(pre-commit): autofix * fix: launch Signed-off-by: badai-nguyen <[email protected]> * refactor: unpublish mask for single task Signed-off-by: badai-nguyen <[email protected]> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <[email protected]> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <[email protected]> * Update perception/tensorrt_yolox/src/tensorrt_yolox.cpp Co-authored-by: Manato Hirabayashi <[email protected]> * style(pre-commit): autofix * docs: update reamde Signed-off-by: badai-nguyen <[email protected]> * style(pre-commit): autofix * fix: skip mask size as yolox output Signed-off-by: badai-nguyen <[email protected]> --------- Signed-off-by: badai-nguyen <[email protected]> Signed-off-by: Manato HIRABAYASHI <[email protected]> Co-authored-by: Manato HIRABAYASHI <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Manato Hirabayashi <[email protected]> * fix(tensorrt_yolox): add run length encoding for sematic segmentation mask (autowarefoundation#7905) * fix: add rle compress Signed-off-by: badai-nguyen <[email protected]> * fix: rle compress Signed-off-by: badai-nguyen <[email protected]> * fix: move rle into utils Signed-off-by: badai-nguyen <[email protected]> * chore: pre-commit Signed-off-by: badai-nguyen <[email protected]> * Update perception/autoware_tensorrt_yolox/src/utils.cpp Co-authored-by: Yukihiro Saito <[email protected]> * fix: remove unused variable Signed-off-by: badai-nguyen <[email protected]> * Update perception/autoware_tensorrt_yolox/src/utils.cpp Co-authored-by: Manato Hirabayashi <[email protected]> * style(pre-commit): autofix * feat: add unit test for utils Signed-off-by: badai-nguyen <[email protected]> * style(pre-commit): autofix * fix: unit test Signed-off-by: badai-nguyen <[email protected]> * chore: change to explicit index Signed-off-by: badai-nguyen <[email protected]> * style(pre-commit): autofix * fix: cuda cmake Signed-off-by: badai-nguyen <[email protected]> * fix: separate unit test into different PR Signed-off-by: badai-nguyen <[email protected]> --------- Signed-off-by: badai-nguyen <[email protected]> Co-authored-by: Yukihiro Saito <[email protected]> Co-authored-by: Manato Hirabayashi <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * refactor(tensorrt_yolox): move utils into perception_utils (autowarefoundation#8435) * chore(tensorrt_yolo): refactor utils Signed-off-by: badai-nguyen <[email protected]> * style(pre-commit): autofix * fix: tensorrt_yolox Signed-off-by: badai-nguyen <[email protected]> --------- Signed-off-by: badai-nguyen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * fix(perception_utils): install library after build (autowarefoundation#8501) Signed-off-by: Manato HIRABAYASHI <[email protected]> * fix(image_projection_based_fusion): resize sematic segmentation mask as input image size (autowarefoundation#7635) Signed-off-by: badai-nguyen <[email protected]> * fix(image_projection_based_fusion): segmentation pointcloud fusion param update (autowarefoundation#7858) * fix(image_projection_based_fusion): add run length decoding for segmentation_pointcloud_fusion (autowarefoundation#7909) * fix: add rle decompress Signed-off-by: badai-nguyen <[email protected]> * style(pre-commit): autofix * fix: use rld in tensorrt utils Signed-off-by: badai-nguyen <[email protected]> * fix: rebase error Signed-off-by: badai-nguyen <[email protected]> * fix: dependency Signed-off-by: badai-nguyen <[email protected]> * fix: skip publish debug mask Signed-off-by: badai-nguyen <[email protected]> * Update perception/autoware_image_projection_based_fusion/src/segmentation_pointcloud_fusion/node.cpp Co-authored-by: kminoda <[email protected]> * style(pre-commit): autofix * Revert "fix: skip publish debug mask" This reverts commit 30fa9ae. --------- Signed-off-by: badai-nguyen <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: kminoda <[email protected]> * feat(tier4_perception_launch): add image segmentation based pointcloud filter (autowarefoundation#7225) * feat(tier4_perception_launch): add image segmentation based pointcloud filter Signed-off-by: badai-nguyen <[email protected]> * chore: typo Signed-off-by: badai-nguyen <[email protected]> * fix: detection launch Signed-off-by: badai-nguyen <[email protected]> * chore: add maintainer Signed-off-by: badai-nguyen <[email protected]> * Revert "chore: add maintainer" This reverts commit 5adfef6. --------- Signed-off-by: badai-nguyen <[email protected]> * style(pre-commit): autofix * fix(autoware_image_projection_based_fusion): resolve issue with segmentation pointcloud fusion node failing with multiple mask inputs (autowarefoundation#8769) --------- Signed-off-by: badai-nguyen <[email protected]> Signed-off-by: Manato HIRABAYASHI <[email protected]> Co-authored-by: Manato HIRABAYASHI <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Manato Hirabayashi <[email protected]> Co-authored-by: Yukihiro Saito <[email protected]> Co-authored-by: kminoda <[email protected]> Co-authored-by: Yi-Hsiang Fang (Vivid) <[email protected]>

github-actions bot added component:perception Advanced sensor data processing and environment understanding. (auto-assigned) component:launch Launch files, scripts and initialization tools. (auto-assigned) labels Jun 3, 2024

badai-nguyen added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Jun 3, 2024

badai-nguyen mentioned this pull request Jun 10, 2024

update perception pipeline node diagram for segmenation_based_filter autowarefoundation/autoware-documentation#569

Closed

4 tasks

badai-nguyen added 2 commits June 24, 2024 12:07

feat(tier4_perception_launch): add image segmentation based pointclou…

952aedd

…d filter Signed-off-by: badai-nguyen <[email protected]>

chore: typo

18047c0

Signed-off-by: badai-nguyen <[email protected]>

badai-nguyen force-pushed the fix/add_segmentation_based_pointcloud_filter_launch branch from 98ed17e to 18047c0 Compare June 24, 2024 03:07

badai-nguyen marked this pull request as ready for review June 24, 2024 03:17

badai-nguyen requested review from kminoda, ktro2828, miursh, tzhong518, YoshiRi and yukkysaito as code owners June 24, 2024 03:17

badai-nguyen marked this pull request as draft July 2, 2024 05:27

Merge branch 'main' into fix/add_segmentation_based_pointcloud_filter…

f03adeb

…_launch

github-actions bot removed the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label Jul 5, 2024

badai-nguyen removed request for ktro2828 and tzhong518 July 5, 2024 03:28

badai-nguyen marked this pull request as ready for review July 5, 2024 03:37

fix: detection launch

f7df7a3

Signed-off-by: badai-nguyen <[email protected]>

badai-nguyen requested a review from technolojin July 5, 2024 07:03

technolojin approved these changes Jul 5, 2024

View reviewed changes

chore: add maintainer

5adfef6

Signed-off-by: badai-nguyen <[email protected]>

YoshiRi approved these changes Jul 5, 2024

View reviewed changes

yukkysaito reviewed Jul 5, 2024

View reviewed changes

launch/tier4_perception_launch/package.xml Outdated Show resolved Hide resolved

Revert "chore: add maintainer"

3134879

This reverts commit 5adfef6.

yukkysaito approved these changes Jul 5, 2024

View reviewed changes

badai-nguyen merged commit b61db78 into autowarefoundation:main Jul 5, 2024
24 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(tier4_perception_launch): add image segmentation based pointcloud filter #7225

feat(tier4_perception_launch): add image segmentation based pointcloud filter #7225

badai-nguyen commented Jun 3, 2024 •

edited

Loading

codecov bot commented Jun 24, 2024 •

edited

Loading

github-actions bot commented Jul 5, 2024 •

edited

Loading

technolojin left a comment

YoshiRi left a comment

yukkysaito left a comment

feat(tier4_perception_launch): add image segmentation based pointcloud filter #7225

feat(tier4_perception_launch): add image segmentation based pointcloud filter #7225

Conversation

badai-nguyen commented Jun 3, 2024 • edited Loading

Description

Releated Link

Tests performed

Effects on system behavior

Interface changes

Pre-review checklist for the PR author

In-review checklist for the PR reviewers

Post-review checklist for the PR author

codecov bot commented Jun 24, 2024 • edited Loading

Codecov Report

github-actions bot commented Jul 5, 2024 • edited Loading

technolojin left a comment

Choose a reason for hiding this comment

YoshiRi left a comment

Choose a reason for hiding this comment

yukkysaito left a comment

Choose a reason for hiding this comment

badai-nguyen commented Jun 3, 2024 •

edited

Loading

codecov bot commented Jun 24, 2024 •

edited

Loading

github-actions bot commented Jul 5, 2024 •

edited

Loading