Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: new minimap overlay #7151

Open
wants to merge 71 commits into
base: main
Choose a base branch
from
Open

feat: new minimap overlay #7151

wants to merge 71 commits into from

Conversation

KhalilSelyan
Copy link
Contributor

Description

This PR introduces a minimap plugin for RViz, designed to provide a 2D overlay of the map within the RViz interface. The plugin leverages Qt and 2D overlay utilities to display map tiles based on GPS coordinates, enhancing situational awareness for users.

Related links

Tests performed

  • Manual testing of the plugin in the RViz environment to ensure the map tiles load correctly based on varying GPS coordinates.
  • Validation of the overlay alignment and responsiveness within RViz.
  • Code review and static analysis for Qt and 2D overlay integration.

Notes for reviewers

  • This PR is dependent on updates to the 2D overlay utilities, which are being handled in a separate PR. Please ensure this dependency is resolved before merging.
  • The map tiles are currently fetched from a predefined tile server. Future enhancements may include configurable tile servers or offline tile support.

Effects on system behavior

  • The introduction of the minimap plugin will provide users with a visual representation of their surroundings based on GPS coordinates, aiding in navigation and spatial awareness.
  • The plugin is designed to be non-intrusive and will not interfere with other RViz functionalities.

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

  • The PR follows the pull request guidelines.
  • The PR has been properly tested.
  • The PR has been reviewed by the code owners.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

  • There are no open discussions or they are tracked via tickets.
  • The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

Sorry, something went wrong.

@KhalilSelyan KhalilSelyan added component:localization Vehicle's position determination in its environment. (auto-assigned) type:ui-ux User interface, user experience, graphical user interfaces. labels May 28, 2024
@KhalilSelyan KhalilSelyan requested a review from xmfcx May 28, 2024 09:26
@KhalilSelyan KhalilSelyan self-assigned this May 28, 2024
@KhalilSelyan KhalilSelyan linked an issue May 28, 2024 that may be closed by this pull request
5 tasks
@KhalilSelyan KhalilSelyan added the component:map Map creation, storage, and loading. (auto-assigned) label May 28, 2024
@github-actions github-actions bot added type:documentation Creating or refining documentation. (auto-assigned) component:common Common packages from the autoware-common repository. (auto-assigned) and removed component:localization Vehicle's position determination in its environment. (auto-assigned) component:map Map creation, storage, and loading. (auto-assigned) labels May 28, 2024
@KhalilSelyan KhalilSelyan marked this pull request as ready for review June 4, 2024 07:24
@KhalilSelyan KhalilSelyan changed the title Minimap overlay feat: new minimap overlay Jun 4, 2024
Copy link

github-actions bot commented Jun 25, 2024

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

@KhalilSelyan
Copy link
Contributor Author

Current Progress, able to visualize both vehicle position and goal pose (mostly correctly) still unsure of why sometimes the calculations get wrong and with the zoom as well it gets very wrong needs improving
https://youtu.be/Eu0zbz_w2Js

@KhalilSelyan
Copy link
Contributor Author

KhalilSelyan commented Jul 9, 2024

@xmfcx @mitsudome-r
I think this should be tested, before merging.
Also, for some reason i was not able to include
autoware_adapi_v1_msgs::msg::VehicleKinematics
tier4_planning_msgs::msg::route_state

It would error out like this, any ideas how to fix this ? there are a few things i need to add and they need both these messages
Screenshot from 2024-07-09 14-56-46

also copyright messages are missing there are a few small details to be fixed

@xmfcx xmfcx added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Jul 10, 2024
Copy link

codecov bot commented Jul 10, 2024

Codecov Report

Attention: Patch coverage is 0% with 448 lines in your changes missing coverage. Please review.

Project coverage is 28.65%. Comparing base (9b20ced) to head (758e202).
Report is 25 commits behind head on main.

Files Patch % Lines
...toware_minimap_overlay_rviz_plugin/src/minimap.cpp 0.00% 185 Missing ⚠️
..._minimap_overlay_rviz_plugin/src/overlay_utils.cpp 0.00% 103 Missing ⚠️
...are_minimap_overlay_rviz_plugin/src/tile_field.cpp 0.00% 88 Missing ⚠️
...ware_minimap_overlay_rviz_plugin/src/goal_pose.cpp 0.00% 39 Missing ⚠️
.../autoware_minimap_overlay_rviz_plugin/src/tile.cpp 0.00% 32 Missing ⚠️
...imap_overlay_rviz_plugin/src/include/goal_pose.hpp 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7151      +/-   ##
==========================================
- Coverage   28.76%   28.65%   -0.12%     
==========================================
  Files        1592     1598       +6     
  Lines      116667   117115     +448     
  Branches    49937    49974      +37     
==========================================
  Hits        33554    33554              
- Misses      74008    74456     +448     
  Partials     9105     9105              
Flag Coverage Δ *Carryforward flag
differential 0.00% <0.00%> (?)
total 28.76% <ø> (ø) Carriedforward from 9b20ced

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@KhalilSelyan
Copy link
Contributor Author

https://youtu.be/mKtN2YPag9w Latest updates including : vehicle position with orientation, trajectory drawn

@KhalilSelyan
Copy link
Contributor Author

I think it is up for testing for whoever wants to try it.
Things that can be improved but haven't been figured out yet :

  • Smooth movement instead of staggered movement in minimap (tried a smoothUpdate function but hasn't shown any upgrades)
  • Edge Cases where goal point disappears out of the minimap while it should be drawn in specific pixel position (rare but still happens, not able to detect the error in drawing logic yet)

Comment on lines +474 to +585
int zone;
bool north_p;
double origin_lat = property_origin_lat_->getFloat();
double origin_lon = property_origin_lon_->getFloat();
double origin_x, origin_y, gamma, k;

// Convert origin to UTM coordinates
GeographicLib::UTMUPS::Forward(
origin_lat, origin_lon, zone, north_p, origin_x, origin_y, gamma, k);

// Calculate global UTM coordinates by adding local offsets
double global_x = origin_x + x;
double global_y = origin_y + y;

// Convert back to geographic coordinates
double lat, lon;
GeographicLib::UTMUPS::Reverse(zone, north_p, global_x, global_y, lat, lon);

return {lat, lon};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This projection depends on how the map is loaded in Autoware.
You should be able to retrieve the information from /map/map_projector_info topic and switch the projection to support different projection methods.

@KhalilSelyan
Copy link
Contributor Author

KhalilSelyan commented Jul 24, 2024

Multiple Providers Demo :

image

Is not committed as of now until confirmation of ToS if it allows the use of these providers being: OpenstreetMaps, ArcGIS satellite tiles and google maps hybrid tiles

@xmfcx
Copy link
Contributor

xmfcx commented Jul 24, 2024

By default, can you set the radius to half of the height of the other overlays? So they all will have the same radius.

Also could you put it to the bottom left, it won't fit to the top the way things are sized.

Forza Horizon 5:
Forza Horizon 5

@KhalilSelyan
Copy link
Contributor Author

@xmfcx none of this is commited for now so it still is circular in the bottom left corner, there was a small conversation happening in the discord here : link

By default, can you set the radius to half of the height of the other overlays? So they all will have the same radius.

It was circular, this screenrecord here was to show armağan his suggestion of having it as a rounded square

Also could you put it to the bottom left, it won't fit to the top the way things are sized.

It was in the bottom left, this screenrecord here was to show armağan his suggestion of having it on top right.

@xmfcx
Copy link
Contributor

xmfcx commented Jul 24, 2024

I'm alright with users being able to select it by tuning radius to their liking.

I was proposing to make it rounded rectangle with radius of height of the other overlays. All will have the same form.

About the position, "Anchors" could be parameterized:

AnchorHorizontal

  • Center
  • Left
  • Right

AnchorVertical

  • Center
  • Top
  • Bottom

Margins

  • Horizontal (in x)
  • Vertical (in y)

I'm also aware of the conversation with @armaganarsln , regardless, it should not be adjacent to any corners, it should have the same margin as the other overlays.

image

Looking at his suggestion, he also put it with margin from the edges too.

@KhalilSelyan
Copy link
Contributor Author

@xmfcx

I'm alright with users being able to select it by tuning radius to their liking.
About the position, "Anchor" could be a parameter with 4 options:

  • TopLeft
  • TopRight
  • BottomLeft
  • BottomRight

And Margins:

  • Horizontal
  • Vertical

Yeah, me too i think this would a good addition as well. I can either open a new PR for it separately for the other 2 already existing overlays or wait until this PR is approved and add it at the end of it and merge only 1 PR with all the changes for extracting the Margins and Anchor positions

Looking at his suggestion, he also put it with margin from the edges too.

Yeah looks like it, on the meeting call he said something about having it stick to the edges that's why i had made it that way when sharing the demo of what it would look like.

@xmfcx
Copy link
Contributor

xmfcx commented Jul 24, 2024

I can either open a new PR for it separately for the other 2 already existing overlays or wait until this PR is approved and add it at the end of it and merge only 1 PR with all the changes for extracting the Margins and Anchor positions

We can start working on the others once this one is finished and merged.

@KhalilSelyan
Copy link
Contributor Author

I added the new properties for the anchor positions and margins, as well as the tile provider although currently only committed the link for the openstreetmap tiles
image

KhalilSelyan and others added 28 commits January 14, 2025 11:12
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
…goalpose

Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
…path

Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Co-authored-by: Ryohsuke Mitsudome <43976834+mitsudome-r@users.noreply.github.com>
…viz_plugin/src/minimap.cpp

Co-authored-by: Ryohsuke Mitsudome <43976834+mitsudome-r@users.noreply.github.com>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
…origin_lat and origin_lon parameters

Signed-off-by: KhalilSelyan <khalil@leodrive.ai>
@mitsudome-r
Copy link
Member

@mitsudome-r Will finish the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:common Common packages from the autoware-common repository. (auto-assigned) run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) status:stale Inactive or outdated issues. (auto-assigned) type:documentation Creating or refining documentation. (auto-assigned) type:ui-ux User interface, user experience, graphical user interfaces.
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

Improve RViz UI - Adding 2D Minimap
4 participants