RawImage proto file frame of reference is not matching the comment

[kevswims]

Description

In the RawImage.proto file here https://github.com/foxglove/schemas/blob/f8d19d032541f8241b5e1576dd833fa09545bf40/schemas/proto/foxglove/RawImage.proto#LL14C1-L14C196 there is a comment that indicates a frame of reference but the parameter is just a string frame_id. I'm not sure which of these is supposed to be there but we would like to have an actual frame of reference parameter on images.

[foxhubber[bot]]

Internal tracking ticket: FG-3725

[jtbandes]

Hi @kevswims, can you describe what you mean by "an actual frame of reference parameter"? This frame_id corresponds with the 3D panel's Transforms feature: https://foxglove.dev/docs/studio/panels/3d#transforms Is there something you're trying to accomplish that the current feature set does not support?

[kevswims]

@jtbandes do you have an example somewhere that sets the frame_id then?

I'm not quite understanding how the frame of reference is supposed to be stored as a string.

[jtbandes]

You can see some example code in nuscenes2mcap which is used to make our sample dataset:

The image is positioned in 3D space based on the FrameTransform messages with the corresponding frame_id.

I confirmed that Studio actually doesn't use the image's frame_id in the app right now, but rather the frame_id of the chosen Camera Info topic. You can find relevant discussion at foxglove/schemas#59. The Image's frame_id is there for completeness and so users can cross-reference raw data to make sure the right topics are being used.

[kevswims]

Thanks @jtbandes , that makes much more sense now. Once I try this out I'll see if I can put up a PR that cleans up the documentation comment on the frame_id.

[defunctzombie]

Closing this out as it seems answered.