My transformation (matrix) skills are rusty, but I think you are effectively saying that in our compose_with structure that H1 * H2 != H2 * H1. Is that correct? That would effectively be the change you are suggesting, correct?

In this green_screen example, point transformation to secondary color camera coordinate first, so it should multiply tr_secondary_depth_to_secondary_color first.

Can you share more detail on this? A link would be fine. Are you able to test this and confirm it improves the results? Due to COVID testing from home is tricky.

My transformation (matrix) skills are rusty, but I think you are effectively saying that in our compose_with structure that H1 * H2 != H2 * H1. Is that correct? That would effectively be the change you are suggesting, correct?

Yes,that is all I mean.

Can you share more detail on this? A link would be fine. Are you able to test this and confirm it improves the results? Due to COVID testing from home is tricky.

In the comment of line 226，it said
// We now have the secondary depth to secondary color transform. We also have the transformation from the // secondary color perspective to the main color perspective from the calibration earlier. Now let's compose the // depth secondary -> color secondary, color secondary -> color main into depth secondary -> color main
so the point transformates to secondary color camera coordinate first.
And then make X the point (x,y,z),

as you can see, the order can`t be exchanged. So we should mutiply the tr_secondary_depth_to_secondary_color first.

And I rewrite green_screen to show an point cloud.

the order in your sample show like this. It make an small angel error with two point cloud.

and when I change the order of mutiplication, the angel disapear.

Sorry, English is not my native language and this paragraph may be difficult to understand.