Is there a function directly for 3d reconstruction?

[lemonci]

If I have already known the camera poses (from a simulator) of the images, is there anyway for 3d reconstruction directly from glomap (a function equal to colmap point_triangulator )? I tried the latter one but the result was not so good. Thanks.

[StonerLing]

Actually GLOMAP use the triangulation function of COLMAP:

glomap/glomap/controllers/track_retriangulation.cc

Lines 13 to 24 in 18a70ee

	bool RetriangulateTracks(const TriangulatorOptions& options,
	const colmap::Database& database,
	std::unordered_map<camera_t, Camera>& cameras,
	std::unordered_map<image_t, Image>& images,
	std::unordered_map<track_t, Track>& tracks) {
	// Following code adapted from COLMAP
	auto database_cache =
	colmap::DatabaseCache::Create(database,
	options.min_num_matches,
	false, // ignore_watermarks
	{} // reconstruct all possible images
	);

And in my opinion, precise camera poses and keypoint tracks are enough for colmap point_triangulator.

[lemonci]

Actually GLOMAP use the triangulation function of COLMAP:

glomap/glomap/controllers/track_retriangulation.cc

Lines 13 to 24 in 18a70ee
bool RetriangulateTracks(const TriangulatorOptions& options,
const colmap::Database& database,
std::unordered_map<camera_t, Camera>& cameras,
std::unordered_map<image_t, Image>& images,
std::unordered_map<track_t, Track>& tracks) {
// Following code adapted from COLMAP
auto database_cache =
colmap::DatabaseCache::Create(database,
options.min_num_matches,
false, // ignore_watermarks
{} // reconstruct all possible images
);

And in my opinion, precise camera poses and keypoint tracks are enough for colmap point_triangulator.

Thanks for your kind and prompt reply!
Actually I am using habitat with the coordinates system y-up and right-handed. Since colmap/opencv is y-down right-handed, I wonder if the transformation between these two systems is TX, -TY, -TZ and QW, QX, -QY, -QZ. Thanks!

[StonerLing]

Not familiar with Habitat, the following are my personal opinions.

When changing the camera coordinate system, we alter the transformation of a 3D point from the world frame to the camera frame.
The original transformation from the world to the original camera is:

$$ P_{oc} = R\cdot P_{w}+T $$

The transformation from the original camera to Habitat's camera is:

$$ P_{hc} = \left[ \begin{array}{} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & -1 \end{array} \right]P_{oc} = R_x(180 \degree)P_{oc} $$

By combining the above transformations, the translation $(t_x,t_y,t_z)$ will change to $(t_x, -t_y, -t_z)$, the rotation $R$ will change to $R_x(180 \degree)R$, and in quaternion form: $i\cdot (w+xi+yj+zk) = −x+wi−zj+yk$. Here, $i$ represents a 180-degree rotation around the x-axis.

[lemonci]

Not familiar with Habitat, the following are my personal opinions.

When changing the camera coordinate system, we alter the transformation of a 3D point from the world frame to the camera frame. The original transformation from the world to the original camera is:

P o c = R ⋅ P w + T

The transformation from the original camera to Habitat's camera is:

P h c = [ 1 0 0 0 − 1 0 0 0 − 1 ] P o c = R x ( 180 ° ) P o c

By combining the above transformations, the translation ( t x , t y , t z ) will change to ( t x , − t y , − t z ) , the rotation R will change to R x ( 180 ° ) R , and in quaternion form: i ⋅ ( w + x i + y j + z k ) = − x + w i − z j + y k . Here, i represents a 180-degree rotation around the x-axis.

Thanks for the discussion.

I agree that it's a 180-degree rotation around the x-axis. So the angle axis is:

$$ (\text{angle, axis}) = (\begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}, \pi) $$

If we convert it to quaternion, it should be

$$ q = \begin{pmatrix} \cos(\pi/2) \\ \sin(\pi/2) \\ \cos(\pi/2) \\ \cos(\pi/2) \end{pmatrix} = [0 \ 1 \ 0 \ 0] $$

So the result will be $$q p q^{-1} = [0 1 0 0] [w x y z] [0 1 0 0]^{-1} = [w x -y -z]$$?

[StonerLing]

@lemonci
Original rotation on $P_w$ in quaternion form is $pP_w p^{-1}$, after 180-degree rotation around the x-axis, it becomes $qpP_w p^{-1}q^{-1} = (qp)P_w (qp)^{-1}$. The quaternion changes from $p$ to $qp$ not $qpq^{-1}$.

[lemonci]

---

@lemonci Original rotation on P w in quaternion form is p P w p − 1 , after 180-degree rotation around the x-axis, it becomes q p P w p − 1 q − 1 = ( q p ) P w ( q p ) − 1 . The quaternion changes from p to q p not q p q − 1 .

Hello, thanks for your help. I tried to visualize the camera poses and point cloud in colmap and found an interesting error.

What I did was selecting a few camera positions in habitat tx, ty, tz as

position_list = [[0.0, 0.0, 0.0], [0.1, 0.0, 0.0], [-0.1, 0.0, 0.0], [0.0, 0.0, -0.1], [0.1, 0.0, -0.1], [-0.1, 0.0, -0.1], [0.0, 0.0, -0.2], [0.1, 0.0, -0.2], [-0.1, 0.0, -0.2], [0.0, 0.0, -0.3], [0.1, 0.0, -0.3], [-0.1, 0.0, -0.3], [0.0, 0.0, -0.4], [0.1, 0.0, -0.4], [-0.1, 0.0, -0.4], [0.0, 0.0, -0.5], [0.1, 0.0, -0.5], [-0.1, 0.0, -0.5], [0.0, 0.0, -0.6], [0.1, 0.0, -0.6], [-0.1, 0.0, -0.6], [0.0, 0.0, -0.7], [0.1, 0.0, -0.7], [-0.1, 0.0, -0.7], [0.0, 0.0, -0.8], [0.1, 0.0, -0.8], [-0.1, 0.0, -0.8], [0.0, 0.0, -0.9], [0.1, 0.0, -0.9], [-0.1, 0.0, -0.9]]

Then at each camera position (index i), I rotate the camera with 10 degree interval along the y axis and take pictures (index j). Then I write down -QX, QW, -QZ, QY, TX, -TY, -TZ as you suggested:

    with open(f"{data_path}/sparse/model/images.txt", 'a') as images_txt:
        images_txt.write(str(i*36+j+1)+" ")
        # Write the rotation quaternion
        images_txt.write(f"{-rotation.x:.6g} ")
        images_txt.write(f"{rotation.w:.6g} ")
        images_txt.write(f"{-rotation.z:.6g} ")
        images_txt.write(f"{rotation.y:.6g} ")
        # Write the translation
        images_txt.write(f"{position[0]:.6g} ")
        images_txt.write(f"{-position[1]:.6g} ")
        images_txt.write(f"{-position[2]:.6g} ")
        # Write the camera id aka 1
        images_txt.write("1 ")
        # Write the image name
        images_txt.write(str(i*36+j+1).zfill(5)+'.jpg\r\n\r\n')
        images_txt.close()

The images.txt file looks like this:

1 -0 0.996195 -0 0.0871557 0 -0.072447 -0 1 00001.jpg

2 -0 0.984808 -0 0.173648 0 -0.072447 -0 1 00002.jpg

3 -0 0.965926 -0 0.258819 0 -0.072447 -0 1 00003.jpg
...
36 -0 -0.573577 -0 -0.819152 0 -0.072447 -0 1 00036.jpg

37 -0 -0.422619 -0 -0.906308 0.1 -0.072447 -0 1 00037.jpg

38 -0 -0.342021 -0 -0.939692 0.1 -0.072447 -0 1 00038.jpg
...
1078 -0 0.965924 -0 -0.258824 -0.1 -0.072447 0.9 1 01078.jpg

1079 -0 0.984807 -0 -0.173653 -0.1 -0.072447 0.9 1 01079.jpg

1080 -0 0.996194 -0 -0.087161 -0.1 -0.072447 0.9 1 01080.jpg

The cameras.txt is (based on 90 degree fov):

1 SIMPLE_PINHOLE 512 512 256.0 256.0 256.0

If I run

colmap feature_extractor --database_path f"{datapath}/database.db --image_path f"{datapath}/images
colmap exhaustive_matcher --database_path f"{datapath}/database.db
glomap mapper --database_path f"{datapath}/database.db --image_path f"{datapath}/images --output_path f"{datapath}/sparse/0

,
the pointcloud and camera poses will be like:

There are some errors but overall the quality is fine

If I run

colmap feature_extractor --database_path f"{datapath}/database.db --image_path f"{datapath}/images
colmap exhaustive_matcher --database_path f"{datapath}/database.db
colmap point_triangulator --database_path f"{datapath}/database.db --image_path f"{datapath}/images --input_path f"{datapath}/sparse/model --output_path f"{datapath}/sparse/0

,
the pointcloud and camera poses will be like:

The camera poses are significantly distorted from the ground truth camera poses. Is there any specific reason for that?

[lemonci]

Not familiar with Habitat, the following are my personal opinions.

When changing the camera coordinate system, we alter the transformation of a 3D point from the world frame to the camera frame. The original transformation from the world to the original camera is:

P o c = R ⋅ P w + T

The transformation from the original camera to Habitat's camera is:

P h c = [ 1 0 0 0 − 1 0 0 0 − 1 ] P o c = R x ( 180 ° ) P o c

By combining the above transformations, the translation ( t x , t y , t z ) will change to ( t x , − t y , − t z ) , the rotation R will change to R x ( 180 ° ) R , and in quaternion form: i ⋅ ( w + x i + y j + z k ) = − x + w i − z j + y k . Here, i represents a 180-degree rotation around the x-axis.

Thanks @StonerLing , I have rethought about this problem. − x + w i − z j + y k seems to be the right solution. However, I still have a question, which is to convert a camera pose in COLMAP back to Habitat. As it is again a 180-degree rotation around x-axis (since both system are right handed), the transformation will be the same, aka − w -x i − y j -z k which is not the same as w x i y j z k. What's the problem here? Thanks.

[StonerLing]

Quaternion $(−w,−x,−y,−z)$ and $(w,x,y,z)$ denote the same rotation. To enforce the convention where the scalar component w is non-negative, you can apply the following transformation:

if (q[0] < 0) {
    q = -q;
}

This ensures that the quaternion's scalar component remains positive by inverting the quaternion if $w$ is negative.

[lemonci]

Quaternion ( − w , − x , − y , − z ) and ( w , x , y , z ) denote the same rotation. To enforce the convention where the scalar component w is non-negative, you can apply the following transformation:

if (q[0] < 0) {
q = -q;
}

This ensures that the quaternion's scalar component remains positive by inverting the quaternion if w is negative.

Thanks for all the help. And eventually I found the quaternion transformation should be x wi -zj -yk based on the point_triangulator result, which is quite bizarre...