the quality of grasp pose is not so good using CloudSamples

[xibeisiber]

Hi
I'm using gpd2.0 and CloudSamples as input to detection the grasp pose for a bottle. To speed up the detection, in the CloudSamples.msg, the cloud_sources includes the bottle and only some other points around it, the samples are the points of the bottle.

If I turn on filter_approach_direction in caffe_params.cfg and set "direction = 0 0 1", to obtain good grasp pose, I have to make the camera (z axis) parallel to the ground. In this picture, the camera (z axis) has a small angle with the ground plane.

If I turn off filter_approach_direction, the grasp poses are not usable. see this picture.

Is there any way to improve the detection quality without using filter_approach_direction?

Thanks very much!

[atenpas]

Add a 2nd camera view (assuming this is taken with one view)?

Instead of making the camera point in a specific direction, you could transform the point cloud into a frame where the directions are easier to specify. For example, say there's a "robot" frame with x forward, y left and z up (as in ROS). Then you can use this frame with filter_approach_direction.

[xibeisiber]

Thanks for your reply!
Yes, the pointcloud is taken with one camera.
It would be difficult for my robot to install a 2nd camera for now. I will consider it later.

The second suggestion would be a sound choice, I think. But since I have no prior knowledge where the bottle is, I have to specify the tf of this vitual frame with others after object detection and before GPD detection, and should keep its tf fixed before I get the grasp pose and transform it to base_link. Only after this can I allow the change of this frame ( with another object to be grasped) . Technological this can be implemented, I will try if this works fine.

Thanks!

[xibeisiber]

I create a virtual frame where the z axis directs from the robot base to the bottle, transform the pointcloud into this frame and send it to GPD, filter the approach direction as this z direction. Now the approach direction seems fine, but the axis&binormal vectors of grasp pose are not desired.... see image and image

will it produce better results if I use openvino instead of caffe?

[atenpas]

Generally, openvino works better yes, because it's what I was using in the lab most recently, and therefore is most maintained. The other frameworks, like Caffe, are basically "leftovers" from previous versions of the code (or when newer frameworks where not available).

Would you mind uploading the point cloud somewhere and the configuration file that you use? I can have a quick look at it sometime.

[xibeisiber]

I just realized yesterday I forgot to turn PCL_GPU off when compiling GPD after switching my pc. After I turned it off, the quality is much better.

I plan to install Openvino again. By the way, are you using OpenVino2020 or OpenVino2018? OpenVino2020 seems to only support ubuntu 18.04 or above. I'm using ubuntu 16.04. But I can't find the installer of openvino2018 on its website.