CMake search path wrongly separated

[nschoe]

Hi,

I have been packaging a few additional ROS packages to the ones already packaged (I'm happy to make a merge request if you're interested), this has worked very well, but I have an issue for pcl_ros.

My fork and the branch in which I have packaged the additional ROS packages can be found here: https://github.com/nschoe/nixpkgs/tree/airapkgs_PCL_ROS . Everything is in pkgs/development/ros-modules/ as you started it.
I'm linking to my own github because I haven't double-checked the code enough to offer a merge request (but I can, if it's better).
Just so you know: I have freshly (like yesterday) git pull your nixos-unstable branch, and merged it in my PCL_ROS branch. So this branch is up-to-date with yours.

So pcl_ros depends on pcl_conversions, so I have packaged pcl_conversions too (https://github.com/nschoe/nixpkgs/blob/airapkgs_PCL_ROS/pkgs/development/ros-modules/pcl_conversions/default.nix).

When I create a simple nix-shell with pcl_conversions in buildInputs, it work no problem.

Then I packaged pcl_ros (https://github.com/nschoe/nixpkgs/blob/airapkgs_PCL_ROS/pkgs/development/ros-modules/pcl_ros/default.nix), but when I put pcl_ros in my shell.nix's buildInputs, this time I get an error (here's the end of it):

-- Found PythonInterp: /nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/bin/python (found version "3.6.6")
-- Using PYTHON_EXECUTABLE: /nix/store/nrl0l79a48924xb0897ap572xf29ciir-python3-3.6.6/bin/python
-- Using default Python package layout
-- Found PY_em: /nix/store/bsga2zrxh8nrd76j3ixv7yqi5h8bwa7k-empy-3.3.2/lib/python3.6/site-packages/em.py
-- Using empy: /nix/store/bsga2zrxh8nrd76j3ixv7yqi5h8bwa7k-empy-3.3.2/lib/python3.6/site-packages/em.py
-- Using CATKIN_ENABLE_TESTING: OFF
-- catkin 0.7.11
CMake Error at /nix/store/fa7sbb3fhaxwmqh1v096gcyc2rd606wc-pcl_conversions-1.6.2/share/pcl_conversions/cmake/pcl_conversionsConfig.cmake:113 (message):
  Project 'pcl_conversions' specifies
  '/nix/store/m8fkjfznkx88qmn6mi4x9q7njamirxjx-eigen-3.3.4//nix/store/m8fkjfznkx88qmn6mi4x9q7njamirxjx-eigen-3.3.4/include/eigen3'
  as an include dir, which is not found.  It does neither exist as an
  absolute directory nor in
  '/nix/store/fa7sbb3fhaxwmqh1v096gcyc2rd606wc-pcl_conversions-1.6.2//nix/store/m8fkjfznkx88qmn6mi4x9q7njamirxjx-eigen-3.3.4//nix/store/m8fkjfznkx88qmn6mi4x9q7njamirxjx-eigen-3.3.4/include/eigen3'.
  Check the issue tracker
  'https://github.com/ros-perception/perception_pcl/issues' and consider
  creating a ticket if the problem has not been reported yet.
Call Stack (most recent call first):
  /nix/store/m71xl0jjv1kax8zpk4fr6qsj89f5kk2d-catkin-0.7.11/share/catkin/cmake/catkinConfig.cmake:76 (find_package)
  CMakeLists.txt:37 (find_package)


-- Configuring incomplete, errors occurred!
See also "/tmp/nix-build-pcl_ros-1.6.2.drv-0/source/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/nix-build-pcl_ros-1.6.2.drv-0/source/build/CMakeFiles/CMakeError.log".
builder for '/nix/store/ishzgx575g1iyxmw44626lmy48nvz73b-pcl_ros-1.6.2.drv' failed with exit code 1
error: build of '/nix/store/ishzgx575g1iyxmw44626lmy48nvz73b-pcl_ros-1.6.2.drv' failed

The weird thing is when it looks for Project 'pcl_conversions' specifies '/nix/store/m8fkjfznkx88qmn6mi4x9q7njamirxjx-eigen-3.3.4//nix/store/m8fkjfznkx88qmn6mi4x9q7njamirxjx-eigen-3.3.4/include/eigen3' as an include dir.
There seems to be a problem separating the paths: it is missing an : between ...njamirxjx-eigen-3.3.4/ and /nix/store....

I'm a bit lost as to where I should look : where is the mistake likely to be made?
Also why does pcl_conversions compile and work without problem in nix-shell, but fails when it's a dependency of pcl_ros?

[akru]

Hi, @nschoe! Glad to see your improvement!

Yes, I catch this error some month ago too. In my opinion it possible to fix via patching CMake scripts provided by pcl_conversions package.

[nschoe]

Thanks for your fast response time, @akru :)
(Do I take it that you'd be interested in a merge request once I've solved this issue and cleaned my files a little?)

Yes, I catch this error some month ago too. In my opinion it possible to fix via patching CMake scripts provided by pcl_conversions package.

Can you tell me a bit more ? (Sorry I'm quite new to NixOS, I try to learn more, but everything takes forever to me ^^).
I'll try to look up the CMake-related files inpcl_conversions but any fingers pointing in the right direction would be greatly appreciated !

(P.S. how did you dirty-fix it yourself ?)

EDIT: are you sure it's in the files in pcl_conversions that must be patched? Because pcl_conversions builds fine. It's pcl_ros that fails to build (but it somehow references pcl_conversions ; I'm quite confused).

[akru]

Of course we are interested in growing ROS package base in airapkgs repository. Building open community of Robotics engineers and entusiasts is our main aim. I'll glad to see your PR at https://github.com/airalab/airapkgs.

I'll try to reproduce this error at weekends and write results here.

[nschoe]

Okay no problem : I'll submit as soon as the error is fixed and I cleaned my files.
(We should consider making a PR to the official NixOS's nixpkgs also, so that lots of nix users can package ROS packages :))

Okay thanks for taking a look. I'll give it a small break because I've been on it for the whole day, then I'll try again.

Hope to solve this soon.

[akru]

Hi @nschoe, may be using catkin_make instead of cmake at #71 fix your issue, could you try it?

[nschoe]

Hi @akru sorry for the delay, some unexpected events synchronized.
Thanks for taking the time to look into this. I have looked at #71, but this is now closed. I'm not sure: did you solve it?

I git pulled right now, and some some changes about using catkin instead of cmake, but now I have a lot of fails.
tf2_eigen fails because of the sha256 hash not being correct, so I have updated it, now I have several other fails, I'm trying to fix them and see if pcl_ros will build.

On your side, did you manage to build pcl_ros yourself?

EDIT: the fails I have all seem to be sha256 mismatch. Any particular reason?
EDIT2: there are a lot of them, at the moment, they seem to be related to the tf* packages

[nschoe]

All right, so I have fixed all the sha256 mismatch and could retry building pcl_ros, but unfortunately, I have the same issue (it's eigen 3.3.5 now, but the error is the same):

-- Found PythonInterp: /nix/store/bwfygfcdvis9wd1c1v51xwnwhw1hx0a0-python3-3.6.6/bin/python (found version "3.6.6")
-- Using PYTHON_EXECUTABLE: /nix/store/bwfygfcdvis9wd1c1v51xwnwhw1hx0a0-python3-3.6.6/bin/python
-- Using default Python package layout
-- Found PY_em: /nix/store/kg52fzv2qm94yp5rdriq9cbc8hdg33yr-empy-3.3.2/lib/python3.6/site-packages/em.py
-- Using empy: /nix/store/kg52fzv2qm94yp5rdriq9cbc8hdg33yr-empy-3.3.2/lib/python3.6/site-packages/em.py
-- Using CATKIN_ENABLE_TESTING: OFF
-- catkin 0.7.11
CMake Error at /nix/store/7drfmax4gf4r9qnswf94wgqdy9i9nvrk-pcl_conversions-1.6.2/share/pcl_conversions/cmake/pcl_conversionsConfig.cmake:113 (message):
  Project 'pcl_conversions' specifies
  '/nix/store/hv9cr94nagryn5z68j78m6gj44jvi0m2-eigen-3.3.5//nix/store/hv9cr94nagryn5z68j78m6gj44jvi0m2-eigen-3.3.5/include/eigen3'
  as an include dir, which is not found.  It does neither exist as an
  absolute directory nor in
  '/nix/store/7drfmax4gf4r9qnswf94wgqdy9i9nvrk-pcl_conversions-1.6.2//nix/store/hv9cr94nagryn5z68j78m6gj44jvi0m2-eigen-3.3.5//nix/store/hv9cr94nagryn5z68j78m6gj44jvi0m2-eigen-3.3.5/include/eigen3'.
  Check the issue tracker
  'https://github.com/ros-perception/perception_pcl/issues' and consider
  creating a ticket if the problem has not been reported yet.
Call Stack (most recent call first):
  /nix/store/7870vlzkdmqhgf7cll4jmlrbp0644f6b-catkin-0.7.11/share/catkin/cmake/catkinConfig.cmake:76 (find_package)
  CMakeLists.txt:37 (find_package)


-- Configuring incomplete, errors occurred!
See also "/tmp/nix-build-pcl_ros-1.6.2.drv-0/source/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/nix-build-pcl_ros-1.6.2.drv-0/source/build/CMakeFiles/CMakeError.log".
builder for '/nix/store/frxm488ybn64ff0kgqrqimv6zfjz991j-pcl_ros-1.6.2.drv' failed with exit code 1
error: build of '/nix/store/frxm488ybn64ff0kgqrqimv6zfjz991j-pcl_ros-1.6.2.drv' failed

[akru]

I try to build your branch airapkgs_PCL_ROS merged with airapkgs upstream.

diff --git a/pkgs/development/ros-modules/pcl_conversions/default.nix b/pkgs/development/ros-modules/pcl_conversions/default.nix
index de22e71a1db..f985baf3e1a 100644
--- a/pkgs/development/ros-modules/pcl_conversions/default.nix
+++ b/pkgs/development/ros-modules/pcl_conversions/default.nix
@@ -1,12 +1,11 @@
 { stdenv
 , mkRosPackage
 , fetchFromGitHub
-, catkin
+, eigen
+, boost
+, pcl
 , cmake_modules
-, eigen3_3
-, boost167
 , pcl_msgs
-, pcl
 , dynamic_reconfigure
 , message_filters
 , nodelet
@@ -28,7 +27,12 @@ in mkRosPackage {
     sha256 = "0wl79qlj1j2a79w3ppvxwb4ifigmrycq82zrzrdzkajcjb4i568z";
   };
 
-  propagatedBuildInputs = [ catkin cmake_modules eigen3_3 boost167 pcl_msgs pcl dynamic_reconfigure message_filters nodelet nodelet_topic_tools ];
+  postPatch = ''
+    sed -i '/find_package(Eigen3 REQUIRED)/d' CMakeLists.txt
+  '';
+
+  buildInputs = [ eigen boost pcl ];
+  propagatedBuildInputs = [ cmake_modules pcl_msgs dynamic_reconfigure message_filters nodelet nodelet_topic_tools ];
 
   meta = with stdenv.lib; {
     description = "PCL (Point Cloud Library) ROS interface stack.";

This patch works well.

[nschoe]

Thanks @akru for the patch, I applied it and tried to rebuild pcl_ros and indeed: it succeeded!
So, nice catch and thanks for the patch.

Could you explain briefly why / how this works?

I'm asking because, now that pcl_ros builds and no longer has this issue caused by pcl_conversions... I have the exact same issue with my shell.nix (for my real project), only this time the error is caused by pcl_ros.

The error message is

CMake Error at /nix/store/31yn97xyrjy1mb8gmayg94bm1jf0qcid-pcl_ros-1.6.2/share/pcl_ros/cmake/pcl_rosConfig.cmake:113 (message):
  Project 'pcl_ros' specifies
  '/nix/store/hv9cr94nagryn5z68j78m6gj44jvi0m2-eigen-3.3.5//nix/store/hv9cr94nagryn5z68j78m6gj44jvi0m2-eigen-3.3.5/include/eigen3'
  as an include dir, which is not found.  It does neither exist as an
  absolute directory nor in
  '/nix/store/31yn97xyrjy1mb8gmayg94bm1jf0qcid-pcl_ros-1.6.2//nix/store/hv9cr94nagryn5z68j78m6gj44jvi0m2-eigen-3.3.5//nix/store/hv9cr94nagryn5z68j78m6gj44jvi0m2-eigen-3.3.5/include/eigen3'.
  Check the issue tracker
  'https://github.com/ros-perception/perception_pcl/issues' and consider
  creating a ticket if the problem has not been reported yet.
Call Stack (most recent call first):
  /nix/store/a52dcjcjnfxlgv2xnnnlhcp5p3f9i9nf-catkin-0.7.14/share/catkin/cmake/catkinConfig.cmake:76 (find_package)
  alexandria/CMakeLists.txt:13 (find_package)

So it's very similar.

I have tried to "replicate" your patch adding the postPatch, and putting eigen and boost inside buildInputs rather than propagatedBuildInputs inside pcl_ros/default.nix, but I still have the error message.

So if you could give me a hint why it worked in your case, I'd be glad :)

EDIT: how come you delete find_package(Eigen3 REQUIRED) from the CMakeLists.txt by the way? How will pcl_conversions work without Eigen?

EDIT2: let me rephrase: right now, entering my nix-shell with nix-shell on my shell.nix, which has pcl_ros inside the buildInput works: it compiles and I'm in. This part work.
What fails, though is when I run catkin_make from my catkin workspace (the one for my project). When I do this, it displays the error message above.
From your earlier comments, I thought this had to do with catkin. So what I tried is manually create a build/ directory, cd inside and manually call cmake ../src/alexandria.
But I get the same error: Project 'pcl_ros' specifies '/nix/store/hv9cr94nagryn5z68j78m6gj44jvi0m2-eigen-3.3.5//nix/store/hv9cr94nagryn5z68j78m6gj44jvi0m2-eigen-3.3.5/include/eigen3' as an include dir, which is not found..

Unfortunately I'm lacking the intuition of where is is that pcl_ros specifies this weird directory as an include dir. I feel like that's the key. Do you happen to have ideas on this? How did your earlier patch work?

Thanks again for your time :)
I try to update as as I try thing ^^

[akru]

Hi @nschoe! As you can see ROS and NixOS have a lot of catkin/cmake magic and a lot of things are not what they seem. Unfortunately in NixOS eigen package have no cmake finder file FindEigen3, for this reason pcl_conversions build fails. But eigen already in CMAKE_PREFIX_PATH because it imported by nix buildInputs for this reason patch should work.

So, in my opinion the reason of your error somewhere in cmake template files, I'll try to found them.

[nschoe]

@akru thanks for answering.
I have just made "progress" (I hope). I have two pieces of information but not sure what they mean, here are they, maybe they make sense to you:

1. I finally found something interesting: inside pcl_ros's CMakeLists.txt file I found this:

## Add include directories
include_directories(include
  ${Boost_INCLUDE_DIRS}
  ${catkin_INCLUDE_DIRS}
  ${EIGEN3_INCLUDE_DIRS}
  ${PCL_INCLUDE_DIRS}
)

And since my issues seem to be related to a weird story of include dirs, I thought about deleting the line about Eigen, so here's my pcl_ros/default.nix's postPatch :

postPatch = ''
    sed -i '/find_package(Eigen3 REQUIRED)/d' CMakeLists.txt;
    sed -i '/''${EIGEN3_INCLUDE_DIRS}/d' CMakeLists.txt
  '';

2. after doing this, I tried to enter the shell with nix-shell on my shell.nix (the one for my project), this compiled. But when I ran catkin_make from my catkin workspace, it complained that it could not find Eigen, then I remember that I had changed eigen3_3 to eigen inside my shell.nix's buildInputs. So I tried to put back eigen3_3 instead and now it doesn't complain about not finding eigen anymore.

It looks like deleting the ${EIGEN3_INCLUDE_DIRS} worked, but I'm not sure, because for a WTF reason, now my catkin_make fails because /nix/store/c4d9qgyyx6p2gydbbh5wpp4mxvwgc64n-qtbase-5.11.1-bin/lib/qt-5.11/plugins/imageformats/libqicns.so file not exiting, which is a totally unrelated problem I had a long time ago and mysteriously re-appeared (probably due to some nix-collect-garbage I ran earlier today).

In the meantime, maybe this can help you find the culprit?

EDIT: fix syntax

[nh2]

Hey, in case it helps:

I got this problem too when using nixpkgs to build https://github.com/hjwdzh/QuadriFlow.

I got:

Failed to find Eigen - Could not find file:
/nix/store/mp82a6rghs77d8qvbjc3fn0g4jcadd7s-eigen-3.3.3//nix/store/mp82a6rghs77d8qvbjc3fn0g4jcadd7s-eigen-3.3.3/include/eigen3/Eigen/src/Core/util/Macros.h

I solved it by giving cmake the flag (docs):

-DEIGEN_PREFER_EXPORTED_EIGEN_CMAKE_CONFIGURATION=FALSE