Upgrade to jazzy

[jimmy-mcelwain]

Do not merge. This branch should stay separate. The PR is only here to review/discuss.

Required for Yaskawa-Global/motoros2#337

I can squash the commits if that would be cleaner.

Notable changes (not just update upstream/cherry pick) include micro-ROS/rcl@jazzy...Yaskawa-Global:micro_ros_rcl:jazzy and ament/ament_cmake@jazzy...Yaskawa-Global:ament_cmake:jazzy.

Please review and comment on those comparisons here (making PRs in those repos doesn't make sense)

ros2_tracetools is also notable because it is the only non-fork that doesn't match up with the jazzy release targets. I had to target a more up-to-date version to avoid breaking the build. The targeted commit is the one that fixes the problem that broke the build.

Additionally, I chose a specific commit for motoros2_interfaces because the most recent release 0.1.3 is not compatible with the most up-to-date commits of MotoROS2.

I did not update jazzy to match rosidl's one_func_per_file branch. I am just pulling directly from upstream. If we need the one_func_per_file style, I can do that.

I also got rid of the rcpputils package. I know that this should probably be a separate PR, and I can do that if needed. But it was just easier because it isn't required and removing it sped up the build (which I had to do many times).

[jimmy-mcelwain]

Required to build libmicroros for Yaskawa-Global/motoros2#337

[jimmy-mcelwain]

I changed the motoros2_interfaces target to 0.1.4 but forgot to add the tag to the actual commit. Now that there's another PR in place I'll wait until Yaskawa-Global/motoros2_interfaces#29 gets merged. But if you want to build libmicroros for the time being reset to the previous commit or manually change the hash to what it was before (c0d62db96b1f54b023fb1bd17796c6d40e1bcfd3)

[ted-miller]

I'm able to successfully build, install, and run with these package versions. It passes all of my testing.

The only issue left is to resolve the merge conflict. Do we want the one_func_per_file version of rosidl to reduce the binary size?

Personally, I'm ok with the larger binary. It loads into the YRC just fine, even alongside my welding application.

[gavanderhoorn]

@ted-miller wrote in https://github.com/Yaskawa-Global/micro_ros_motoplus_buildscripts/pull/162#discussion_r1884468752:

My final binary is 2402 KB

that's more than 240% of the size of the current YRC1 Humble binary.

I would indeed recommend we get the rosidl patch ported. That helps reduce it.

[gavanderhoorn]

@jimmy-mcelwain: could you squash some of the fixups and rebase this?

[jimmy-mcelwain]

@jimmy-mcelwain: could you squash some of the fixups and rebase this?

I cleaned up the history on my fork. I didn't add any "new" commits, but I did a lot of squashing. I have not done a lot of manipulation of git history so I don't want to push it here until somebody gives the green light.

[jimmy-mcelwain]

I would indeed recommend we get the rosidl patch ported. That helps reduce it.

I will work on this

[gavanderhoorn]

@jimmy-mcelwain: if you really want to work on it (I can also do it): an actual rebase is probably not productive. There will be many conflicts, as quite a bit has changed in rosidl between Humble and Jazzy.

The 'only' thing that changed in that patch is all/most functions got their own .c file.

So instead of one single .c with all definitions, it's just one .h with the declarations and then N source files, one for each function (IIRC).

Of course you have to take symbol visibility into account, but that's basically it.

Lots of duplication, but that's not important.

[jimmy-mcelwain]

I have been working on updating rosidl. I have a branch here. I originally had a binary size of 2385 KB. With my first of 2 commits in the branch I sent, I just created parity with Humble's one_func_per_file branch. I didn't rebase, but I only added the new files and split the ones that were already in the humble branch. I got the binary down to 2297 KB, so a reduction of 88 KB, but that's nowhere near the 400 KB mentioned here https://github.com/Yaskawa-Global/micro_ros_motoplus_buildscripts/pull/162#discussion_r1860543325.

rosidl has added some new files where the first one_func_per_file branch made changes, so I thought that I would apply the one function per file (or something closer to it) rule to the new files too, but now after splitting some of the new files I get 2380 KB, so the file size went up between the two patches.

I haven't done thorough testing, but I have MR2 running using the libmicroros built with that branch that I linked.

I don't know how you are getting closer to 1MB @gavanderhoorn. I thought that maybe the flags that you mentioned in the thread I linked above. But I'm not sure.

[jimmy-mcelwain]

Since making these changes, I've also noticed that my build times (for target_ws/libmicroros) have increased on my machine (~15 min -> ~25 min). I'm guessing that the increased number of files is the reason.

[gavanderhoorn]

I don't know how you are getting closer to 1MB @gavanderhoorn. I thought that maybe the flags that you mentioned in the thread I linked above. But I'm not sure.

I'll take a look.

re: flags: yes, there's a good chance. You can see (most of) the flags in the build_info.yaml in the releases we ship.

Since making these changes, I've also noticed that my build times (for target_ws/libmicroros) have increased on my machine (~15 min -> ~25 min). I'm guessing that the increased number of files is the reason.

25 minutes? On my machine it takes 2 mins 35 seconds to complete a build.

I find that on Windows processes like these are often bottlenecked on IO. NTFS is a slow FS, more so with (multiple) virus and threat scanners. If possible, try to configure/add some exceptions. Or wait half an hour of course.

[ted-miller]

25 minutes? On my machine it takes 2 mins 35 seconds to complete a build.

Your IT dept buys better equipment than ours :)

I find that on Windows processes like these are often bottlenecked on IO. NTFS is a slow FS, more so with (multiple) virus and threat scanners. If possible, try to configure/add some exceptions. Or wait half an hour of course.

Unfortunately, we do have multiple scanners that we cannot add exceptions to. My one scanner continuously takes 40-50% of my CPU at all times. It destroys my battery-life.

Having said that, 25 minutes is excessive even on my machine. We'll see if we can improve that.

[jimmy-mcelwain]

I have cleaned up the history and clarified the comment about the commit on rolling that I referenced for ros2_tracing.

[ted-miller]

@gavanderhoorn Have you had a chance to look at the binary size. Yesterday, I tried a build and the final binary was 2397 KB. I explicitly verified that I am using Jimmy's one-func-per-file branch of rosidl.

I also did a build using the rolling branch of rosidl. The libmicros build was significantly smaller using this branch (like 4 MB difference), but the final MR2.out was insignificantly different (something like 24xx KB).

[gavanderhoorn]

@gavanderhoorn Have you had a chance to look at the binary size.

yes, I have.

Not 100% sure yet, but I think I've got it down to these differences:

1. you're most likely building without any optimisation enabled, but debug enabled. That significantly increases the sizes of objects and binaries
2. the new packages in Jazzy (mostly ROS IDL related) have the same problem as rosidl that I split: their objects contain multiple functions, and they get linked-in wholesale
3. (minor): data and function sections are generated in some packages
4. the new packages in Jazzy (again, mostly ROS IDL related) introduce functions with really long symbol names. These contribute to the total size of binaries as the strtab just gets a whole lot bigger

Yesterday, I tried a build and the final binary was 2397 KB. I explicitly verified that I am using Jimmy's one-func-per-file branch of rosidl.

yes, that's about the size I also see when I don't do anything special.

I also did a build using the rolling branch of rosidl. The libmicros build was significantly smaller using this branch (like 4 MB difference), but the final MR2.out was insignificantly different (something like 24xx KB).

You can't really mix rolling branches with Jazzy, but it doesn't matter in this case.

The reason you're seeing a smaller libmicroros is because vanilla upstream doesn't have the overhead of splitting each RMW related function into its own file (which has to duplicate certain things in order to still be able to compile). That overhead gets added for each function, for each message/interface.

So libmicroros is smaller, but the resulting binary is bigger.

---

Having written that: without some work on infrastructure we won't be able to address this. We probably don't need to block releasing a Jazzy version of MotoROS2; I'll work on reducing binary size later, and I can build the binary as well.

For reference, with some minor tweaking (so nothing too invasive) the final binary for YRC1 is about 1.4 MB. Still an almost 50% increase, but better than the 2.4MB (+ in some cases) you and @jimmy-mcelwain reported earlier.

[gavanderhoorn]

My suggestion for how to move forward here:

• do a final rebase on-top of current humble
• squash the two commits bumping the version of motoros2_interfaces (we really need just the
  one)
• (optional): squash 7fbd46e into e11f261
• I'll:
  • create an initial set of prerelease-... tags
  • build an initial pre-release version of libmicroros for Jazzy
  • add a commit to Jazzy Jalisco motoros2#337 to have it build against that pre-release version
  • once that all builds, create actual release tags and update Jazzy Jalisco motoros2#337 to use those instead of the pre-release
• make jazzy the default branch here in micro_ros_motoplus

At that point I believe we can consider libmicroros targetting Jazzy as 'released' and we could close this PR.

[jimmy-mcelwain]

Rebased and squashed unnecessary commits

[ted-miller]

Release tags have been created. Thanks @gavanderhoorn for building the libs.

Closing this PR and making jazzy the default branch.