liboptparse.so has mysterious unresolved jansson symbols on focal

[garlick]

Problem: as noted in flux-framework/flux-pmix#102, flux-pmix stopped working in the focal image. The problem is that a test executable cannot be linked:

 garlick@system76-pc:/usr/src/t/src$ make V=1 barrier
/bin/bash ../../libtool  --tag=CC   --mode=link gcc -Wall -Werror -Wno-strict-aliasing -Wno-error=deprecated-declarations  -g -O2   -o barrier barrier.o ./libtestutil.la ../../src/common/libutil/libutil.la -lflux-optparse -lflux-idset -Wl,-rpath -Wl,/usr/lib -Wl,--enable-new-dtags -lmpi -Wl,-rpath -Wl,/usr/lib -Wl,--enable-new-dtags -lpmix 
libtool: link: gcc -Wall -Werror -Wno-strict-aliasing -Wno-error=deprecated-declarations -g -O2 -o barrier barrier.o -Wl,-rpath -Wl,/usr/lib -Wl,--enable-new-dtags -Wl,-rpath -Wl,/usr/lib -Wl,--enable-new-dtags  ./.libs/libtestutil.a ../../src/common/libutil/.libs/libutil.a /usr/lib/libflux-optparse.so /usr/lib/libflux-idset.so /usr/lib/libmpi.so /usr/lib/libpmix.so -pthread
/usr/bin/ld: /usr/lib/libflux-optparse.so: undefined reference to `json_object_iter_value'
/usr/bin/ld: /usr/lib/libflux-optparse.so: undefined reference to `json_object_key_to_iter'
/usr/bin/ld: /usr/lib/libflux-optparse.so: undefined reference to `json_object_set_new_nocheck'
/usr/bin/ld: /usr/lib/libflux-optparse.so: undefined reference to `json_object_iter_next'
/usr/bin/ld: /usr/lib/libflux-optparse.so: undefined reference to `json_object_iter_key'
/usr/bin/ld: /usr/lib/libflux-optparse.so: undefined reference to `json_object_get'
/usr/bin/ld: /usr/lib/libflux-optparse.so: undefined reference to `json_object_iter'
collect2: error: ld returned 1 exit status
make: *** [Makefile:516: barrier] Error 1

Apparently the installed libflux-optparse.so (0.69.0-55-g7cd67057e) has unresolved jansson symbols in it.

This is not a problem in the el8 and jammy builders.

The last action in the flux-pmix repo was a succeeding one 10 months ago, so that does not help narrow things down.
I thought the the pmix check in the flux-core CI had been failing for some time, but looking through past actions, that doesn't seem to be the case! Also, that check is done in ubuntu-latest.

A concern is that this problem emerged after the linkage changes for macos portability last week. It will require more investigation.

[garlick]

Oh I suppose it goes without saying that libflux-optparse.so does not use jansson, but I should have metioned it. It's a mystery how that's leaked in, and only on focal apparently.

[garlick]

The installed dso had no symbols in the fluxrm image, but if I use the flux-core focal image to build flux, I can see the following unresolved symbols:

 garlick@system76-pc:/usr/src/src/common/.libs$ nm libflux-optparse.so|grep 'U '
                 U __assert_fail@@GLIBC_2.2.5
                 U __ctype_b_loc@@GLIBC_2.3
                 U __errno_location@@GLIBC_2.2.5
                 U __fprintf_chk@@GLIBC_2.3.4
                 U __snprintf_chk@@GLIBC_2.3.4
                 U __sprintf_chk@@GLIBC_2.3.4
                 U __stack_chk_fail@@GLIBC_2.4
                 U __tls_get_addr@@GLIBC_2.3
                 U __vfprintf_chk@@GLIBC_2.3.4
                 U __vsnprintf_chk@@GLIBC_2.3.4
                 U abort@@GLIBC_2.2.5
                 U argz_create_sep@@GLIBC_2.2.5
                 U calloc@@GLIBC_2.2.5
                 U exit@@GLIBC_2.2.5
                 U fcntl@@GLIBC_2.2.5
                 U fflush@@GLIBC_2.2.5
                 U fputc@@GLIBC_2.2.5
                 U free@@GLIBC_2.2.5
                 U fwrite@@GLIBC_2.2.5
                 U getenv@@GLIBC_2.2.5
                 U json_object_get
                 U json_object_iter
                 U json_object_iter_key
                 U json_object_iter_next
                 U json_object_iter_value
                 U json_object_key_to_iter
                 U json_object_set_new_nocheck
                 U malloc@@GLIBC_2.2.5
                 U memcpy@@GLIBC_2.14
                 U memmove@@GLIBC_2.2.5
                 U read@@GLIBC_2.2.5
                 U realloc@@GLIBC_2.2.5
                 U snprintf@@GLIBC_2.2.5
                 U stderr@@GLIBC_2.2.5
                 U strchr@@GLIBC_2.2.5
                 U strcmp@@GLIBC_2.2.5
                 U strdup@@GLIBC_2.2.5
                 U strerror@@GLIBC_2.2.5
                 U strlen@@GLIBC_2.2.5
                 U strncmp@@GLIBC_2.2.5
                 U strncpy@@GLIBC_2.2.5
                 U strtod@@GLIBC_2.2.5
                 U strtol@@GLIBC_2.2.5
                 U strtold@@GLIBC_2.2.5
                 U strtoull@@GLIBC_2.2.5
                 U write@@GLIBC_2.2.5

So this really is a problem with linkage on focal rather than a problem specifically with the image.

[garlick]

Ah json_object_update_recursive() is compiled into libmissing on focal since it only has jansson 2.12.

And apparently I blew it on the $ld_gc_sections make variable - it was only supposed to be unset on macos but the logic to set it is missing. When I add that back in for focal, those undefined symbols go away. 🤦

[grondo]

Good catch!