Error loading shared libraries in basilisk

[ChangqingW]

We are trying to use a python package that imports matplotlib, but matplotlib could not be imported in basilisk:

basiliskRun(env = FLAMES:::flames_env, fun = function() {
  python_path <- system.file("python", package = "FLAMES")
  count <- reticulate::import_from_path("count_gene", python_path)
})
ImportError: /lib64/libz.so.1: version `ZLIB_1.2.9' not found (required by /vast/scratch/users/wang.ch/.cache/R/basilisk/1.12.1/FLAMES/1.7.2/flames_env/lib/python3.10/site-packages/matplotlib/../../.././libpng16.so.16)

It seems like python is trying to use the system's zlib.so, despite there is a zlib.so in the envirnment:

ldd ~/.cache/R/basilisk/1.12.1/FLAMES/1.7.2/flames_env/lib/python3.10/site-packages/matplotlib/../../.././libpng16.so.16 | grep libz
        libz.so.1 => /vast/scratch/users/wang.ch/.cache/R/basilisk/1.12.1/FLAMES/1.7.2/flames_env/lib/python3.10/site-packages/matplotlib/../../../././libz.so.1 (0x00007f9b253bf000)

I also tried running conda activate {basilisk env folder} directly in bash and matplotlib can be successfully imported in python.

[ChangqingW]

Adding the testload argument did not help:

> basiliskRun(env = FLAMES:::flames_env, testload = "matplotlib", fun = function() {
+   python_path <- system.file("python", package = "FLAMES")
+   count <- reticulate::import_from_path("count_gene", python_path)
+ })
Error in py_module_import(module, convert = convert) :
  ImportError: /lib64/libz.so.1: version `ZLIB_1.2.9' not found (required by /vast/scratch/users/wang.ch/.cache/R/basilisk/1.12.1/FLAMES/1.7.3/flames_env/lib/python3.10/site-packages/matplotlib/../../.././libpng16.so.16)
Run `reticulate::py_last_error()` for details.
Error in .activate_fallback(proc, testload, env = env, envpath = envpath) :
  ImportError: /lib64/libz.so.1: version `ZLIB_1.2.9' not found (required by /vast/scratch/users/wang.ch/.cache/R/basilisk/1.12.1/FLAMES/1.7.3/flames_env/lib/python3.10/site-packages/matplotlib/../../.././libpng16.so.16)
Run `reticulate::py_last_error()` for details.

[LTLA]

Never seen this before, but my guess is that this is a reticulate problem. If R is linked to a certain Zlib, and reticulate tries to load some Python libraries, then they might all be forced to use R's Zlib, regardless of what ldd says.

As you may have already figured out, we see the same problem for the libstdc++ (see #20) and it may be possible to extend that solution to this case. It might be as simple as generalizing this line:

https://github.com/LTLA/basilisk/blob/54dee79a176d9e3f72fd0f53cb1434759a2337e9/R/basiliskStart.R#L261

to catch ZLIB failures as well. Perhaps you could try it out and see whether it helps.

[ChangqingW]

Hi Aaron, thanks for the swift reply! This does seem to be a very complicated and annoying issue. I think we will specify installing matplotlib with pip for now.
grepl("^ImportError: .*\\.so.*version.*not found", msg) would seem a reasonable generalisation if I understood it correctly.

[LTLA]

I updated basilisk in BioC-devel (1.13.4) to use an adaptation of your regex above, so maybe that might improve the situation in the future.