[#27] Add support for building universal lua-https modules.

[erinmaus]

What

• Add support for building universal lua-https modules on macOS (see Add artifact for Apple Silicon #27)

How

• Since leafo's Github actions does not build universal LuaJIT modules, we have to build LuaJIT ourselves like done on Windows.

Attached is a sample LÖVE app using the produced artifact to fetch the LÖVE website and print the response to the screen. (The app extension is named weird because GitHub won't allow uploading a .love file but obviously will run fine in LÖVE for macOS).

https.love.zip

Result:

[slime73]

Awesome, thanks for doing this!

[tpimh]

The library extensions just got my attention.

1. LuaJIT is built as libluajit.so which is incorrect (correct extension for shared libraries on macOS is dylib), so it's later corrected by renaming it to libluajit.dylib.
2. lua-https is built as https.so on macOS. This seems correct, given that this is a "bundle" (a dynamically loaded module) rather than shared library.

LuaJIT happily loads https.so on macOS if it is in cpath (LuaJIT/LuaJIT#76). One possible upside of having it named https.dylib is that this name wouldn't collide with Linux library, allowing to have https.dll, https.so and https.dylib to be stored in the same directory, so LuaJIT would load the one that matches the platform that it was built for.

Can anyone comment on this?

[erinmaus]

1. I'm not sure why the libluajit output has the .so extension, we could link to liblua.dylib but it's immaterial since it has no affect on usage with LÖVE or otherwise. We just need the symbols to build lua-https.
2. This would requiring switching the target from MODULE to SHARED in the CMakeLists.txt. (For reference I have to do this if I want to use lua-https on iOS). Not sure what other ramifications it has for other platforms. This is out of my wheelhouse.

[tpimh]

What are the actual differences between the MH_DYLIB and MH_BUNDLE in the context of Lua? As I understand from the SO answer linked in my previous message, it's a common convention to use .dylib and .so extensions respectively for these file types.

I can see that you are building devi as a dynamic library rather than a bundle for macOS. Is there a specific reason for this? Should the CMake target be switched to SHARED for lua-https as well?

[slime73]

One possible upside of having it named https.dylib is that this name wouldn't collide with Linux library, allowing to have https.dll, https.so and https.dylib to be stored in the same directory, so LuaJIT would load the one that matches the platform that it was built for.

Can anyone comment on this?

Unfortunately stock Lua made a decision to only look for the .so extension for dynamic libraries on macOS in its C module loader used for require, even though the proper extension should be .dylib. LuaJIT is just following the standard set by stock Lua.

LÖVE adds its own C module loader that goes through love.filesystem, and that looks for .so for compatibility, as well as .dylib for correctness. Using .dylib for this library would still stop it from working outside of LÖVE with the default require setup though, so I'd like to avoid that.

All that being said, within a single Linux or Windows platform you can have multiple dll/.so files for different architectures (x86 versus x64 etc), so putting everything in one folder and just relying on extensions wouldn't work with that regardless.

[tpimh]

I think that Lua devs are correct in using .so extension on macOS. I read a bit about the difference between shared libraries and modules, and it seems that lua-https should be built as a module (and this is how it's done now).

Shared libraries can be linked against and then loaded at runtime. Modules can not be linked against, only loaded at runtime. So in the context of Lua, building native libraries as a modules makes more sense, they are not linked against and loaded dynamically (however nothing stops one from loading a shared library at runtime without linking against it).

In the context of macOS, shared libraries have the .dylib extension, and modules (or "bundles" in macOS terminlogy) can have any extension. Apple recommends using .bundle, but most ported software is using .so (and this is also what CMake is using by default, overridable by setting CMAKE_SHARED_MODULE_SUFFIX). If I interpret the term "any" correctly, then it can have the .dylib extension as well (other platforms are using .so for both, and it doesn't seem to be a source of confusion).

To conclude, on macOS:

1. libluajit.so is incorrect as it's a shared library; libluajit.dylib is correct.
2. https.so is correct as it's a module. If set(CMAKE_SHARED_MODULE_SUFFIX ".dylib") is added to CMakeLists.txt, then https.dylib would be built, and it would've been correct as well ("any"), but it wouldn't work with current Lua interpreters without modifying cpath.
3. If we change the library type in CMakeLists.txt from MODULE to SHARED, then libhttps.dylib would be built (unless CMAKE_SHARED_LIBRARY_PREFIX and CMAKE_SHARED_LIBRARY_SUFFIX are set to something else), and it would also be correct, but as this library is not supposed to be linked against, this is unnecessary (except for iOS case maybe?)

As always, if you feel like I made a mistake, feel free to correct me.

[erinmaus]

I can see that you are building devi as a dynamic library rather than a bundle for macOS. Is there a specific reason for this? Should the CMake target be switched to SHARED for lua-https as well?

I'm building it as a dylib so it's trivial to convert to a framework for iOS...

Re: libluajit.so, it's actually a shared library:

$ otool -hv libluajit.so
libluajit.so:
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64    ARM64        ALL  0x00       DYLIB    15       1800   NOUNDEFS DYLDLINK TWOLEVEL NO_REEXPORTED_DYLIBS MH_HAS_TLV_DESCRIPTORS

It's just built with a .so extension... for some reason. We can't do anything about that, since it's not our code.

edit: used literally instead of actually for some reason 🤦