Packaging libraries with C++ modules

"C++20 Modules: The Packaging and Binary Redistribution Story" (Cppcon 2023)

• Abstract
• PDF slides
• YouTube (to be published)

Experiments

CMake with add_subdirectory() or FetchContent

This is the "no packaging" solution - where we can use modules if all dependencies that also use modules are built within the project. This should work with:

• CMake 3.28, provided there are no conflicts or side effects when including an external project:
  • add_subdirectory(), e.g. if the dependency is vendored-in, or added as a git submodule.
  • the FetchContent module - when the external project is downloaded at configure time.
• Visual Studio 2022 17.4. Visual C++ projects will do dependency scanning by default. If you have multiple C++ projects in a solution, you may have to ensure that Project References are correctly set up.

See subfolder experiments/01-no-packaging for an example. Tested with:

• Linux: Clang 16, CMake 3.28rc1, Ninja 1.11.1
• Windows: MSVC 19.34, CMake 3.28rc1, Visual Studio 2022 or Ninja 1.11.1

cd experiments/01-no-packaging
mkdir build
cd build
cmake .. -GNinja
cmake --build . 
./hello_world

Packaging BMIs

This experiment packages the Binary Module Interface (BMI) artifacts alongside the library binaries, and uses compiler flags for consumers (importers) to use:

• -fmodule-file=fmt=/path/to/fmt.pcm (Clang)
• /reference fmt=fmt.cc.ifc", f"/ifcSearchDir C:/path/to/fmt/bmi (msvc)

This example is only supported by Clang or msvc. Because this uses existing abstractions that are agnostic of C++ modules, this works with older versions of CMake. With Conan, it should also work with any other existing build system integration.

Note: packaging BMIs is discouraged by compiler vendors. A BMI will only be compatible on the importer side if it uses the same compiler, compiler version and in most cases, compiler options, as when it was first produced. This is only provided for illustration purposes.

Conan 2.0 is used for this example:

cd experiments/02-bmi-packaging
conan create fmt-recipe
conan install .
cmake --preset conan-release
cmake --build --preset conan-release
./build/Release/hello_world

The contents of the fmt package that are visible to the consumer are simply the BMI and the library file. Includes are not needed (because consumers are using import, and they are not producing a BMI from sources at all:

|-- bmi
|   `-- fmt.pcm
`-- lib
    `-- libfmt.a

Imported CMake 3.28 targets

CMake 3.28 adds support for C++ modules, even for IMPORTED targets. With this approach, the module interfaces (C++ source files where the module interfaces are exported) are shipped alongside the built library artifacts (.a, .so, ...).

This requires the build system to have support for:

• Dependency scanning of sources external to the project
• Generating the BMIs for the module interfaces, if they are required (via the import keyword) by any sources in the local project.

In this instance, the BMIs are generated on the consumer side, and thus that guarantees that they are, at least, the same compiler and compiler version as the importer. This is in line with the recommendations from compiler vendors, where BMIs are seen as a binary artifact that is not distributable, but is generated (or regenerated) by the build system on demand.

See experiments/03-imported-targets for an example, where {fmt} is packaged with Conan using CMake 3.28, and the consumer project reads the files generated by CMake, which now have the relevant properties to enable the behavior described above.

Note that this requires Clang 17 or MSVC 19.34, and a Conan profile with compiler.cppstd=20 or higher, as well as Ninja.

cd experiments/03-imported-targets
conan create fmt-recipe
conan install . 
cmake --preset conan-release
cmake --build --preset conan-release
./build/Release/hello_world

The package has these contents:

|-- include
|   `-- fmt
|       |-- args.h
|       |-- chrono.h
|       |-- color.h
|       |-- compile.h
|       |-- core.h
|       |-- format-inl.h
|       |-- format.h
|       |-- os.h
|       |-- ostream.h
|       |-- printf.h
|       |-- ranges.h
|       |-- std.h
|       `-- xchar.h
`-- lib
    |-- cmake
    |   `-- fmt
    |       |-- fmt-config-version.cmake
    |       |-- fmt-config.cmake
    |       |-- fmt-targets-release.cmake
    |       `-- fmt-targets.cmake
    |-- cxx
    |   `-- miu
    |       `-- src
    |           |-- fmt.cc
    |           |-- format.cc
    |           `-- os.cc
    |-- libfmt.a
    `-- pkgconfig
        `-- fmt.pc

Where the fmt-targets.cmake contains the relevant information to reconstruct the BMI on the consumer side. The {fmt} headers are needed by fmt.cc (which exports the fmt named module) - but are otherwise not used by consumers that do import fmt;.

C++ libraries with module support

The following libraries have (possibly experimental) support for C++ modules with the new CMake features.

{fmt}

• With FMT_MODULE CMake option set to ON
• CMake 3.28 support pending - see fmtlib/fmt#3679
• Issues building with gcc, see GCC Bugzilla – Bug 111785

mp-units

• Pending mpusz/mp-units#350
• With MP_UNITS_BUILD_MODULES CMake option set to ON
• Tested with Clang 17

Vulkan-Hpp

• With VULKAN_HPP_ENABLE_EXPERIMENTAL_CPP20_MODULES CMake option set to ON
• Target is called VulkanHppModule

Relevant bug tracking

• Internal compiler error with gcc when compiling fmt as module, see GCC Bugzilla – Bug 111785
• CMake: error with imported targets when generator is Visual Studio (see CMake issue 25328)
• GCC module support with -fvisibility=hidden (see GCC Bugzilla – Bug 105397 )
• clang-scan-deps may error finding system headers if the compiler is a symlink
  • See llvm/llvm-project#61006
  • See Clang documentation about this issue
  • See CMake issue 25180