Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Link LLVM-based tools dynamically to LLVM #397

Closed
wants to merge 3 commits into from
Closed

Link LLVM-based tools dynamically to LLVM #397

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1d07052
Link LLVM-based tools dynamically to LLVM
alexcrichton Mar 3, 2024
e90dac7
debug
alexcrichton Mar 9, 2024
31cd0a4
more
alexcrichton Mar 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ on:
push:
branches:
- main
- debug-a-bit
pull_request:
workflow_dispatch:

Expand All @@ -22,6 +23,7 @@ jobs:
steps:
- uses: actions/cache@v4
with:
save-always: true
path: ~/.cache/ccache
# Bump the prefix number to evict all previous caches and
# enforce a clean build, in the unlikely case that some
Expand All @@ -33,6 +35,7 @@ jobs:
if: matrix.os == 'ubuntu-latest'
- uses: actions/cache@v4
with:
save-always: true
path: ~/Library/Caches/ccache
key: 0-cache-macos-latest-${{ github.run_id }}
restore-keys: |
Expand Down Expand Up @@ -91,6 +94,7 @@ jobs:
steps:
- uses: actions/cache@v4
with:
save-always: true
path: ~/AppData/Local/ccache
key: 0-${{ format( 'cache-windows-latest-{0}', matrix.arch) }}-${{ github.run_id }}
restore-keys: |
Expand All @@ -114,13 +118,28 @@ jobs:
- name: Build
shell: msys2 {0}
run: |
make package LLVM_CMAKE_FLAGS=-DLLVM_CCACHE_BUILD=ON
NINJA_FLAGS=-v make package LLVM_CMAKE_FLAGS=-DLLVM_CCACHE_BUILD=ON
make check
- name: Does it work sans msys2?
run: |
C:\wasi-sdk\bin\clang.exe --version
C:\wasi-sdk\bin\llvm-ar.exe --version
C:\wasi-sdk\bin\wasm-ld.exe --version
- if: always()
shell: msys2 {0}
run: find build/compiler-rt
- if: always()
shell: msys2 {0}
run: find C:/wasi-sdk
- if: always()
shell: msys2 {0}
run: ls -l C:/wasi-sdk
- if: always()
shell: msys2 {0}
run: find /c/wasi-sdk
- if: always()
shell: msys2 {0}
run: ls -l /c/wasi-sdk
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
Expand All @@ -134,6 +153,7 @@ jobs:
steps:
- uses: actions/cache@v4
with:
save-always: true
path: ~/.ccache
key: 0-cache-ubuntu-bionic-${{ github.run_id }}
restore-keys: |
Expand Down
4 changes: 4 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ build/llvm.BUILT:
-DCLANG_LINKS_TO_CREATE="$(call join-with,;,$(CLANG_LINKS_TO_CREATE))" \
-DCMAKE_BUILD_TYPE=MinSizeRel \
-DLLVM_ENABLE_TERMINFO=OFF \
-DLLVM_LINK_LLVM_DYLIB=ON \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me if it works. I assume the rpath stuff in the binaries is setup so that they can find the .so files relative to themselves?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe so yeah, bin/clang has:

$ readelf -a ./bin/clang | rg -i runpath
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../lib]

I wasn't certain for other platforms so I was relying on CI here during make check to fail if there was an issue

-DLLVM_VERSION_SUFFIX=-wasi-sdk \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean that bin/clang-18 is now bin/clang-18-wasi-sdk? Or does it just apply the .so file?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it's just the *.so files, from the installation folder I get:

 $ fd -- -wasi-sdk
lib/libLLVM-18-wasi-sdk.so
lib/libLLVM.so.18.1-wasi-sdk
lib/libclang-cpp.so.18.1-wasi-sdk

can confirm that bin/clang-18 is still named just that, no wasi-sdk

-DLLVM_ENABLE_ZLIB=OFF \
-DLLVM_ENABLE_ZSTD=OFF \
-DLLVM_STATIC_LINK_CXX_STDLIB=ON \
Expand Down Expand Up @@ -118,6 +120,8 @@ build/llvm.BUILT:
install-objdump \
install-objcopy \
install-c++filt \
install-LLVM \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this install anything more than libLLVM ? e.g. does it also install headers?

Is this technically needed? Does install-clang, for example, not automatically also install libLLVM.so?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this I found that as-is without adding this libLLVM wasn't installed, so the lib dir was empty, meaning that all the binaries failed with:

$ ./bin/clang --version
./bin/clang: error while loading shared libraries: libclang-cpp.so.18.1-wasi-sdk: cannot open shared object file: No such file or directory

to see the missing deps I ran:

$ ldd bin/clang
        linux-vdso.so.1 (0x00007ffd18265000)
        libclang-cpp.so.18.1-wasi-sdk => not found
        libLLVM.so.18.1-wasi-sdk => not found
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0511b62000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f0511dc4000)

I wasn't sure how best to install these so I poked around the tab-completions of ninja install* and found install-{LLVM,clang-cpp} did the trick.

I ran ninja install-LLVM manually and it printed:

$ DESTDIR=$HOME/code/wasi-sdk/build ninja install-LLVM
[0/1] cd /home/alex/code/wasi-sdk/build/llvm/tools/llv...home/alex/code/wasi-sdk/build/llvm/cmake_install.cmake
-- Install configuration: "MinSizeRel"
-- Installing: /home/alex/code/wasi-sdk/build/opt/wasi-sdk/lib/libLLVM.so.18.1-wasi-sdk
-- Set non-toolchain portion of runtime path of "/home/alex/code/wasi-sdk/build/opt/wasi-sdk/lib/libLLVM.so.18.1-wasi-sdk" to "$ORIGIN/../lib"
-- Installing: /home/alex/code/wasi-sdk/build/opt/wasi-sdk/lib/libLLVM.so
-- Creating libLLVM-18-wasi-sdk.so

and for clang-cpp:

 $ DESTDIR=$HOME/code/wasi-sdk/build ninja install-clang-cpp
[0/1] cd /home/alex/code/wasi-sdk/build/llvm/tools/clang/tools/clang-shlib && /home/alex/root/bin/cmake -DCMAKE_INSTALL_COMPONENT="clang-cpp" -P /home/alex/code/wasi-sdk/build/llvm/cmake_install.cmake
-- Install configuration: "MinSizeRel"
-- Installing: /home/alex/code/wasi-sdk/build/opt/wasi-sdk/lib/libclang-cpp.so.18.1-wasi-sdk
-- Set non-toolchain portion of runtime path of "/home/alex/code/wasi-sdk/build/opt/wasi-sdk/lib/libclang-cpp.so.18.1-wasi-sdk" to "$ORIGIN/../lib"
-- Installing: /home/alex/code/wasi-sdk/build/opt/wasi-sdk/lib/libclang-cpp.so

Sorry if this is a bit verbose, but I don't really have much of an idea of what I'm doing, I'm sort of just hitting the first thing that works and putting it here. I am by no means an expert in CMake, LLVM's build system, or how all this is supposed to interact.

My buest guest to your questions is:

Does this install anything more than libLLVM ? e.g. does it also install headers?

Judging from the above I think not, but I wouldn't say that with certainty.

Is this technically needed? Does install-clang, for example, not automatically also install libLLVM.so?

Apparently not, because if I left this out libLLVM.so wasn't installed.

I'm not sure if there's some other umbrella install-* target that does what we want though.

install-clang-cpp \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this line install? cpp I assume refers to the pre-processor rather than clang++?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I commented a bit more above on this, but I thought clang-cpp was the preprocessor as well. Apparently though this deals with libclang-cpp.so and may just be slightly-oddly-named.

llvm-config
touch build/llvm.BUILT

Expand Down
Loading