Clang support #83
Closed
Clang support #83
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
timkpaine
added
lang: c++
timkpaine
force-pushed
the
tkp/clang
branch
2 times, most recently
from
102f847 to
6db7b18
Compare
timkpaine
removed
the
tag: wip
Not sure if there are performance reasons to favor STL vectors on gcc, but otherwise would it make sense to use boost on gcc too? Mostly to avoid the ifdefs. |
|
I don’t think universal2 wheels are particularly useful. It would be nice if we could cross-compile x86_64 wheels on arm but the docs only talk about cross-compiling for arm on intel: https://cibuildwheel.readthedocs.io/en/stable/faq/#how-to-cross-compile |
|
What’s the best way to test out building with clang. these changes seem more intrusive than need be |
We’ll likely cross-compile to arm from x86 as the x86 mac runners are more plentiful on github actions than the arm runners (at least as of right now) |
timkpaine
added
the
type: enhancement
I looked at building with clang on RHEL 7 today. It looks like all you need to do on top of the conda-forge environment created from Then build I tried first with the There are some compiler warnings and for whatever reason |
|
I just tried this and it bombed early on when building its deps ( bombed in boost build )
"/home/user/github/csp_venv/envs/csp/bin/clang++" -c -x c++ -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/user/github/csp_venv/envs/csp/include -fPIC -g -m64 -pthread -O0 -fno-inline -Wall -g --target=x86_64-pc-linux -DBOOST_ALL_NO_LIB=1 -I"../include" -I"/home/user/github/csp/_skbuild/linux-x86_64-3.11/cmake-build/vcpkg_installed/x64-linux/include" -o "/home/user/github/csp/vcpkg/buildtrees/boost-regex/x64-linux-dbg/boost/build/8e64508ecf602f44566e7329cfeb278a/wide_posix_api.o" "../build/../src/wide_posix_api.cpp"
In file included from ../build/../src/wide_posix_api.cpp:21:
In file included from ../include/boost/regex/config.hpp:76:
In file included from /home/user/github/csp/_skbuild/linux-x86_64-3.11/cmake-build/vcpkg_installed/x64-linux/include/boost/config.hpp:44:
/home/user/github/csp/_skbuild/linux-x86_64-3.11/cmake-build/vcpkg_installed/x64-linux/include/boost/config/detail/select_stdlib_config.hpp:26:14: fatal error: 'cstddef' file not found
26 | # include <cstddef>
| ^~~~~~~~~
1 error generated.
I can try building deps first with gcc then csp with clang
|
|
Hmm, I think that might be provided by |
|
I have that
***@***.*** csp]$ conda list | grep std
libstdcxx-devel_linux-64 12.3.0 h8bca6fd_105 conda-forge
|
|
Sorry, I snuck in an edit, you probably need I'm a little confused why |
|
Looks like it did install it
( its already installed ), so that’s not it
***@***.*** csp]$ mamba install libclang-cpp
…__ __ __ __
/ \ / \ / \ / \
/ \/ \/ \/ \
███████████████/ /██/ /██/ /██/ /████████████████████████
/ / \ / \ / \ / \ \____
/ / \_/ \_/ \_/ \ o \__,
/ _/ \_____/ `
|/
███╗ ███╗ █████╗ ███╗ ███╗██████╗ █████╗
████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
██╔████╔██║███████║██╔████╔██║██████╔╝███████║
██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
██║ ╚═╝ ██║██║ ██║██║ ╚═╝ ██║██████╔╝██║ ██║
╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚═════╝ ╚═╝ ╚═╝
mamba (1.4.2) supported by @QuantStack
GitHub: https://github.com/mamba-org/mamba
Twitter: https://twitter.com/QuantStack
█████████████████████████████████████████████████████████████
Looking for: ['libclang-cpp']
conda-forge/linux-64 Using cache
conda-forge/noarch Using cache
pkgs/r/noarch No change
pkgs/r/linux-64 No change
pkgs/main/noarch No change
pkgs/main/linux-64 6.7MB @ 4.4MB/s 1.6s
Pinned packages:
- python 3.11.*
Transaction
Prefix: /home/user/github/csp_venv/envs/csp
All requested packages already installed
|
|
Looks like its mixing up clang and gcc for some reason, is that expected?
***@***.*** csp]$ clang++ -v a.cpp
clang version 17.0.6 (https://github.com/conda-forge/clangdev-feedstock 58d7390157388f8f78fd73851f0ccf1ceb9501ba)
Target: x86_64-conda-linux-gnu
Thread model: posix
InstalledDir: /home/user/github/csp_venv/envs/csp/bin
Configuration file: /home/user/github/csp_venv/envs/csp/bin/x86_64-conda-linux-gnu-clang++.cfg
Found candidate GCC installation: /home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0
Selected GCC installation: /home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0
Candidate multilib: .***@***.***
Selected multilib: .***@***.***
"/home/user/github/csp_venv/envs/csp/bin/clang-17" -cc1 -triple x86_64-conda-linux-gnu -emit-obj -mrelax-all -dumpdir a- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name a.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -v -fcoverage-compilation-dir=/home/user/github/csp -resource-dir /home/user/github/csp_venv/envs/csp/lib/clang/17 -isysroot /home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot -internal-isystem /home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/include/c++/12.3.0 -internal-isystem /home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/include/c++/12.3.0/x86_64-conda-linux-gnu -internal-isystem /home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/include/c++/12.3.0/backward -internal-isystem /home/user/github/csp_venv/envs/csp/lib/clang/17/include -internal-isystem /home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/usr/local/include -internal-isystem /home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/include -internal-externc-isystem /home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/include -internal-externc-isystem /home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/usr/include -fdeprecated-macro -fdebug-compilation-dir=/home/user/github/csp -ferror-limit 19 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/a-f4a772.o -x c++ a.cpp
clang -cc1 version 17.0.6 based upon LLVM 17.0.6 default target x86_64-conda-linux-gnu
ignoring nonexistent directory "/home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/usr/local/include"
ignoring nonexistent directory "/home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/include"
#include "..." search starts here:
#include <...> search starts here:
/home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/include/c++/12.3.0
/home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/include/c++/12.3.0/x86_64-conda-linux-gnu
/home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/include/c++/12.3.0/backward
/home/user/github/csp_venv/envs/csp/lib/clang/17/include
/home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/include
/home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/usr/include
End of search list.
"/home/user/github/csp_venv/envs/csp/bin/x86_64-conda-linux-gnu-ld" --sysroot=/home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot -pie --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out /home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/usr/lib/../lib64/Scrt1.o /home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/usr/lib/../lib64/crti.o /home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/crtbeginS.o -L/home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0 -L/home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/lib/../lib64 -L/home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/usr/lib/../lib64 -L/home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/lib -L/home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/lib -L/home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/usr/lib /tmp/a-f4a772.o -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /home/user/github/csp_venv/envs/csp/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/crtendS.o /home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/usr/lib/../lib64/crtn.o
/home/user/github/csp_venv/envs/csp/bin/x86_64-conda-linux-gnu-ld: /home/user/github/csp_venv/envs/csp/x86_64-conda-linux-gnu/sysroot/usr/lib/../lib64/Scrt1.o: in function `_start':
(.text+0x20): undefined reference to `main'
|
|
Somehow when I did this earlier I ended up with clang 14, which doesn't seem to hit the issue you're hitting. You could try downgrading clangdev with When I created a new environment from scratch I ended up with clang 17 and hit the same error you did compiling Another thing we could try is a chimera linux or FreeBSD docker image or VM image - both systems are built on LLVM so we won't hit any issues from mixing LLVM and GCC. |
|
Thanks Nathan, I was able to get this to work and have cleaned up all the clang build issues.
I have the changes ready but still cant push to GH from on-prem, I’m hoping they’ll have a fix ready tomorrow
|
robambalu
added a commit
that referenced
this pull request
Mar 4, 2024
…me clang warnings Signed-off-by: Ambalu, Robert <[email protected]>
This was referenced Mar 5, 2024
Merged
warnings pointed out by clang Signed-off-by: Tim Paine <[email protected]>
robambalu
added a commit
that referenced
this pull request
Mar 15, 2024
Signed-off-by: Rob Ambalu <[email protected]>
robambalu
added a commit
that referenced
this pull request
Mar 27, 2024
* fixes #83 - code updates to support clang compilation / cleaned up some clang warnings Signed-off-by: Ambalu, Robert <[email protected]> * fixes #132 - final touches, all tests pass. Note this had to comment out the --arch options in CMakeLists.txt, not sure what the implications are Signed-off-by: Rob Ambalu <[email protected]> * fixes #132 - attempt to kick off macos build Signed-off-by: Rob Ambalu <[email protected]> * unset CC in mac cibuildwheel setup Signed-off-by: Nathan Goldbaum <[email protected]> * Add support for mac conda builds Signed-off-by: Nathan Goldbaum <[email protected]> * Update docs for mac builds Signed-off-by: Nathan Goldbaum <[email protected]> * attempt to fix macos-12 conda build Signed-off-by: Nathan Goldbaum <[email protected]> * one more try for intel mac support Signed-off-by: Nathan Goldbaum <[email protected]> * reduce length of timeout for test_threaded_run Signed-off-by: Nathan Goldbaum <[email protected]> * make this test more meaningful now that it has a shorter timeout Signed-off-by: Nathan Goldbaum <[email protected]> * CMakeLists.txt - attempt tp align RPATH on linux, drop ../lib Signed-off-by: Rob Ambalu <[email protected]> * #132 temporarily add some debug logging to threaded engine to try to figure out why it fails on GH Signed-off-by: Rob Ambalu <[email protected]> * #132 fix lint for test logging Signed-off-by: Rob Ambalu <[email protected]> * #132 enable stdout logging for tests, revert test_engine back to old timing Signed-off-by: Rob Ambalu <[email protected]> * #132 - next atempt Signed-off-by: Rob Ambalu <[email protected]> * #132 - next attempt Signed-off-by: Rob Ambalu <[email protected]> * #132 - next attempt Signed-off-by: Rob Ambalu <[email protected]> * fixes #132 - Fix race condition exposed in macos build where push events can be deferred indefinitely if timer executes before event ( wait will keep waiting until max time, even if an event is pending ) Simplify QueueWaiter ( remove it entirely ) and move condvar/mutex/blocking logic directly into SRMWLockFreeQueue so that we can check on m_head directly. Revert all intermediate changes that were committed for testing / tracking down this issue Signed-off-by: Rob Ambalu <[email protected]> * fixes #132 - undo intrusive LFQueue changes ( for now ), simplify the fix and attack the queue issue properly another day. fix cpp time test Signed-off-by: Rob Ambalu <[email protected]> * fixes #132 - fix comment typo Signed-off-by: Rob Ambalu <[email protected]> * clean up ci/cd code and ensure mac tests run Signed-off-by: Tim Paine <[email protected]> --------- Signed-off-by: Ambalu, Robert <[email protected]> Signed-off-by: Rob Ambalu <[email protected]> Signed-off-by: Rob Ambalu <[email protected]> Signed-off-by: Nathan Goldbaum <[email protected]> Signed-off-by: Tim Paine <[email protected]> Co-authored-by: Nathan Goldbaum <[email protected]> Co-authored-by: Tim Paine <[email protected]> Co-authored-by: Tim Paine <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds clang support. Right now it does so by:
template<>blocks that g++ is comfortable elidingconstexprtoTypeTraitsthat g++ is comfortable elidingvector<bool>specialization onclangcompiler to rely onboost::containers::vectorwhich avoids bitwise specialization that makes life annoyingg++does not noticeoverridesThe mac builds have both been ported to
clang++and the builds are passing (tests passing locally).Unblocks #10 and #32
Full Build: https://github.com/Point72/csp/actions/runs/7938321028
TODO
boolfields from structs astsuniversal2binariesg++