Add libclang.so to SDK

[Hollingsworth-martin-bl]

When Adding clang in generated SDK toolchain following the README guide, in the x86_64 sysroot only libclang-cpp.so is added to the lib directory.

My application wants to link against libclang.so. How do I add it this as well to the SDK?

[Hollingsworth-martin-bl]

With the original question I want to find out if I am doing something stupidly wrong when using meta-clang or if I have a misunderstanding what the difference is between libclang-cpp.so and libclang.so.

For everyone who needs a hacky workaround solution I want to share my hack. Basically I have an extra recipe which copies libclang.so from sysroot-native (i.e. x86-64) into SDK sysroot x86-64.

SUMMARY = "Hack libclang.so into x86-64 sysroot of the SDK"

LICENSE = "CLOSED"
LIC_FILES_CHKSUM = ""

SRC_URI = ""
PV = "1.0"

BBCLASSEXTEND = "native nativesdk"
DEPENDS = "clang-native"

do_install:class-nativesdk () {
    install -d ${D}${libdir}
    cp --no-preserve=ownership  --no-dereference ${RECIPE_SYSROOT_NATIVE}/usr/lib/libclang.so* ${D}${libdir}
}

FILES:${PN} += "${libdir}"

# Disable a lot of QA checks, this is indeed a dirty hack ;-)
INSANE_SKIP:${PN} = "already-stripped dev-elf pkgvarcheck file-rdeps"

Not nice, but gets the job done. A clean solution is still appreciated 😄

[kraj]

With the original question I want to find out if I am doing something stupidly wrong when using meta-clang or if I have a misunderstanding what the difference is between libclang-cpp.so and libclang.so.

For everyone who needs a hacky workaround solution I want to share my hack. Basically I have an extra recipe which copies libclang.so from sysroot-native (i.e. x86-64) into SDK sysroot x86-64.

SUMMARY = "Hack libclang.so into x86-64 sysroot of the SDK"

LICENSE = "CLOSED"
LIC_FILES_CHKSUM = ""

SRC_URI = ""
PV = "1.0"

BBCLASSEXTEND = "native nativesdk"
DEPENDS = "clang-native"

do_install:class-nativesdk () {
    install -d ${D}${libdir}
    cp --no-preserve=ownership  --no-dereference ${RECIPE_SYSROOT_NATIVE}/usr/lib/libclang.so* ${D}${libdir}
}

FILES:${PN} += "${libdir}"

# Disable a lot of QA checks, this is indeed a dirty hack ;-)
INSANE_SKIP:${PN} = "already-stripped dev-elf pkgvarcheck file-rdeps"

Not nice, but gets the job done. A clean solution is still appreciated 😄

yeah this is perhaps not the best, however, can you share what is the contents of images/ folder inside nativesdk-clang build area. I wonder if this file is getting lost during do_install

[Hollingsworth-martin-bl]

yeah this is perhaps not the best, however, can you share what is the contents of images/ folder inside nativesdk-clang build area. I wonder if this file is getting lost during do_install

Sure, so I set up a clean poky build following the README from meta-clang:

git clone https://git.yoctoproject.org/git/poky
cd poky
git clone https://github.com/kraj/meta-clang.git
$ . ./oe-init-build-env

bitbake-layers add-layer ../meta-clang
echo "MACHINE = \"qemuarm64\"" > ./conf/local.conf
echo "CLANGSDK = \"1\"" > ./conf/local.conf

bitbake core-image-minimal -c populate_sdk

Here is the folder content (shortened by removing *.h, *.py and *.a files):

tmp/work/x86_64-nativesdk-pokysdk-linux/nativesdk-clang/18.1.8/image/
└── usr
    └── local
        └── oe-sdk-hardcoded-buildpath
            └── sysroots
                └── x86_64-pokysdk-linux
                    └── usr
                        ├── bin
                        │   ├── amdgpu-arch
                        │   ├── analyze-build
                        │   ├── bugpoint
                        │   ├── c-index-test
                        │   ├── clang -> clang-18
                        │   ├── clang++ -> clang
                        │   ├── clang-18
                        │   ├── clang-apply-replacements
                        │   ├── clang-change-namespace
                        │   ├── clang-check
                        │   ├── clang-cl -> clang
                        │   ├── clang-cpp -> clang
                        │   ├── clang-doc
                        │   ├── clang-extdef-mapping
                        │   ├── clang-format
                        │   ├── clang-include-cleaner
                        │   ├── clang-include-fixer
                        │   ├── clang-linker-wrapper
                        │   ├── clang-move
                        │   ├── clang-offload-bundler
                        │   ├── clang-offload-packager
                        │   ├── clang-pseudo
                        │   ├── clang-pseudo-gen
                        │   ├── clang-query
                        │   ├── clang-refactor
                        │   ├── clang-rename
                        │   ├── clang-reorder-fields
                        │   ├── clang-repl
                        │   ├── clang-scan-deps
                        │   ├── clang-tblgen
                        │   ├── clang-tblgen18.1.8 -> clang-tblgen
                        │   ├── clang-tidy
                        │   ├── clang-tidy-confusable-chars-gen
                        │   ├── clangd
                        │   ├── clangd-indexer
                        │   ├── diagtool
                        │   ├── dsymutil
                        │   ├── find-all-symbols
                        │   ├── git-clang-format
                        │   ├── hmaptool
                        │   ├── intercept-build
                        │   ├── ld.lld -> lld
                        │   ├── ld64.lld -> lld
                        │   ├── llc
                        │   ├── lld
                        │   ├── lld-link -> lld
                        │   ├── lldb
                        │   ├── lldb-argdumper
                        │   ├── lldb-dap
                        │   ├── lldb-instr
                        │   ├── lldb-server
                        │   ├── lli
                        │   ├── llvm-addr2line -> llvm-symbolizer
                        │   ├── llvm-ar
                        │   ├── llvm-as
                        │   ├── llvm-bcanalyzer
                        │   ├── llvm-bitcode-strip -> llvm-objcopy
                        │   ├── llvm-c-test
                        │   ├── llvm-cat
                        │   ├── llvm-cfi-verify
                        │   ├── llvm-config
                        │   ├── llvm-config18.1.8 -> llvm-config
                        │   ├── llvm-cov
                        │   ├── llvm-cvtres
                        │   ├── llvm-cxxdump
                        │   ├── llvm-cxxfilt
                        │   ├── llvm-cxxmap
                        │   ├── llvm-debuginfo-analyzer
                        │   ├── llvm-debuginfod
                        │   ├── llvm-debuginfod-find
                        │   ├── llvm-diff
                        │   ├── llvm-dis
                        │   ├── llvm-dlltool -> llvm-ar
                        │   ├── llvm-dwarfdump
                        │   ├── llvm-dwarfutil
                        │   ├── llvm-dwp
                        │   ├── llvm-exegesis
                        │   ├── llvm-extract
                        │   ├── llvm-gsymutil
                        │   ├── llvm-ifs
                        │   ├── llvm-install-name-tool -> llvm-objcopy
                        │   ├── llvm-jitlink
                        │   ├── llvm-lib -> llvm-ar
                        │   ├── llvm-libtool-darwin
                        │   ├── llvm-link
                        │   ├── llvm-lipo
                        │   ├── llvm-lto
                        │   ├── llvm-lto2
                        │   ├── llvm-mc
                        │   ├── llvm-mca
                        │   ├── llvm-ml
                        │   ├── llvm-modextract
                        │   ├── llvm-mt
                        │   ├── llvm-nm
                        │   ├── llvm-objcopy
                        │   ├── llvm-objdump
                        │   ├── llvm-opt-report
                        │   ├── llvm-otool -> llvm-objdump
                        │   ├── llvm-pdbutil
                        │   ├── llvm-profdata
                        │   ├── llvm-profgen
                        │   ├── llvm-ranlib -> llvm-ar
                        │   ├── llvm-rc
                        │   ├── llvm-readelf -> llvm-readobj
                        │   ├── llvm-readobj
                        │   ├── llvm-readtapi
                        │   ├── llvm-reduce
                        │   ├── llvm-remarkutil
                        │   ├── llvm-rtdyld
                        │   ├── llvm-sim
                        │   ├── llvm-size
                        │   ├── llvm-split
                        │   ├── llvm-stress
                        │   ├── llvm-strings
                        │   ├── llvm-strip -> llvm-objcopy
                        │   ├── llvm-symbolizer
                        │   ├── llvm-tblgen
                        │   ├── llvm-tblgen18.1.8 -> llvm-tblgen
                        │   ├── llvm-tli-checker
                        │   ├── llvm-undname
                        │   ├── llvm-windres -> llvm-rc
                        │   ├── llvm-xray
                        │   ├── modularize
                        │   ├── nvptx-arch
                        │   ├── opt
                        │   ├── pp-trace
                        │   ├── run-clang-tidy
                        │   ├── sancov
                        │   ├── sanstats
                        │   ├── scan-build
                        │   ├── scan-build-py
                        │   ├── scan-view
                        │   ├── verify-uselistorder
                        │   └── wasm-ld -> lld
                        ├── include
                        │   ├── [...]
                        ├── lib
                        │   ├── LLVMgold.so
                        │   ├── clang
                        │   │   └── 18
                        │   │       └── include
                        │   │           ├── [...]
                        │   ├── libLLVM-18.so -> libLLVM.so.18.1
                        │   ├── libLLVM.so -> libLLVM.so.18.1
                        │   ├── libLLVM.so.18.1
                        │   ├── libLLVMAArch64AsmParser.a
                        │   ├── [...]
                        │   ├── liblldb.so -> liblldb.so.18.1
                        │   ├── liblldb.so.18.1 -> liblldb.so.18.1.8
                        │   ├── liblldb.so.18.1.8
                        │   ├── liblldbIntelFeatures.so -> liblldbIntelFeatures.so.18.1
                        │   ├── liblldbIntelFeatures.so.18.1
                        │   ├── libscanbuild
                        │   │   ├── __init__.py
                        │   │   ├── analyze.py
                        │   │   ├── arguments.py
                        │   │   ├── clang.py
                        │   │   ├── compilation.py
                        │   │   ├── intercept.py
                        │   │   ├── report.py
                        │   │   ├── resources
                        │   │   │   ├── scanview.css
                        │   │   │   ├── selectable.js
                        │   │   │   └── sorttable.js
                        │   │   └── shell.py
                        │   └── python3.12
                        │       └── [...]
                        ├── libexec
                        │   ├── analyze-c++
                        │   ├── analyze-cc
                        │   ├── c++-analyzer
                        │   ├── ccc-analyzer
                        │   ├── intercept-c++
                        │   └── intercept-cc
                        └── share
                            ├── clang
                            │   ├── bash-autocomplete.sh
                            │   ├── clang-doc-default-stylesheet.css
                            │   ├── clang-format-bbedit.applescript
                            │   ├── clang-format-diff.py
                            │   ├── clang-format-sublime.py
                            │   ├── clang-format.el
                            │   ├── clang-format.py
                            │   ├── clang-include-fixer.el
                            │   ├── clang-include-fixer.py
                            │   ├── clang-rename.el
                            │   ├── clang-rename.py
                            │   ├── clang-tidy-diff.py
                            │   ├── index.js
                            │   └── run-find-all-symbols.py
                            ├── man
                            │   └── man1
                            │       └── scan-build.1
                            ├── opt-viewer
                            │   ├── opt-diff.py
                            │   ├── opt-stats.py
                            │   ├── opt-viewer.py
                            │   ├── optpmap.py
                            │   ├── optrecord.py
                            │   └── style.css
                            ├── scan-build
                            │   ├── scanview.css
                            │   └── sorttable.js
                            └── scan-view
                                ├── Reporter.py
                                ├── ScanView.py
                                ├── bugcatcher.ico
                                └── startfile.py

256 directories, 4373 files

It seems that the image/ directory contains neither libclang.so nor libclang-cpp.so, while I have verified that the SDK x86-64 sysroot directory contains only libclang-cpp.so

Side note, while above listings are from master branch, I originally found this on langdale branch (clang 15.0.6).

[kraj]

what does [...] in that output mean ? I see it in few places. Is it truncating stuff ?
may be just run find cmd and paste the output

[Hollingsworth-martin-bl]

what does [...] in that output mean ? I see it in few places. Is it truncating stuff ? may be just run find cmd and paste the output

With [...] I marked all removed lines. I removed only folders that contain only *.h *.py or *.a files. The find command would list 256 directories, 4373 files which is a little much for this forum 🤣 Are you searching for something specific? In particular neither libclang.so nor libclang-cpp.so are present in the image/ directory!

[kraj]

@Hollingsworth-martin-bl since libclang is packaged into a package of its own, it needs to be added explicitly in order for it to be included into SDK as SDK host part.

Something like below in local.conf will get it

TOOLCHAIN_HOST_TASK:append = " nativesdk-libclang"

to get libclang.so ( the dev symlink ) into host part of SDK then you also need to add nativesdk-clang-dev to above.

HOWEVER, I wonder why are you needing libraries on SDK host and specifically dev packages e.g. libclang.so, It would make sense if you asked for including the target clang and libclang, since I would expect that normally on SDKs here since they are cross SDKs. So if you clarify your usecase a bit perhaps it can help us to understand the situation a bit better.

[Hollingsworth-martin-bl]

@Hollingsworth-martin-bl since libclang is packaged into a package of its own, it needs to be added explicitly in order for it to be included into SDK as SDK host part.

Something like below in local.conf will get it

TOOLCHAIN_HOST_TASK:append = " nativesdk-libclang"

Can confirm that this works. Thanks for the solution and for helping where I overlooked this obvious part from the clang_git.bb source code 🤦‍♂️

HOWEVER, I wonder why are you needing libraries on SDK host and specifically dev packages e.g. libclang.so, It would make sense if you asked for including the target clang and libclang, since I would expect that normally on SDKs here since they are cross SDKs. So if you clarify your usecase a bit perhaps it can help us to understand the situation a bit better.

My use case is to cross compile a software package written in RUST. The package has a dependency to cargo crate avahi-sys (https://crates.io/crates/avahi-sys) which internally uses crate bindgen (https://crates.io/crates/bindgen/0.69.4). bindgen needs the clang x86-64 binaries present as it generates rust bindings to c code (specific: bindings to avahi), and thus avahi-sys needs to link against x86-64 libcargo.so.

Thank you for offering your help, but I think this usecase goes so far, that I cannot ask your time on that. I wanted to document it however here, that others hitting this roadblock with Yocto SDK and RUST can find it.

[kraj]

You could try to build and add nativesdk version of bindgen-cli recipe, I think the recipe only provides native and target variants as of now so it has to be BBCLASSEXTEND'ed to nativesdk and then nativesdk-bindgen-cli added to TOOLCHAIN_HOST_TASK