Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance AOCC EasyBlock to correctly pass GCC toolchain and compiler driver #3480

Merged

Conversation

Thyre
Copy link
Contributor

@Thyre Thyre commented Oct 14, 2024

Recent AOCC versions saw changes in how the driver for compilation is detected. The AOCC EasyBlock uses wrappers right now, where the original compiler name is replaced with a wrapper calling [compiler].orig with the GCC toolchain.

However, with AOCC 4.2.0 and AOCC 5.0.0, I noticed that this breaks building C++ and Fortran codes. For C++, libstdc++ is not linked anymore. For Fortran, several libraries are missing.
To fix this, two things are implemented:

  1. For clang and clang++, switch to the newer config files, which pass the argument. This is more robust than the wrapper script.
  2. For flang, update the wrapper to override the flang.orig name with flang. This lets compilation work normally.

To ensure that everything works, checks for the GCC toolchain and compilation were added. The toolchain checks were taken from #3373, where similar things are checked.

@Thyre
Copy link
Contributor Author

Thyre commented Oct 14, 2024

@SebastianAchilles, can you take a look at these changes? You've developed most of the EasyBlock.
@Crivella I reused parts of your LLVM PR (#3373) for the config file generation and sanity check. Is this okay for you? I can add you as a co-author if wanted.

@Thyre Thyre force-pushed the enhance-aocc-clang-config-files branch 2 times, most recently from 6e1beb2 to 336eae9 Compare October 14, 2024 09:26
@Thyre
Copy link
Contributor Author

Thyre commented Oct 14, 2024

Test report by @Thyre

Overview of tested easyconfigs (in order)

  • SUCCESS AOCC-4.0.0-GCCcore-12.2.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
Linux - Linux EndeavourOS UNKNOWN, x86_64, AMD Ryzen 7 7800X3D 8-Core Processor, 1 x NVIDIA NVIDIA GeForce RTX 3070, 560.35.03, Python 3.12.7
See https://gist.github.com/Thyre/ddd69e55f9c2eb4951bc608eb712414e for a full test report.

@Thyre
Copy link
Contributor Author

Thyre commented Oct 14, 2024

See easybuilders/easybuild-easyconfigs#21629 (comment) for AOCC 5.0.0

@Crivella
Copy link
Contributor

@Thyre Fine by me, whatever you prefer.
Maybe if this are functions meant/needed to be used across multiple easyblocks i could try to generalize them so they can just be imported and used with the appropriate version checks

@Thyre
Copy link
Contributor Author

Thyre commented Oct 14, 2024

@Thyre Fine by me, whatever you prefer. Maybe if this are functions meant/needed to be used across multiple easyblocks i could try to generalize them so they can just be imported and used with the appropriate version checks

@Crivella I think it would be a good idea, as similar sanity checks might be useful for other LLVM-based compilers (like AOCC, AOMP, oneAPI).

@Thyre Thyre force-pushed the enhance-aocc-clang-config-files branch from 13497ab to 1b2bb61 Compare October 14, 2024 12:37
@Thyre
Copy link
Contributor Author

Thyre commented Oct 14, 2024

Test report by @Thyre

Overview of tested easyconfigs (in order)

  • SUCCESS AOCC-4.0.0-GCCcore-12.2.0.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
Linux - Linux EndeavourOS UNKNOWN, x86_64, AMD Ryzen 7 7800X3D 8-Core Processor, 1 x NVIDIA NVIDIA GeForce RTX 3070, 560.35.03, Python 3.12.7
See https://gist.github.com/Thyre/27176ff279dcc36509c6a16434627367 for a full test report.

@Thyre Thyre changed the title Enhance AOCC EasyBlock to use config files for GCC toolchain when required Enhance AOCC EasyBlock to correctly pass GCC toolchain and compiler driver Oct 14, 2024
@SebastianAchilles
Copy link
Member

@boegelbot please test @ jsc-zen3
EB_ARGS="AOCC-3.2.0-GCCcore-11.3.0.eb AOCC-4.0.0-GCCcore-12.3.0.eb"

@boegelbot
Copy link

@SebastianAchilles: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3480 EB_ARGS="AOCC-3.2.0-GCCcore-11.3.0.eb AOCC-4.0.0-GCCcore-12.3.0.eb" EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3480 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 5071

Test results coming soon (I hope)...

- notification for comment with ID 2411244077 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS AOCC-3.2.0-GCCcore-11.3.0.eb
  • SUCCESS AOCC-4.0.0-GCCcore-12.3.0.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/17e753953e6d2a19c348c13ec30d56d5 for a full test report.

Thyre and others added 3 commits October 15, 2024 09:24
…uired

With AOCC 4.2.0 and 5.0.0, it was observed that wrappers for `clang++` and
`flang` will cause compile errors since some libraries are not linked anymore.
To work around this, create config files for `clang` and `clang++` instead.
Since `flang` doesn't understand config files yet, keep the old behavior.

Also add a sanity check to ensure that the correct toolchain is selected.

Signed-off-by: Jan André Reuter <[email protected]>
Co-authored-by: crivella <[email protected]>
Signed-off-by: Jan André Reuter <[email protected]>
While the previous commit fixed the observed issues for `clang++`, `flang`
continued to be broken, as the driver did not detect a Fortran compiler due
to the renaming. To work around this, call the compiler via `exec` to fix
the naming, which causes the driver to work correctly.

Signed-off-by: Jan André Reuter <[email protected]>
Signed-off-by: Jan André Reuter <[email protected]>
@Thyre Thyre force-pushed the enhance-aocc-clang-config-files branch from 1b2bb61 to 4085c51 Compare October 15, 2024 07:25
@SebastianAchilles
Copy link
Member

@boegelbot please test @ jsc-zen3
EB_ARGS="AOCC-3.2.0-GCCcore-11.3.0.eb AOCC-4.0.0-GCCcore-12.3.0.eb"

@boegelbot
Copy link

@SebastianAchilles: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=3480 EB_ARGS="AOCC-3.2.0-GCCcore-11.3.0.eb AOCC-4.0.0-GCCcore-12.3.0.eb" EB_REPO=easybuild-easyblocks EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_3480 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

  • exit code: 0
  • output:
Submitted batch job 5074

Test results coming soon (I hope)...

- notification for comment with ID 2413144605 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

@boegelbot
Copy link

Test report by @boegelbot

Overview of tested easyconfigs (in order)

  • SUCCESS AOCC-3.2.0-GCCcore-11.3.0.eb
  • SUCCESS AOCC-4.0.0-GCCcore-12.3.0.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/53f19feaee328b967d001b1f44065dc8 for a full test report.

@Thyre
Copy link
Contributor Author

Thyre commented Oct 15, 2024

The PR should be ready from my side. I only moved the wrapper script definition back to its initial place to stay consistent.

@SebastianAchilles
Copy link
Member

@Thyre Fine by me, whatever you prefer. Maybe if this are functions meant/needed to be used across multiple easyblocks i could try to generalize them so they can just be imported and used with the appropriate version checks

@Crivella I think it would be a good idea, as similar sanity checks might be useful for other LLVM-based compilers (like AOCC, AOMP, oneAPI).

I think this is a good idea as well. I would suggest that we merge this PR as is to be able to use the newer AOCC. If we want to generalize the features, we could do that in a follow-up PR.

Copy link
Member

@SebastianAchilles SebastianAchilles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@SebastianAchilles
Copy link
Member

Going in, thanks @Thyre!

@SebastianAchilles SebastianAchilles merged commit 3a6cb1a into easybuilders:develop Oct 15, 2024
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants