From 4ab92a1d92c603247cd16a32cc9c33e9eaf779b5 Mon Sep 17 00:00:00 2001 From: Antonin MAIRE <26769675+maireiphc@users.noreply.github.com> Date: Thu, 11 Aug 2022 16:38:21 +0200 Subject: [PATCH 1/3] alibuild recipe (reference) : requirements Adding extra details about the logic I understood for the fields : requires, build_requires, runtime_requires (after mail exchanges with Adrian Sevcenco, Giulio Eulisse + some local tests on one alidist recipe (lhapdf.sh). To be revised by Giulio to make sure that this is correct. --- docs/reference.markdown | 31 ++++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/docs/reference.markdown b/docs/reference.markdown index 60c0755a..5554e0d7 100644 --- a/docs/reference.markdown +++ b/docs/reference.markdown @@ -88,7 +88,7 @@ The following entries are optional in the header: software to be built. This can be a tag name, a branch name or a commit hash. - `env`: dictionary whose key-value pairs are environment variables to be set - after the recipe is built. The values are interpreted as the contents of a + _after_ the recipe is built. The values are interpreted as the contents of a double-quoted shell string, so you can reference other environment variables as `$VARIABLE`, which will be substituted each time another recipe is built. For example: @@ -104,7 +104,7 @@ The following entries are optional in the header: `export VARIABLE=value` in the recipe body. - `prepend_path`: dictionary whose key-value pairs are an environment variable name and a path to be prepended to it, as it happens in `LD_LIBRARY_PATH`. - This happens only after the package declaring the `prepend_path` in question + This happens only _after_ the package declaring the `prepend_path` in question is built, so it is not available in the same recipe (just like variables declared using `env`). You can append multiple paths to a single variable by specifying a list too, *e.g.*: @@ -120,7 +120,9 @@ The following entries are optional in the header: - `append_path`: same as `prepend_path` but paths are appended rather than prepended. Like `append_path` and `env`, this **does not** affect the environment of the current recipe. - - `requires`: a list of run-time dependencies for the package, *e.g.*: + - `requires`: a list of dependencies for the package, + that will be needed **both** at _built-time_ and _run-time_, + *e.g.*: ```yaml package: AliRoot @@ -129,7 +131,9 @@ The following entries are optional in the header: ... ``` - The specified dependencies will be built before building the given package. + The specified dependencies will be built before building the given package; + the "required" corresponding binaries, libraries and environment variables + will be made available (by loading the dependency module) before the current built starts. You can specify platform-specific dependencies by appending `:` to the dependency name. Such a regular expression will be matched against the architecture provided via `--architecture`, and if it does not match, the @@ -143,15 +147,20 @@ The following entries are optional in the header: will make sure that `IgProf` is only built on platforms whose name does not begin with `osx`. - - `build_requires`: a list of build-time dependencies for the package. Like + - `build_requires`: a list of build-time _only_ dependencies for the package. Like `requires`, these packages will be built before the current package is built. - - Packages in this list are marked specially in the dependency graph + + Packages in this list are marked specially in the dependency graph produced by `aliDeps`. Other tools treat these packages differently from `requires`: for instance, RPMs produced for a package won't depend on its `build_requires`, and `alibuild-generate-module` won't pull in build requirements' modulefiles. + + - `runtime_requires` : a list of run-time _only_ dependencies for the package. + The corresponding module(s) will be loaded together with the current one. + FIXME : how it will appear in aliDeps ? + - `force_rebuild`: set it to `true` to force re-running the build recipe every time you invoke alibuild on it. - `prefer_system_check`: a script which is used to determine whether @@ -203,10 +212,10 @@ Some environment variables are made available to the script. write files outside this directory. - `BUILDROOT`: it contains `BUILDDIR` and the log file for the build. - `SOURCEDIR`: where the sources are cloned. - - `REQUIRES`: space-separated list of all dependencies, both runtime and build - only. - - `BUILD_REQUIRES`: space-separated list of all build dependencies, not needed - at runtime. + - `REQUIRES`: space-separated list of all dependencies, needed at **both** + runtime and built-time. + - `BUILD_REQUIRES`: space-separated list of all build dependencies, that are needed + at built-time only but not at runtime. - `RUNTIME_REQUIRES`: space-separated list of all runtime dependencies only. For each dependency already built, the corresponding environment file is loaded. From c3eb50065aea070dd7b9536744da410b261580fe Mon Sep 17 00:00:00 2001 From: Antonin MAIRE <26769675+maireiphc@users.noreply.github.com> Date: Fri, 12 Aug 2022 17:15:13 +0200 Subject: [PATCH 2/3] aliBuild/docs - recipe : correction about "requires" Apparently it is not the full module of the dependency which is loaded before the build requiring it starts, it is the build environment only. --- docs/reference.markdown | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/reference.markdown b/docs/reference.markdown index 5554e0d7..7b2cad0e 100644 --- a/docs/reference.markdown +++ b/docs/reference.markdown @@ -133,7 +133,9 @@ The following entries are optional in the header: The specified dependencies will be built before building the given package; the "required" corresponding binaries, libraries and environment variables - will be made available (by loading the dependency module) before the current built starts. + will be made available (by loading the build environment of the dependency) + before the current built starts. + You can specify platform-specific dependencies by appending `:` to the dependency name. Such a regular expression will be matched against the architecture provided via `--architecture`, and if it does not match, the From f18e0f9fefc2fdd712014a46df2955b77a09c4cf Mon Sep 17 00:00:00 2001 From: Antonin MAIRE <26769675+maireiphc@users.noreply.github.com> Date: Mon, 22 Aug 2022 15:29:19 +0200 Subject: [PATCH 3/3] aliBuild/docs - Table of content (reference.md) Update of the TOC to match the section content (build env, runtime env, ...) --- docs/reference.markdown | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/reference.markdown b/docs/reference.markdown index 7b2cad0e..bd0ea331 100644 --- a/docs/reference.markdown +++ b/docs/reference.markdown @@ -9,6 +9,11 @@ layout: main 2. [The body](#the-body) 3. [Defaults, common requirements for builds](#defaults) 2. [Relocation](#relocation) +3. [Build environment](#build-environment) +4. [Runtime environment: module file](#runtime-environment) + 1. [Creating a module file](#creating-a-module-file) + 2. [Generating (partially) a module file: alibuild-recipe-tools](#generating-in-part-a-module-file) + ## Recipe formats @@ -370,6 +375,8 @@ load the build environment via [direnv](https://direnv.net), *e.g.* for easy ## Runtime environment +### Creating a module file + Runtime environment is usually provided via [environment modules](https://modules.readthedocs.io/en/latest/). @@ -414,6 +421,9 @@ Please keep in mind the following recommendation when writing the modulefile: make sure that only dependencies which were actually built via `aliBuild` will be included in the modulefile. + +### Generating in part a module file + It's also now possible to generate automatically the initial part of the modulefile, up to the `# Our environment` line, by using the `alibuild-recipe-tools` helper scripts. In order to do this you need to add