builder2

builder2, a.k.a builder of builder is a script that allows pain-less builds of build tool containers.
The script takes a JSON file, called toolchain decriptor file, that contains all the required information to download, compile and install every single supported tool, and simply builds each tool following those rules and install them in the target directory.
Every installation generates a installation summary file, that contains all the metadata about each tool (well knows paths, version, custom envrionment variables that may be added, Java certificates keystores paths, autogenerated Conan profiles paths, etc.) that is used for other purposes such as:

• Shell bootstrapping: Loads the user configured shell with the environments variables of the summary (default paths if needed, JAVA_HOME environment variable, etc.) and adds custom CA certificates, if provided, to the system truststore and recognised tools (now only Java jdk is supported).
• Source: Allows bootstrapping a shell with all the environment variables and certificates loaded in environments in where is not possible to use the bootstrap command, and a simple bash source command can be used. This command generates a script that can be sourced by bash to bootstrap the environment. Specially useful in Gitlab Kubernetes executor, that overrides the build container entrypoint skipping the default entrypoint of each container that just calls the bootstrap command.
• Load certificates: Loads the given custom CA certificates to the system truststore and recognised tools (now only Java jdk is supported).
• Query: Returns the value of some useful installation params as plain text through stdout. Specially useful in CI/CD scripting to avoid hardcoding tool paths, Conan profiles (that may change by tool version and release type, etc...). Actual supported query parameters are:
  • Wellknown Path: Some type of installers (clang, gcc and java ones) recognises and store some recognised paths if present. Those paths can be queried by using the query command for a specific tool.
  • Conan:n For clang and gcc installations Conan profiles can be automatically generated for Debug and Release targets. This command allows retreiving the path of each profile for a given tool.
  • Version: Simply returns the tools version.
  • Triplet: For C/C++ compilers this command returns the triplet of the tool (arch and os information).
  • Path: Just tells the path in where the tool is installed.

Installation examples

These are some examples of toolchain metadata files that generate C/C++ and Java builder2 based installations:

• C/C++
• JDK

Docker

The script is meant to be used in contenerized environments. Here it an example on how to build a Dockerfile for C/CPP tools:

FROM registry.internal.pablintino.net/tools/builder2:v1.0.4  # Base image that contains the builder2


ARG BUILDER_INSTALLATION=/tools
ARG BUILDER_CUSTOM_CERTS=/etc/ssl/custom
ARG BUILDER_MAX_CPU_COUNT=10
ARG BUILDER_TIMEOUT_MULTIPLIER=100
ENV BUILDER_INSTALLATION $BUILDER_INSTALLATION
ENV BUILDER_CUSTOM_CERTS $BUILDER_CUSTOM_CERTS


COPY entrypoint /opt/builder2/entrypoint
COPY cpp-toolchain-metadata.json /opt/builder2/toolchain-metadata.json # Copy the toolchain metadata file (see examples)


RUN builder2 install -f /opt/builder2/toolchain-metadata.json  \       # Perform installation in this layer
    -j $BUILDER_MAX_CPU_COUNT \
    -t $BUILDER_TIMEOUT_MULTIPLIER \
    -d $BUILDER_INSTALLATION

ENTRYPOINT ["/opt/builder2/entrypoint"]
CMD ["/bin/bash"]

Th entrypoint of the container can be now replaced by the builder2 bootstrap command, that way, every container will run on a bootstrapped shell with all the needed environment variables and certificates loaded.

#! /bin/bash

builder2 bootstrap --quiet --certs "$BUILDER_CUSTOM_CERTS" -- "$@"