From 3ba34f61b5f607b27b9b36d4256acffd7e6579ae Mon Sep 17 00:00:00 2001 From: Scott Fryer Date: Mon, 20 May 2024 14:31:55 +0100 Subject: [PATCH 01/12] Add installer packages docs. --- README.md | 4 + docs/Guide_To_The_Linux_Installers.md | 599 ++++++++++++++++++++++++++ 2 files changed, 603 insertions(+) create mode 100644 docs/Guide_To_The_Linux_Installers.md diff --git a/README.md b/README.md index a667ddc0a..4f929c618 100644 --- a/README.md +++ b/README.md @@ -63,6 +63,10 @@ The following documentation describes how to create and publish these Linux inst - under sub-folder "\/\/\/Packages/" - file "temurin-\-jdk-*.\.rpm" exist, e.g: temurin-19-jdk-19.0.1.0.0.10-1.armv7hl.rpm for jdk-19.0.1 hotspot JDK +## More Information + +For more information on updating the linux package installers, additional documentation can be found in the docs subfolder of this project. + ## Troubleshooting Tips Should errors occur during the upload process, and errant files created on Artifactory, any rogue files must be deleted by an artifactory administrator. Each file should be deleted individually, ahead of any errant folders being deleted. This ensures the artifactory metadata gets refreshed correctly. This has been seen in this [issue](https://github.com/adoptium/temurin-build/issues/3618) and on rare other occasions. diff --git a/docs/Guide_To_The_Linux_Installers.md b/docs/Guide_To_The_Linux_Installers.md new file mode 100644 index 000000000..66128131f --- /dev/null +++ b/docs/Guide_To_The_Linux_Installers.md @@ -0,0 +1,599 @@ +# Maintaining The Linux Installer Packaging Process + +This document is intended to provide a guide to the most common works needed to maintain and produce the current set of linux installer packages. + +## 1. Overview Of The Linux Installer Packaging Process + +The linux installer packaging process, current consists of the following elements: + + ### 1.1. Jenkins Linux Packaging Job + + This job which has restricted access, requires several inputs that drive the packaging process, these inputs are: + + + JDK / JRE - Whether you would like the packaging process to produce the installer packages for the JDK or JRE + + Java version option (Which version of Java to produce packages for) + + Current Options : 8, 11, 17, 21, 22 + + Architecture - (Which hardware platform to build for) + + Current Options : + + x86_64 + + armv7hl ( RHEL / Suse Arm 32 Platform Option ) + + armv7l ( Debian Arm 32 Platform Option ) + + aarch64 + + ppc64le + + s390x + + riscv64 + + all (This will attempt to run for all of the available architectures) + + Distribution - (Which Linux distributions to build for) + + Current Options : + + Alpine + + Debian + + RHEL + + Suse + + all (This will attempt to run for all of the available distributions) + +There are a number of restrictions around combinations of architecture and platforms, these are detailed in Appendix A, below, but the [jenkinsfile](https://github.com/adoptium/installer/blob/master/linux/Jenkinsfile) has been coded to exclude these from the build processes, details here are included for reference. + + ### 1.2. Gradle / Docker Package Build & Test Process + +The jenkins job above once triggered, performs a build, using gradle, and a dockerfile to produce, installer packages for the supported versions of each of the 4 distributions that we currently support, ie .apk (Alpine), .deb (Debian) & .rpm (RHEL & Suse). The versions of each specific distribution supported can be viewed in the [jenkinsfile](https://github.com/adoptium/installer/blob/master/linux/Jenkinsfile), by searching for the deb_versions variable, or the distro_Package variable, the table below shows the current supported versions, but this does change on a regular basis. + +Supported Linux Distros: +Distribution Type| Supported Versions +----------|--------- +Apk (Alpine)| All supported version. +Deb (Debian)| Trixie (Debian 13)
Bookworm (Debian 12)
Bullseye (Debian 11)
Buster (Debian 10)
Noble (Ubuntu 24.04)
Jammy (Ubuntu 24.04)
Focal (Ubuntu 20.04)
Bionic (Ubuntu 18.04) +RPM (RHEL)| centos 7
rocky 8
RHEL7 , RHEL8 & RHEL9
Fedora 35, 36, 37, 38 ,39 , 40
Oracle Linux 7 & 8
Amazon Linux 2 +RPM(SUSE) | Opensuse 15.3
Opensuse 15.4
Opensuse 15.5
SLES 12
SLES15 + +As part of this process, the packages are first built, using the appropriate mechanisms, and then a set of unit tests are executed to ensure the builds have completed correctly. + + ### 1.3. Package Signing & Upload + + The final part of the jenkins job is triggered when either/both of the enable enableGpgSigning and uploadPackage options are selected. + + The Gpg signing uses the Adoptium Gpg key to sign the installer packages prior to their upload to the [Jfrog artifactory, hosted at packages.adoptium.net](https://packages.adoptium.net/) + + + +## 2. Preparing The Source Code For Package Builds & Releases + +Prior to building and releasing any packages, the source code that controls the process requires a number of updates. These updates typically happen once all of the linux platforms for a particular Java version have been released and the binaries uploaded to Github. + +Prior to updating the source code, its important to have an understanding of how the source code is structured so that the correct files can be updated. The callout below, highlights the structure of the source code so for example underneath the linux top level directory, a selection of either jdk/jre, followed by the distribution, and then the vendor (temurin in the example), followed by the Java version. + +``` +-- linux + -- jdk / jre + -- alpine / debian / rhel / suse (Distribution) + -- src + -- main + -- packaging + -- temurin / dragonwell / openj9 (Package vendor) + -- 8 / 11 / 17 / 21/ 22 (Java version) +``` + +The following sections will detail the source changes required when a release is done, similar changes are made to both jdk/jre and are also similar between versions. Changes should be made in the specific vendor version being edited. This document assumes a Temurin release is being undertaken. + + ### Per Distribution Guide To Code Changes + +
2.1 Alpine Linux + + To update the source files for each new Alpine release, the ``ÀPKBUILD`` file ( located under _jdk_ / alpine / src / main / packaging /_temurin_ / _21_for example) must be updated in several ways, there are 2 sections of the APKBUILD file that need to be considered.. + +
2.1.1 The header section (example below)
+ + In this section, the key fields that must be amended are as follows + + pkg_ver : This field should be amended to the version tag of the release being processed, e.g , 21.0.3_p9.
+ pkgrel : This field should be reset to 0 for the first release of a package, and then incremented should there be patches or additional rebuilds of this package version.
+ + And additionally were an additional architecture being added (Alpine is currently only supported on x86_64 and Alpine from JDK version 21 onwards), then the arch field would also need appending. + + Once all changes are made in the header section, its time to move on to the footer section. + + ``` + pkgname=temurin-21 + pkgver=21.0.3_p9 + # replace _p1 with _1 + _pkgver=${pkgver/_p/_} + _pkgverplus=${pkgver/_p/+} + _pkgvername=${_pkgverplus/+/%2B} + pkgrel=1 + pkgdesc="Eclipse Temurin 21" + provider_priority=21 + url="https://adoptium.net" + arch="aarch64 x86_64" + ``` + +
2.1.2 The footer section (example below)
+ + In this section, the key fields that must be amended are the architecture specific checksum, as shown below. These checksums are released alongside the binary files uploaded to Github, so these checksums can be obtained from there. + + Again, were another architecture being added an additional element in the case statement below would be required, along with its checksum. + + NB: Note how the case statement uses the values from the arch field in the header section to determine the correct checksum to use. + + +``` +case "$CARCH" in + x86_64) + _arch_sum="8e861638bf6b08c6d5837de6dc929930550928ec5fcc81b9fa7e8296afd0f9c0" ;; + aarch64) + _arch_sum="0f68a9122054149861f6ce9d1b1c176bbe30dd76b36b74c916ba897c12e9d970" + ;; +esac + +sha256sums=" +$_arch_sum OpenJDK21u-jdk_${CARCH/x86_64/x64}_alpine-linux_hotspot_$_pkgver.tar.gz +e9185736dde99a4dc570a645a20407bdb41c1f48dfc34d9c3eb246cf0435a378 HelloWorld.java +22d2ff9757549ebc64e09afd3423f84b5690dcf972cd6535211c07c66d38fab0 TestCryptoLevel.java +9fb00c7b0220de8f3ee2aa398459a37d119f43fd63321530a00b3bb9217dd933 TestECDSA.java +``` + +
+
2.2 Debian Distributions +
+To update the source files for each new Debian release, there are potentially 3 files that must be updated. These 3 files are ```changelog , control & rules``` ( located under _jdk_ / debian / src / main / packaging /_temurin_ / _21_for example) below are the files, and how to update them. + +
2.2.1 The changelog file (example below:)
+ +When actioning a new release, or a patch to an existing release the changelog file, should have an additional section added to the top of the file (the most recent change is at the head). The version number should be updated appropriately (e.g. 21.0.3.0.0+9) and then additionally the final element (-1) should be changed to be -0 for the first release, and then incremented by 1 for each subsequent release. In addition to the version number the release date and timestamps should also be updated. + +``` +temurin-21-jdk (21.0.3.0.0+9-1) STABLE; urgency=medium + + * Eclipse Temurin 21.0.3.0.0+9-1 release. + + -- Eclipse Adoptium Package Maintainers Wed, 17 Apr 2024 00:00:00 +0000 +``` + +
2.2.2 The rules file (example below:)
+ +The second file that should be updated as part of a release is the Debian rules file. This contains the URLs of the binaries published to Github along with their checksum. It contains a pair of values (a tarball url, and a checksum) per architecture, and it is these values used to create the .deb packages. + +Should a new architecture require support, the a new value pair should be added here in addition to the other changes. + +``` +pkg_name = temurin-21-jdk +priority = 2111 +jvm_tools = jar jarsigner java javac javadoc javap jcmd jconsole jdb jdeprscan jdeps jfr jhsdb jimage jinfo jlink jmap jmod jpackage jps jrunscript jshell jstack jstat jstatd jwebserver keytool rmiregistry serialver jexec jspawnhelper +amd64_tarball_url = https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.3%2B9/OpenJDK21U-jdk_x64_linux_hotspot_21.0.3_9.tar.gz +amd64_checksum = fffa52c22d797b715a962e6c8d11ec7d79b90dd819b5bc51d62137ea4b22a340 +arm64_tarball_url = https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.3%2B9/OpenJDK21U-jdk_aarch64_linux_hotspot_21.0.3_9.tar.gz +arm64_checksum = 7d3ab0e8eba95bd682cfda8041c6cb6fa21e09d0d9131316fd7c96c78969de31 +ppc64el_tarball_url = https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.3%2B9/OpenJDK21U-jdk_ppc64le_linux_hotspot_21.0.3_9.tar.gz +ppc64el_checksum = 9a1079d7f0fc72951fdc9a0029e49a15f6ba114683aee626f882ee2c761f1d57 +s390x_tarball_url = https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.3%2B9/OpenJDK21U-jdk_s390x_linux_hotspot_21.0.3_9.tar.gz +s390x_checksum = f57a078d417614e5d78c07c77a6d8a04701058cf692c8e2868d593582be92768 +riscv64_tarball_url = https://github.com/adoptium/temurin21-binaries/releases/download/jdk-21.0.3%2B9/OpenJDK21U-jdk_riscv64_linux_hotspot_21.0.3_9.tar.gz +riscv64_checksum = 246acb1db3ef69a7e3328fa378513b2e606e64710626ae8dd29decc0e525359b +``` + + +
2.2.3 The control file (example below:)
+ +The final file, that typically only requires updates if this is the first release of a new jdk, or alternatively a new architecture is being added. + +In the event of a new architecture being added, the Architecture line should have the new value appended. + +In the event that this is the first release of a new JDK version, then the Provides field should also be updated with the new values as appropriate. + +``` +Source: temurin-21-jdk +Section: java +Priority: optional +Maintainer: Eclipse Adoptium Package Maintainers +Build-Depends: debhelper (>= 11), lsb-release + +Package: temurin-21-jdk +Architecture: amd64 arm64 ppc64el s390x riscv64 +Provides: +java15-sdk-headless, +java16-sdk-headless, +java17-sdk-headless, +java18-sdk-headless, +java19-sdk-headless, +java2-sdk-headless, +java20-sdk-headless, +java21-sdk-headless, +``` +
+ +
2.3 RHEL/Suse Distributions +
+The process for updating the source files for both RHEL & Suse based distributions is identical. The source file that drives the build process for the produced rpm files is typically named like this vendor-version-jdk ```temurin-21-jdk.spec``` ( located under _jdk_ / redhat / src / main / packaging / _temurin_ / _21_for example). Details on how to update/amend the relevant sections of the spec file are below: + +
2.3.1 The header section(example below:)
+ +This section requires updates to the following 2 fields: + +global spec version:
This field should be amended to the FULL version number of the release being processed, e.g , 21.0.3.0.0.9. +global spec release:
This field should be reset to 0 for the first release of a package, and then incremented should there be patches or additional rebuilds of this package version. The first version should be 1. This will ultimately be appended to the final package name, e.g temurin-21-jdk-21.0.3.0.0.9-1. + +In the event that this is a new java major version release, the global priority should also be updated appropriately. + + +``` +%global upstream_version 21.0.3+9 +# Only [A-Za-z0-9.] allowed in version: +# https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/#_upstream_uses_invalid_characters_in_the_version +# also not very intuitive: +# $ rpmdev-vercmp 21.0.0.0.0___21.0.0.0.0+1 +# 21.0.0.0.0___1 == 21.0.0.0.0+35 +%global spec_version 21.0.3.0.0.9 +%global spec_release 1 +%global priority 1161 +``` +
2.3.2 The footer/changelog section(example below:)
+ +The second step in updating a RHEL/Suse spec for in preperation for release, involves adding a new section to the changelog, being sure to put it at the top of the list directly below %changelog and ensuring that the date, time and version numbers are updated appropriately, as shwon in the example below. + +``` +%changelog +* Wed Apr 17 2024 Eclipse Adoptium Package Maintainers 21.0.3.0.0.9-1 +- Eclipse Temurin 21.0.3+9 release. +``` + +
2.3.3 JDK8 / Arm32 Specific Change
+ +If you are updating the spec files for JDK8, for the arm32 platform there is an additional field which should be updated to match the upstream tag used to build the arm32 JDK. + +``` +# jdk8 arm32 has different top directory name https://github.com/adoptium/temurin-build/issues/2795 +%global upstream_version 8u412-b08-aarch32-20240419 +``` + +
2.3.4 Adding a new supported architecture
+ +In the event that a new architecture requires adding to the spec file, there are a number of changes that require adding, firstly a new section should be added to the architecture mapping summary, as detailed blow, the additional architecture should be added as %global vers_arch6 somenew to ALL ifarch section, and both the %global src_num & %global sha_src_num should be incremented to be the next two values in the list, taking the list below as an example, the new values would be :<
%global src_num 10
%global sha_src_num 11 +
+ +The new section should be added prior to this section: + +``` +# Allow for noarch SRPM build +%ifarch noarch +%global src_num 0 +%global sha_src_num 1 +%endif +``` + +and should look something like this: + +``` +%ifarch somenew +%global vers_arch x64 +%global vers_arch2 ppc64le +%global vers_arch3 aarch64 +%global vers_arch4 s390x +%global vers_arch5 riscv64 +%global vers_arch6 somenew +%global src_num 10 +%global sha_src_num 11 +%endif +``` + +
+Full Architectures Section +# Map architecture to the expected value in the download URL; Allow for a +# pre-defined value of vers_arch and use that if it's defined + +%ifarch x86_64 +%global vers_arch x64 +%global vers_arch2 ppc64le +%global vers_arch3 aarch64 +%global vers_arch4 s390x +%global vers_arch5 riscv64 +%global src_num 0 +%global sha_src_num 1 +%endif +%ifarch ppc64le +%global vers_arch x64 +%global vers_arch2 ppc64le +%global vers_arch3 aarch64 +%global vers_arch4 s390x +%global vers_arch5 riscv64 +%global src_num 2 +%global sha_src_num 3 +%endif +%ifarch aarch64 +%global vers_arch x64 +%global vers_arch2 ppc64le +%global vers_arch3 aarch64 +%global vers_arch4 s390x +%global vers_arch5 riscv64 +%global src_num 4 +%global sha_src_num 5 +%endif +%ifarch s390x +%global vers_arch x64 +%global vers_arch2 ppc64le +%global vers_arch3 aarch64 +%global vers_arch4 s390x +%global vers_arch5 riscv64 +%global src_num 6 +%global sha_src_num 7 +%endif +%ifarch riscv64 +%global vers_arch x64 +%global vers_arch2 ppc64le +%global vers_arch3 aarch64 +%global vers_arch4 s390x +%global vers_arch5 riscv64 +%global src_num 8 +%global sha_src_num 9 +%endif +# Allow for noarch SRPM build +%ifarch noarch +%global src_num 0 +%global sha_src_num 1 +%endif +
+
+ +Once the additional section has been added above, the 2 new source files need adding to the list ( sources 10 & 11 in this example), and once again, a new section similar to the previous ones needs adding the section of the file detailed below. Note that the vers_arch6 value represents the somenew architecture value added above and the Source10 & Source11 values have been added. + +The section being added should be similar to this example: +``` +# Sixth architecture (somenew) +Source10: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch6}_linux_hotspot_%{upstream_version_no_plus}.tar.gz +Source11: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch6}_linux_hotspot_%{upstream_version_no_plus}.tar.gz.sha256.txt +``` + +Finally the ExclusiveArch line in the file should also be extended, and should look similar to this: + +``` +ExclusiveArch: x86_64 ppc64le aarch64 s390x riscv64 somenew +``` + +
+Architectures To Process & Source Version Lines Example + +```ExclusiveArch: x86_64 ppc64le aarch64 s390x riscv64 + +# First architecture (x86_64) +Source0: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch}_linux_hotspot_%{upstream_version_no_plus}.tar.gz +Source1: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch}_linux_hotspot_%{upstream_version_no_plus}.tar.gz.sha256.txt +# Second architecture (ppc64le) +Source2: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch2}_linux_hotspot_%{upstream_version_no_plus}.tar.gz +Source3: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch2}_linux_hotspot_%{upstream_version_no_plus}.tar.gz.sha256.txt +# Third architecture (aarch64) +Source4: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch3}_linux_hotspot_%{upstream_version_no_plus}.tar.gz +Source5: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch3}_linux_hotspot_%{upstream_version_no_plus}.tar.gz.sha256.txt +# Fourth architecture (s390x) +Source6: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch4}_linux_hotspot_%{upstream_version_no_plus}.tar.gz +Source7: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch4}_linux_hotspot_%{upstream_version_no_plus}.tar.gz.sha256.txt +# Fifth architecture (riscv64) +Source8: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch5}_linux_hotspot_%{upstream_version_no_plus}.tar.gz +Source9: %{source_url_base}/jdk-%{upstream_version_url}/OpenJDK21U-jdk_%{vers_arch5}_linux_hotspot_%{upstream_version_no_plus}.tar.gz.sha256.txt +``` +
+
+ + +## 3. Adding/Removing An Additional Supported Distribution + +Should a new version of a Debian, RHEL or Suse distribution require adding to the list of distributions we publish packages for or alternatively a distribution requires removing as it is no longer supported, there are a few changes required to the [jenkinsfile](https://github.com/adoptium/installer/blob/master/linux/Jenkinsfile) that drives the build and packaging process. + +The versions of each specific distribution supported can be viewed can be found , by searching for the deb_versions variable (Debian), or the distro_Package variable for (RHEL & Suse). The table below shows the current supported versions, but this does change on a regular basis + +
+Table Of Currently Supported Versions As Of 05/2024 +Supported Linux Distros: +Distribution Type| Supported Versions +----------|--------- +Deb (Debian)| Trixie (Debian 13)
Bookworm (Debian 12)
Bullseye (Debian 11)
Buster (Debian 10)
Noble (Ubuntu 24.04)
Jammy (Ubuntu 24.04)
Focal (Ubuntu 20.04)
Bionic (Ubuntu 18.04) +RPM (RHEL)| centos 7
rocky 8
RHEL7 , RHEL8 & RHEL9
Fedora 35, 36, 37, 38 ,39 , 40
Oracle Linux 7 & 8
Amazon Linux 2 +RPM(SUSE) | Opensuse 15.3
Opensuse 15.4
Opensuse 15.5
SLES 12
SLES15 +

+ +To add an additional RHEL/Suse simply add new entries to relevant array as shown below and the packaging process will automatically upload the packages to the relevant sections of artifactory as part of the release process. +``` +def distro_Package = [ + 'redhat' : [ + 'rpm/centos/7', + 'rpm/rocky/8', + 'rpm/rhel/7', + 'rpm/rhel/8', + 'rpm/rhel/9', + 'rpm/fedora/35', + 'rpm/fedora/36', + 'rpm/fedora/37', + 'rpm/fedora/38', + 'rpm/fedora/39', + 'rpm/fedora/40', + 'rpm/oraclelinux/7', + 'rpm/oraclelinux/8', + 'rpm/amazonlinux/2' + ], + 'suse' : [ + 'rpm/opensuse/15.3', + 'rpm/opensuse/15.4', + 'rpm/opensuse/15.5', + 'rpm/sles/12', + 'rpm/sles/15' + ] +``` +For Debian based distributions a similar process is required, firstly add the distribution into the relevant section of the jenkinsfile shown below. In addition to this addition though, the build script used for creating and testing the packages will also require updates, and additionally the cacerts package, that the JDK & JRE are dependent on will also need updating, this is a seperate process, but must be carried out ahead of publishing the packages. To find out how to do this, please see section 4 of this document. Below is an example of the Debian distributions from the jenkinsfile. + +``` + def deb_versions = [ + "trixie", // Debian/13 + "bookworm", // Debian/12 + "bullseye", // Debian/11 + "buster", // Debian/10 + "noble", // Ubuntu/24.04 (LTS) + "jammy", // Ubuntu/22.04 (LTS) + "focal", // Ubuntu/20.04 (LTS) + "bionic" // Ubuntu/18.04 (LTS) + ] +``` + +In addition to the updates detailed above, it is also important to change the following 4 files to add the new distributions (or remove redundant distributions), in the 2 build scripts located below: + +``` + linux/jdk/debian/src/main/packaging/build.sh + linux/jre/debian/src/main/packaging/build.sh +``` + +the following line should be changed : + +``` +debVersionList="trixie bookworm bullseye buster noble jammy focal bionic" +``` + +And similarly in the following two files + +``` + linux/jdk/debian/src/packageTest/java/packaging/DebianFlavours.java + linux/jre/debian/src/packageTest/java/packaging/DebianFlavours.java +``` + +The array needs to be updated to add or remove distributions as necessary as shown below: +``` + return Stream.of( + Arguments.of("debian", "trixie"), // Debian/13 (testing) + Arguments.of("debian", "bookworm"), // Debian/12 (testing) + Arguments.of("debian", "bullseye"), // Debian/11 (stable) + Arguments.of("debian", "buster"), // Debian/10 (oldstable) + Arguments.of("ubuntu", "noble"), // Ubuntu/24.04 (LTS) + Arguments.of("ubuntu", "jammy"), // Ubuntu/22.04 (LTS) + Arguments.of("ubuntu", "focal"), // Ubuntu/20.04 (LTS) + Arguments.of("ubuntu", "bionic") // Ubuntu/18.04 (LTS) + ); +``` + +## 4. Updating the Adoptium cacerts package + +When adding new Debian based distribution support to the installer packages, it is vital to update the dependent cacerts packages. This is a prerequisite package that the JDK/JRE Debian packages require. Unlike the other installer packages the cacerts package is built and tested by a Github action, when the relevant source code files are changed and added to a PR. Once the PR with the necessary code changes, has built and been tested cleanly, the cacerts package will get automatically uploaded to the package repository when the PR is merged. + +The changes to the source code involve updating the Github action and both the files involved in building the new package, and the files that perform the automated testing. The following sections will detail the changes required. + +### 4.1 Updating the Github action. + +To update the Github action the following file must be updated .github/workflows/cacert-publish.yml. + +Simply add or remove the supported distributions to the debVersionList line as shown in the example below. Note that this uses the "codename" for each Debian & Ubuntu version. + +``` +- name: Upload deb file to Artifactory + if: steps.check-deb.outputs.file_exists == 'false' + run: | + debVersionList=("bookworm" "bullseye" "buster" "jammy" "focal" "bionic") + for debVersion in "${debVersionList[@]}"; do + distroList+="deb.distribution=${debVersion};" + done +``` + +### 4.2 Updating the cacerts package build source. + +There are 2 changes required in the build source files, when creating an update for the cacerts package, the first is to change the following file: +``` +linux/ca-certificates/debian/build.gradle +``` + +Again, new distributions should be added or removed in the array. + +``` +def deb_versions = [ + "trixie", // Debian/13 + "bookworm", // Debian/12 + "bullseye", // Debian/11 + "buster", // Debian/10 + "noble", // Ubuntu/24.04 (LTS) + "jammy", // Ubuntu/22.04 (LTS) + "focal", // Ubuntu/20.04 (LTS) + "bionic" // Ubuntu/18.04 (LTS) +] +``` + +The second change that is required is to the following file: + +``` +linux/ca-certificates/debian/src/main/packaging/debian/changelog +``` +A new section should be added, at the top of the changelog file, and the version number (e.g 1.0.3-1) should be updated appropriately, again for this example, the new version would expect to be 1.0.4-1 , with the x.x.x element being the version number, with the -x element being the release number number. + +``` +adoptium-ca-certificates (1.0.3-1) STABLE; urgency=medium + + * Add Debian Trixie & Ubuntu Noble to the list of supported releases. + + -- Eclipse Adoptium Package Maintainers Thu, 25 April 2024 10:30:30 +0000 +``` + +### 4.3 Updating the cacerts package test source. + +In addition to the previous changes, the automated test source code also needs updating to reflect the new distributions, in the first file: + +``` +linux/ca-certificates/debian/src/packageTest/java/org/adoptium/cacertificates/AptOperationsTest.java +``` + +This file requires that the .contains("Version: 1.0.3-1") line be updated to reflect the new version number added to the changelog above. +``` +result = runShell(container, "apt-cache show adoptium-ca-certificates"); + assertThat(result.getExitCode()).isEqualTo(0); + assertThat(result.getStdout()) + .contains("Package: adoptium-ca-certificates") + .contains("Version: 1.0.3-1") + .contains("Priority: optional") + .contains("Architecture: all") + .contains("Status: install ok installed"); +``` + +The other two files that require changing are detailed below: + +``` +linux/ca-certificates/debian/src/packageTest/java/org/adoptium/cacertificates/ChangesVerificationTest.java +``` + +``` + private static String[] versionsList = { + "trixie", // Debian/13 + "bookworm", // Debian/12 + "bullseye", // Debian/11 + "buster", // Debian/10 + "noble", // Ubuntu/24.04 (LTS) + "jammy", // Ubuntu/22.04 (LTS) + "focal", // Ubuntu/20.04 (LTS) + "bionic", // Ubuntu/18.04 (LTS) + }; +``` + +``` +linux/ca-certificates/debian/src/packageTest/java/org/adoptium/cacertificates/DebianFlavours.java +``` + +``` + return Stream.of( + Arguments.of("debian", "trixie"), // Debian/13 (testing) + Arguments.of("debian", "bookworm"), // Debian/12 (testing) + Arguments.of("debian", "bullseye"), // Debian/11 (stable) + Arguments.of("debian", "buster"), // Debian/10 (oldstable) + Arguments.of("ubuntu", "noble"), // Ubuntu/24.04 (LTS) + Arguments.of("ubuntu", "jammy"), // Ubuntu/22.04 (LTS) + Arguments.of("ubuntu", "focal"), // Ubuntu/20.04 (LTS) + Arguments.of("ubuntu", "bionic") // Ubuntu/18.04 (LTS) + ); +``` + +The change that is required to both these files is to update the array of arguments by adding/removing the new versions as appropriate. + +For a specific example of these changes being made, see this PR : https://github.com/adoptium/installer/pull/868/files + +When the PR is created it will perform the necessary builds and tests, and then once merged, the package should be uploaded to the package repository. + +## Appendices +
+Appendix A Invalid Installer Job Combinations +The following architecture, version and platform combinations are considered invalid, and may error, or produce no output, these are considered invalid and unsupported version, architecture and platform combinations, the installer build process is designed to not produce artifacts for these combinations, but the list is included here as a reference: + +JDK Version |Architecture |Distribution |Reason +----------|---------|----------|-------- +8 | s390x | ALL | s390x is not a supported architecture for JDK8 +8 & 11 | riscv64 | ALL | riscv64 is not a supported architecture for JDKs prior to version 17 +8 , 11 & 17 | aarch64 | Alpine | Alpine linux is only supported on x64 for JDK versions prior to 21 +21 & 22 | s390x, ppc64le, armv7l, armv7hl , riscv64 | Alpine | Alpine linux is only supported on x86_64 & aarch64 for JDK versions later than 21. +21 & 22 | armv7l & armv7hl | All | Arm32 bit is no longer supported for JDKs later than version 21 +
\ No newline at end of file From 66b01996ea13c03d492e49ac65a0d4b90241d439 Mon Sep 17 00:00:00 2001 From: Scott Fryer Date: Thu, 23 May 2024 13:44:11 +0100 Subject: [PATCH 02/12] Update. --- docs/Guide_To_The_Linux_Installers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Guide_To_The_Linux_Installers.md b/docs/Guide_To_The_Linux_Installers.md index 66128131f..bab74c3a2 100644 --- a/docs/Guide_To_The_Linux_Installers.md +++ b/docs/Guide_To_The_Linux_Installers.md @@ -72,7 +72,7 @@ Prior to updating the source code, its important to have an understanding of how -- 8 / 11 / 17 / 21/ 22 (Java version) ``` -The following sections will detail the source changes required when a release is done, similar changes are made to both jdk/jre and are also similar between versions. Changes should be made in the specific vendor version being edited. This document assumes a Temurin release is being undertaken. +The following sections will detail the source changes required when a release is done, similar changes are made to both jdk/jre and are also similar between versions. Changes should be made in the specific vendor version being edited. The examples shown in this document assume a Temurin release is being undertaken. ### Per Distribution Guide To Code Changes From 9c6f1400536b4ceca01121d4c36d6a05dc91d40c Mon Sep 17 00:00:00 2001 From: Scott Fryer Date: Mon, 3 Jun 2024 09:16:15 +0100 Subject: [PATCH 03/12] Add link to guide. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 4f929c618..09cf3f3dd 100644 --- a/README.md +++ b/README.md @@ -65,7 +65,7 @@ The following documentation describes how to create and publish these Linux inst ## More Information -For more information on updating the linux package installers, additional documentation can be found in the docs subfolder of this project. +For more information on updating the linux package installers, additional documentation can be found in the docs subfolder of this project. [See Here](./docs/Guide_To_The_Linux_Installers.md) ## Troubleshooting Tips From fe6b49ebd714d2f8e26cfcdb9caf4fe0ce68213a Mon Sep 17 00:00:00 2001 From: Scott Fryer <60462088+steelhead31@users.noreply.github.com> Date: Mon, 3 Jun 2024 09:16:51 +0100 Subject: [PATCH 04/12] Update docs/Guide_To_The_Linux_Installers.md Co-authored-by: Martijn Verburg --- docs/Guide_To_The_Linux_Installers.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/Guide_To_The_Linux_Installers.md b/docs/Guide_To_The_Linux_Installers.md index bab74c3a2..953a98038 100644 --- a/docs/Guide_To_The_Linux_Installers.md +++ b/docs/Guide_To_The_Linux_Installers.md @@ -4,7 +4,8 @@ This document is intended to provide a guide to the most common works needed to ## 1. Overview Of The Linux Installer Packaging Process -The linux installer packaging process, current consists of the following elements: +The linux installer packaging process, currently consists of the following elements: + ### 1.1. Jenkins Linux Packaging Job From 73e3ae8b7e8e13a0d6ca555784330a8504f4cd32 Mon Sep 17 00:00:00 2001 From: Scott Fryer <60462088+steelhead31@users.noreply.github.com> Date: Mon, 3 Jun 2024 09:16:58 +0100 Subject: [PATCH 05/12] Update docs/Guide_To_The_Linux_Installers.md Co-authored-by: Martijn Verburg --- docs/Guide_To_The_Linux_Installers.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/Guide_To_The_Linux_Installers.md b/docs/Guide_To_The_Linux_Installers.md index 953a98038..135dc1267 100644 --- a/docs/Guide_To_The_Linux_Installers.md +++ b/docs/Guide_To_The_Linux_Installers.md @@ -9,7 +9,8 @@ The linux installer packaging process, currently consists of the following eleme ### 1.1. Jenkins Linux Packaging Job - This job which has restricted access, requires several inputs that drive the packaging process, these inputs are: + This job which has restricted access. It requires several inputs that drive the packaging process: + + JDK / JRE - Whether you would like the packaging process to produce the installer packages for the JDK or JRE + Java version option (Which version of Java to produce packages for) From dc756e9b62902da17cfcc07546577daf77e79aa1 Mon Sep 17 00:00:00 2001 From: Scott Fryer <60462088+steelhead31@users.noreply.github.com> Date: Mon, 3 Jun 2024 09:17:04 +0100 Subject: [PATCH 06/12] Update docs/Guide_To_The_Linux_Installers.md Co-authored-by: Martijn Verburg --- docs/Guide_To_The_Linux_Installers.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/Guide_To_The_Linux_Installers.md b/docs/Guide_To_The_Linux_Installers.md index 135dc1267..33c404478 100644 --- a/docs/Guide_To_The_Linux_Installers.md +++ b/docs/Guide_To_The_Linux_Installers.md @@ -33,7 +33,8 @@ The linux installer packaging process, currently consists of the following eleme + Suse + all (This will attempt to run for all of the available distributions) -There are a number of restrictions around combinations of architecture and platforms, these are detailed in Appendix A, below, but the [jenkinsfile](https://github.com/adoptium/installer/blob/master/linux/Jenkinsfile) has been coded to exclude these from the build processes, details here are included for reference. +There are a number of restrictions around combinations of architecture and platforms detailed in Appendix A below. For our CI/CD pipelines, the [jenkinsfile](https://github.com/adoptium/installer/blob/master/linux/Jenkinsfile) has been coded to exclude these from the build processes, details are included here for reference. + ### 1.2. Gradle / Docker Package Build & Test Process From d80ad61c9c1ba16b70a44db41f13b8106026bb29 Mon Sep 17 00:00:00 2001 From: Scott Fryer <60462088+steelhead31@users.noreply.github.com> Date: Mon, 3 Jun 2024 09:17:11 +0100 Subject: [PATCH 07/12] Update docs/Guide_To_The_Linux_Installers.md Co-authored-by: Martijn Verburg --- docs/Guide_To_The_Linux_Installers.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/Guide_To_The_Linux_Installers.md b/docs/Guide_To_The_Linux_Installers.md index 33c404478..abef8eeff 100644 --- a/docs/Guide_To_The_Linux_Installers.md +++ b/docs/Guide_To_The_Linux_Installers.md @@ -38,7 +38,8 @@ There are a number of restrictions around combinations of architecture and platf ### 1.2. Gradle / Docker Package Build & Test Process -The jenkins job above once triggered, performs a build, using gradle, and a dockerfile to produce, installer packages for the supported versions of each of the 4 distributions that we currently support, ie .apk (Alpine), .deb (Debian) & .rpm (RHEL & Suse). The versions of each specific distribution supported can be viewed in the [jenkinsfile](https://github.com/adoptium/installer/blob/master/linux/Jenkinsfile), by searching for the deb_versions variable, or the distro_Package variable, the table below shows the current supported versions, but this does change on a regular basis. +Once the jenkins job is triggered, it performs a build using Gradle, and a Dockerfile to produce installer packages for the supported versions of each of the 4 distributions that we currently support, i.e., `.apk` (Alpine), `.deb` (Debian) & `.rpm` (RHEL & Suse). The versions of each specific distribution supported can be viewed in the [jenkinsfile](https://github.com/adoptium/installer/blob/master/linux/Jenkinsfile) by searching for the `deb_versions` variable, or the `distro_Package` variable. The table below shows the currently supported versions, but this does change on a regular basis. + Supported Linux Distros: Distribution Type| Supported Versions From 32379a771dc33c734597f5d0dddc63ebec39f24e Mon Sep 17 00:00:00 2001 From: Scott Fryer <60462088+steelhead31@users.noreply.github.com> Date: Mon, 3 Jun 2024 09:17:18 +0100 Subject: [PATCH 08/12] Update docs/Guide_To_The_Linux_Installers.md Co-authored-by: Martijn Verburg --- docs/Guide_To_The_Linux_Installers.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/Guide_To_The_Linux_Installers.md b/docs/Guide_To_The_Linux_Installers.md index abef8eeff..b9566c7b7 100644 --- a/docs/Guide_To_The_Linux_Installers.md +++ b/docs/Guide_To_The_Linux_Installers.md @@ -137,7 +137,8 @@ e9185736dde99a4dc570a645a20407bdb41c1f48dfc34d9c3eb246cf0435a378 HelloWorld.jav
2.2 Debian Distributions
-To update the source files for each new Debian release, there are potentially 3 files that must be updated. These 3 files are ```changelog , control & rules``` ( located under _jdk_ / debian / src / main / packaging /_temurin_ / _21_for example) below are the files, and how to update them. +To update the source files for each new Debian release, there are potentially 3 files that must be updated. These 3 files are ```changelog , control & rules``` ( located under _jdk_ / debian / src / main / packaging /_temurin_ / _21_) below are the files, and how to update them. +
2.2.1 The changelog file (example below:)
From 53c68635d2a2a1f26bf8caf3779fec5ba6606aba Mon Sep 17 00:00:00 2001 From: Scott Fryer <60462088+steelhead31@users.noreply.github.com> Date: Mon, 3 Jun 2024 09:17:28 +0100 Subject: [PATCH 09/12] Update docs/Guide_To_The_Linux_Installers.md Co-authored-by: Martijn Verburg --- docs/Guide_To_The_Linux_Installers.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/Guide_To_The_Linux_Installers.md b/docs/Guide_To_The_Linux_Installers.md index b9566c7b7..cc9a8a143 100644 --- a/docs/Guide_To_The_Linux_Installers.md +++ b/docs/Guide_To_The_Linux_Installers.md @@ -142,7 +142,8 @@ To update the source files for each new Debian release, there are potentially 3
2.2.1 The changelog file (example below:)
-When actioning a new release, or a patch to an existing release the changelog file, should have an additional section added to the top of the file (the most recent change is at the head). The version number should be updated appropriately (e.g. 21.0.3.0.0+9) and then additionally the final element (-1) should be changed to be -0 for the first release, and then incremented by 1 for each subsequent release. In addition to the version number the release date and timestamps should also be updated. +When actioning a new release, or patching to an existing release, the changelog file should have an additional section added to the top of the file (the most recent change is at the head). The version number should be updated appropriately (e.g. 21.0.3.0.0+9) and then additionally the final element (-1) should be changed to be -0 for the first release, and then incremented by 1 for each subsequent release. In addition to the version number the release date and timestamps should also be updated. + ``` temurin-21-jdk (21.0.3.0.0+9-1) STABLE; urgency=medium From bb68affd866f2ae0203ff07b5021a4d5b747be2d Mon Sep 17 00:00:00 2001 From: Scott Fryer <60462088+steelhead31@users.noreply.github.com> Date: Mon, 3 Jun 2024 09:17:35 +0100 Subject: [PATCH 10/12] Update docs/Guide_To_The_Linux_Installers.md Co-authored-by: Martijn Verburg --- docs/Guide_To_The_Linux_Installers.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/Guide_To_The_Linux_Installers.md b/docs/Guide_To_The_Linux_Installers.md index cc9a8a143..727f88d40 100644 --- a/docs/Guide_To_The_Linux_Installers.md +++ b/docs/Guide_To_The_Linux_Installers.md @@ -155,7 +155,8 @@ temurin-21-jdk (21.0.3.0.0+9-1) STABLE; urgency=medium
2.2.2 The rules file (example below:)
-The second file that should be updated as part of a release is the Debian rules file. This contains the URLs of the binaries published to Github along with their checksum. It contains a pair of values (a tarball url, and a checksum) per architecture, and it is these values used to create the .deb packages. +The second file that should be updated as part of a release is the Debian rules file. This contains the URLs of the binaries published to GitHub along with their checksum. It contains a pair of values (a tarball url, and a checksum) per architecture, and it is these values used to create the .deb packages. + Should a new architecture require support, the a new value pair should be added here in addition to the other changes. From 1c0a597f30974225aeba690bdafc35ea214dc42c Mon Sep 17 00:00:00 2001 From: Scott Fryer <60462088+steelhead31@users.noreply.github.com> Date: Mon, 3 Jun 2024 09:17:40 +0100 Subject: [PATCH 11/12] Update docs/Guide_To_The_Linux_Installers.md Co-authored-by: Martijn Verburg --- docs/Guide_To_The_Linux_Installers.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/Guide_To_The_Linux_Installers.md b/docs/Guide_To_The_Linux_Installers.md index 727f88d40..d633ee740 100644 --- a/docs/Guide_To_The_Linux_Installers.md +++ b/docs/Guide_To_The_Linux_Installers.md @@ -176,7 +176,6 @@ riscv64_tarball_url = https://github.com/adoptium/temurin21-binaries/releases/do riscv64_checksum = 246acb1db3ef69a7e3328fa378513b2e606e64710626ae8dd29decc0e525359b ``` -
2.2.3 The control file (example below:)
The final file, that typically only requires updates if this is the first release of a new jdk, or alternatively a new architecture is being added. From 08627c68ff1d6e5daaf8e57aed48d6023c60b87c Mon Sep 17 00:00:00 2001 From: Scott Fryer <60462088+steelhead31@users.noreply.github.com> Date: Mon, 3 Jun 2024 09:17:47 +0100 Subject: [PATCH 12/12] Update docs/Guide_To_The_Linux_Installers.md Co-authored-by: Martijn Verburg --- docs/Guide_To_The_Linux_Installers.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/Guide_To_The_Linux_Installers.md b/docs/Guide_To_The_Linux_Installers.md index d633ee740..2046be792 100644 --- a/docs/Guide_To_The_Linux_Installers.md +++ b/docs/Guide_To_The_Linux_Installers.md @@ -178,7 +178,8 @@ riscv64_checksum = 246acb1db3ef69a7e3328fa378513b2e606e64710626ae8dd29decc0e5253
2.2.3 The control file (example below:)
-The final file, that typically only requires updates if this is the first release of a new jdk, or alternatively a new architecture is being added. +The final file typically only requires updates if this is the first release of a new jdk, or alternatively if a new architecture is being added. + In the event of a new architecture being added, the Architecture line should have the new value appended.