Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial support for splitting var as a separated partition #1422

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions meta-lmp-base/classes/lmp.bbclass
Original file line number Diff line number Diff line change
Expand Up @@ -125,10 +125,18 @@ IMAGE_CMD:ota:append () {
if [ "${APP_PRELOAD_WITHIN_OE_BUILD}" = "1" ]; then
preload_apps
fi

# Split content /var into a separated folder so it can be consumed by WKS separately
if [ "${OSTREE_SPLIT_VAR}" = "1" ]; then
rm -rf ${OTA_VAR}
mv ${OTA_SYSROOT}/ostree/deploy/${OSTREE_OSNAME}/var ${OTA_VAR}
mkdir -p ${OTA_SYSROOT}/ostree/deploy/${OSTREE_OSNAME}/var
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using mv will fail if we run this task more than one time because on the second one the ${OSTREE_OSNAME}/var is clean. It should be better to copy all the files with cp -a and we can drop the mkdir.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to have the content in ${OTA_SYSROOT}/ostree/deploy/${OSTREE_OSNAME}/var after this code.

But I agree we may need to move the content only once like

if [ "${OSTREE_SPLIT_VAR}" = "1" ] && ! -d ${OTA_VAR}; then
		mv ${OTA_SYSROOT}/ostree/deploy/${OSTREE_OSNAME}/var ${OTA_VAR}
		mkdir -p ${OTA_SYSROOT}/ostree/deploy/${OSTREE_OSNAME}/var
fi

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OTA_SYSROOT is already defined for dir and cleandirs:

do_image_ota[dirs] = "${OTA_SYSROOT}"
do_image_ota[cleandirs] = "${OTA_SYSROOT}"

So a re-run wouldn't be an issue here.

mkdir might not necessarily be needed indeed, will check.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, best to let var in deploy as that is expected by ostree when creating .ostree-selabeled during deploy.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hadn't even taken into account possible dependencies that have more impact than running more than once

fi
}
OTA_BOOT = "${WORKDIR}/ota-boot"
do_image_ota[dirs] += "${OTA_BOOT}"
do_image_ota[cleandirs] += "${OTA_BOOT}"
OTA_VAR = "${WORKDIR}/ota-var"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be more logical and readable to place setting vars above using them?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Funny enough most tasks end up defining the variables and cleaning actions after the task itself is defined (if you see the other tasks in this class). We can probably have a separated commit to move things around if needed (here I would prefer to just add the new variable to avoid a larger diff).

do_image_ota[dirs] += "${OTA_BOOT} ${OTA_VAR}"
do_image_ota[cleandirs] += "${OTA_BOOT} ${OTA_VAR}"

# Adapted from oe_mkext234fs in image_types.bbclass
oe_mkotaespfs() {
Expand Down
1 change: 1 addition & 0 deletions meta-lmp-base/conf/distro/lmp.conf
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ OSTREE_KERNEL ?= "${@oe.utils.conditional('KERNEL_IMAGETYPE', 'fitImage', '${KER
OSTREE_KERNEL_ARGS_COMMON ?= "root=LABEL=otaroot rootfstype=ext4"
OSTREE_KERNEL_ARGS ?= "${OSTREE_KERNEL_ARGS_COMMON}"
OSTREE_SPLIT_BOOT ?= "0"
OSTREE_SPLIT_VAR ?= "0"
OSTREE_LOADER_LINK ?= "1"
OSTREE_DEPLOY_USR_OSTREE_BOOT ?= "0"
DISTRO_FEATURES:append = " sota"
Expand Down
11 changes: 11 additions & 0 deletions meta-lmp-base/recipes-core/base-files/base-files_%.bbappend
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,17 @@ FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"

SRC_URI += "file://tmpfiles.conf"

# fstab handling when var is in a separated partition (required by ostree)
OSTREE_SPLIT_VAR ?= "0"

do_compile:append () {
if ${@bb.utils.contains('OSTREE_SPLIT_VAR', '1', 'true', 'false', d)}; then
if ! grep -q '/var[[:blank:]]' ${WORKDIR}/fstab; then
echo "LABEL=var /var ext4 defaults 0 1" >> ${WORKDIR}/fstab
fi
fi
}

do_install:append () {
if ${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'true', 'false', d)}; then
install -D -m 0644 ${WORKDIR}/tmpfiles.conf ${D}${nonarch_libdir}/tmpfiles.d/${PN}.conf
Expand Down
Original file line number Diff line number Diff line change
@@ -1,27 +1,10 @@
#!/bin/sh
# Copyright (c) Fathi Boudra <[email protected]>
# Copyright (c) 2019-2023 Foundries.IO
#
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#
# SPDX-License-Identifier: BSD-2-Clause

# must exit on error
set -e
Expand Down Expand Up @@ -51,7 +34,6 @@ fi
# get the partition number and type
INFO=$(udevadm info --query=property --name=${ROOT_DEVICE})
PART_ENTRY_NUMBER=$(echo "${INFO}" | grep '^ID_PART_ENTRY_NUMBER=' | cut -d'=' -f2)
PART_ENTRY_NAME=$(echo "${INFO}" | { grep '^ID_PART_ENTRY_NAME=' || true; } | cut -d'=' -f2)

# in case the root device is not on a partitioned media
if [ "x$PART_ENTRY_NUMBER" = "x" ]; then
Expand All @@ -64,37 +46,40 @@ PART_TABLE_TYPE=$(echo "${INFO}" | grep '^ID_PART_TABLE_TYPE=' | cut -d'=' -f2)
DEVICE=$(udevadm info --query=path --name=${ROOT_DEVICE} | awk -F'/' '{print $(NF-1)}')
DEVICE="/dev/${DEVICE}"

# only manipulate partition table if rootfs is the last partition
LAST_PART=$(partx -s -o nr ${DEVICE} | tail -n1)
if [ ${PART_ENTRY_NUMBER} -eq ${LAST_PART} ]; then
SIZE=$(blockdev --getsz ${DEVICE})
TYPE="p\n"
if [ "$PART_TABLE_TYPE" = "gpt" ]; then
${SGDISK} -e ${DEVICE}
SIZE=$(($SIZE - 33)) # the GPT end of disk is 34 sectors
TYPE=""
fi

# Use fdisk to repartition
# fdisk uses a ram image of the parition table until a write is performed
# so the sequence is to delete the partition and recreate it at the same
# starting point but making it the size of the available disk.
#
END=$((${SIZE} - 1))
PARTOF=$(echo "${INFO}" | grep '^ID_PART_ENTRY_OFFSET=' | cut -d'=' -f2)
echo -e "d\n${PART_ENTRY_NUMBER}\nn\n${TYPE}${PART_ENTRY_NUMBER}\n${PARTOF}\n${END}\nw\n" | ${FDISK} ${DEVICE}
# only manipulate the last partition from the same disk used by the rootfs
LAST_PART=$(partx -s -o nr ${DEVICE} | tail -n1 | awk -F' ' '{print $1}')
RESIZE_PART="${DEVICE}${LAST_PART}"
INFO=$(udevadm info --query=property --name=${RESIZE_PART})
PART_ENTRY_NUMBER=$(echo "${INFO}" | grep '^ID_PART_ENTRY_NUMBER=' | cut -d'=' -f2)
PART_ENTRY_NAME=$(echo "${INFO}" | { grep '^ID_PART_ENTRY_NAME=' || true; } | cut -d'=' -f2)

if [ -n "${PART_ENTRY_NAME}" ]; then
${SGDISK} --change-name=${PART_ENTRY_NUMBER}:"${PART_ENTRY_NAME}" ${DEVICE}
fi
${PARTX} -u ${DEVICE}
# resize the partition first and after the actual file system
SIZE=$(blockdev --getsz ${DEVICE})
TYPE="p\n"
if [ "$PART_TABLE_TYPE" = "gpt" ]; then
${SGDISK} -e ${DEVICE}
SIZE=$(($SIZE - 33)) # the GPT end of disk is 34 sectors
TYPE=""
fi
# Use fdisk to repartition
# fdisk uses a ram image of the parition table until a write is performed
# so the sequence is to delete the partition and recreate it at the same
# starting point but making it the size of the available disk.
#
END=$((${SIZE} - 1))
PARTOF=$(echo "${INFO}" | grep '^ID_PART_ENTRY_OFFSET=' | cut -d'=' -f2)
echo -e "d\n${PART_ENTRY_NUMBER}\nn\n${TYPE}${PART_ENTRY_NUMBER}\n${PARTOF}\n${END}\nw\n" | ${FDISK} ${DEVICE}
if [ -n "${PART_ENTRY_NAME}" ]; then
${SGDISK} --change-name=${PART_ENTRY_NUMBER}:"${PART_ENTRY_NAME}" ${DEVICE}
fi
${PARTX} -u ${DEVICE}

if [ -n "${DM_NAME}" ] && cryptsetup isLuks ${ROOT_DEVICE}; then
if command -v cryptsetup > /dev/null && cryptsetup isLuks ${RESIZE_PART}; then
# OP-TEE: use TEE Identity for pkcs11 authentication
export CKTEEC_LOGIN_TYPE=user
DM_NAME=`lsblk -n --list -o NAME,TYPE ${RESIZE_PART} | awk '$2=="crypt" {print $1}'`
cryptsetup resize --token-only ${DM_NAME}
${RESIZE2FS} "/dev/mapper/${DM_NAME}"
else
${RESIZE2FS} "${ROOT_DEVICE}"
${RESIZE2FS} "${RESIZE_PART}"
fi
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[Unit]
Description=Resize root filesystem to fit available disk space
Description=Resize last partition and filesystem to fit available disk space
Wants=systemd-udevd.service systemd-udev-trigger.service
After=systemd-remount-fs.service systemd-udevd.service luks-reencryption.service

Expand Down
9 changes: 9 additions & 0 deletions meta-lmp-base/wic/efidisk-split-var-sota.wks.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# short-description: Create an OTA-enabled EFI disk image
# long-description: Creates an OTA-enabled EFI disk image that the user
# can directly dd to boot media.

part /boot --source bootimg-sota-efi --sourceparams="loader=${EFI_PROVIDER}" --rootfs-dir=${WORKDIR}/ota-boot --ondisk sda --active --align 1024 --use-uuid ${OSTREE_WKS_EFI_SIZE} --label boot
part / --source otaimage --ondisk sda --part-name=otaroot --fstype=ext4 --align 1024 --use-uuid
part /var --source rootfs --ondisk sda --rootfs-dir=${WORKDIR}/ota-var --fstype=ext4 --part-name var --label var --part-type 0x8310 --align 4096 --use-uuid

bootloader --source bootimg-sota-efi --timeout=1 --ptable gpt
2 changes: 1 addition & 1 deletion meta-lmp-bsp/conf/machine/include/lmp-machine-custom.inc
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ EFI_PROVIDER:intel-corei7-64 = "systemd-boot"
OSTREE_SPLIT_BOOT:intel-corei7-64 = "1"
OSTREE_LOADER_LINK:intel-corei7-64 = "0"
KERNEL_CLASSES:intel-corei7-64 = " kernel-lmp-efi "
WKS_FILE:intel-corei7-64:sota ?= "efidisk-sota.wks.in"
WKS_FILE:intel-corei7-64:sota ?= "${@bb.utils.contains('OSTREE_SPLIT_VAR', '1', 'efidisk-split-var-sota.wks.in', 'efidisk-sota.wks.in', d)}"
WKS_FILE_DEPENDS:append:intel-corei7-64 = " ${INITRD_IMAGE_LIVE} efitools"
WKS_FILE_DEPENDS_BOOTLOADERS:remove:intel-corei7-64 = "grub-efi"
## wic-based installer requires image to be available via IMAGE_BOOT_FILES
Expand Down