feat: add mirror variables

[seatonjiang]

A simple way to implement mirror source replacement

[pajamian]

This is a good PR, but unfortunately it will only affect the initial system package swap. Once migrate2rocky pulls in the rocky-repos package then the remaining distro-sync stage will reference the mirrorlist specified in the .repo files there. In order to make this work for the distro-sync stage then we must allso at least temporarily update those files.

[seatonjiang]

Hi @pajamian , before distro-sync, I added a replace mirror method, And tested successfully.

rocky-tools/migrate2rocky/migrate2rocky.sh

Lines 852 to 858 in b5afd71

	# Replace mirror
	if [ -n "$ROCKY_LINUX_ENV_MIRROR_URL" ]; then
	sed -e 's|^mirrorlist=|#mirrorlist=|g' \
	-e 's|^#baseurl=http://dl.rockylinux.org/$contentdir|baseurl='${ROCKY_LINUX_MIRROR_URL}'|g' \
	-i.bak \
	/etc/yum.repos.d/Rocky-*.repo
	fi

[pajamian]

I was working on something similar but would add options to the migrate2rocky command instead of using a variable like you're doing. I like the idea of using a variable as it's vastly simpler than parsing options, but I'd like to also have the option to not make the changes permanent, so with that options may make more sense. Also please note the following:

Use [[ ... ]] instead of [ ... ], so you would have this on line 853: if [[ $ROCKY_LINUX_ENV_MIRROR_URL ]]; then.

On line 855 you're not double-quoting ${ROCKY_LINUX_MIRROR_URL}. This will result in issues with word-splitting and shell meta-characters. I want to say we should be using dnf config-manager instead of sed here, but that would require us to have another way to populate the baseurl (we would need to get the old one from the file, manipulate it, and add it in with config-manager). That is certainly possible but it will end up requiring awk or sed or an incredibly complex series of bash builtin commands to accomplish, so I'm trying to decide if your way is better than that in its simplicity. At any rate, at least quote the variable:

-e 's|^#baseurl=http://dl.rockylinux.org/$contentdir|baseurl='"$ROCKY_LINUX_MIRROR_URL"'|g' \

...even after quoting it we will still have problems with paths that contain characters special to sed (which is why I'm considering not wanting to use sed here).

...and use -i.migrate2rocky instead of .bak, then if we don't get the flag to make it permanent you simply restore the .migrate2rocky version of the files from the EXIT trap.

[m41kc0d3]

Sounds a little bit like #92 ;)

[pajamian]

Yeah, I was working on something to replace 92, but haven't gotten there yet. Just been way overwhelmed with work.

[pajamian]

If we make sure to limit the files to those in rocky-repos then it will help against certain attacks:

if [[ $ROCKY_MIRROR_URL ]]; then 
    local -a repo_files
    readarray -t repo_files < <(rpm -ql rocky-repos | grep '\.repo$')
    sed -e 's|^mirrorlist=|#mirrorlist=|g' \ 
         -e 's|^#baseurl=http://dl.rockylinux.org/$contentdir|baseurl='"$ROCKY_MIRROR_URL"'|g' \ 
         -i.migrate2rocky \ 
         "${repo_files[@]}"
 fi 

...that in conjunction with an option to set the variable and another option to make it permanent.

[seatonjiang]

Using the variable approach to this problem seems to be a temporary solution until the function of adding options to the migrate2rocky command is completed, thanks to @pajamian for the security optimization.

[pajamian]

Thanks seatonjiang, I'll have a look over it tomorrow and probably add a couple of commits to it myself to make it complete.

[bendem]

By doing this, you uncomment baseurl in files where whole sections are commented, breaking the ini file (i.e. Rocky-Sources.repo

*** /etc/yum.repos.d/Rocky-Sources.repo 2021-12-22 03:25:09.000000000 +0100
--- /proc/self/fd/14    2022-02-16 14:35:54.429740900 +0100
***************
*** 1,58 ****
  # Rocky-Sources.repo

  [baseos-source]
  name=Rocky Linux $releasever - BaseOS - Source
! mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=BaseOS-$releasever-source
! #baseurl=http://dl.rockylinux.org/$contentdir/$releasever/BaseOS/source/tree/
  gpgcheck=1
  enabled=0
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  [appstream-source]
  name=Rocky Linux $releasever - AppStream - Source
! mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=AppStream-$releasever-source
! #baseurl=http://dl.rockylinux.org/$contentdir/$releasever/AppStream/source/tree/
  gpgcheck=1
  enabled=0
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  #[extras-source]
  #name=Rocky Linux $releasever - Extras - Source
  #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=extras-$releasever-source
! #baseurl=http://dl.rockylinux.org/$contentdir/$releasever/extras/source/tree/
  #gpgcheck=1
  #enabled=0
  #gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  #[plus-source]
  #name=Rocky Linux $releasever - Plus - Source
  #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=plus-$releasever-source
! #baseurl=http://dl.rockylinux.org/$contentdir/$releasever/Plus/source/tree/
  #gpgcheck=1
  #enabled=0
  #gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  [ha-source]
  name=Rocky Linux $releasever - High Availability - Source
! mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=HighAvailability-$releasever-source
! #baseurl=http://dl.rockylinux.org/$contentdir/$releasever/HighAvailability/source/tree/
  gpgcheck=1
  enabled=0
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  [powertools-source]
  name=Rocky Linux $releasever - PowerTools - Source
! mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=PowerTools-$releasever-source
! #baseurl=http://dl.rockylinux.org/$contentdir/$releasever/PowerTools/source/tree/
  gpgcheck=1
  enabled=0
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  [resilient-storage-source]
  name=Rocky Linux $releasever - Resilient Storage - Source
! mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=ResilientStorage-$releasever-source
! #baseurl=http://dl.rockylinux.org/$contentdir/$releasever/ResilientStorage/source/tree/
  gpgcheck=1
  enabled=0
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

--- 1,58 ----
  # Rocky-Sources.repo

  [baseos-source]
  name=Rocky Linux $releasever - BaseOS - Source
! #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=BaseOS-$releasever-source
! baseurl=/$releasever/BaseOS/source/tree/
  gpgcheck=1
  enabled=0
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  [appstream-source]
  name=Rocky Linux $releasever - AppStream - Source
! #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=AppStream-$releasever-source
! baseurl=/$releasever/AppStream/source/tree/
  gpgcheck=1
  enabled=0
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  #[extras-source]
  #name=Rocky Linux $releasever - Extras - Source
  #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=extras-$releasever-source
! baseurl=/$releasever/extras/source/tree/
  #gpgcheck=1
  #enabled=0
  #gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  #[plus-source]
  #name=Rocky Linux $releasever - Plus - Source
  #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=plus-$releasever-source
! baseurl=/$releasever/Plus/source/tree/
  #gpgcheck=1
  #enabled=0
  #gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  [ha-source]
  name=Rocky Linux $releasever - High Availability - Source
! #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=HighAvailability-$releasever-source
! baseurl=/$releasever/HighAvailability/source/tree/
  gpgcheck=1
  enabled=0
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  [powertools-source]
  name=Rocky Linux $releasever - PowerTools - Source
! #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=PowerTools-$releasever-source
! baseurl=/$releasever/PowerTools/source/tree/
  gpgcheck=1
  enabled=0
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial

  [resilient-storage-source]
  name=Rocky Linux $releasever - Resilient Storage - Source
! #mirrorlist=https://mirrors.rockylinux.org/mirrorlist?arch=source&repo=ResilientStorage-$releasever-source
! baseurl=/$releasever/ResilientStorage/source/tree/
  gpgcheck=1
  enabled=0
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rockyofficial