[Framework] Add error detection to python pip calls

[mreid-tt]

Description

This proposed framework update aims to rectify issues associated with Python-based package installations. Specifically, it addresses a situation where, in the presence of installation errors, the system erroneously reports a successful installation to the user.

Fixes #5926

Checklist

• Build rule all-supported completed successfully
• Any needed documentation is updated/created

Type of change

• Bug fix
• Includes small framework changes

[hgy59]

we should only log, but not exit on error.
the following set_unix_permissions is mandatory for DSM 6.

[mreid-tt]

but the error is already logged now as shown in the referenced issue. the challenge is that it results in a package being installed that is incomplete (see #5926).

[th0ma7]

Suggesting we always refer back to python packaged pip version... such as:

read -r -a PIP_VERSION 2>/dev/null <<< $(python3 -m pip --version)
[ "${PIP_VERSION[1]}" ] && PIP="pip==${PIP_VERSION[1]}" || PIP="pip"
if ! python3 -m pip install --upgrade ${PIP}; then
   echo "Error: Python upgrade failed" 1>&2
fi

This way when ever we update pip in python3x, module installation will also follow.

[th0ma7]

but the error is already logged now as shown in the referenced issue. the challenge is that it results in a package being installed that is incomplete (see #5926).

And I am also tempted to agree, to do exit 1 to confirm package did not installed correctly.

[hgy59]

We should add an error detection in validate_preinst (when ever possible) to prevent the installation in such situations.

If the validate_preinst succeeds but the later installation fails, we must handle such error ("exit 1" ?) in a way, that the DSM Package Center "repair" button (introduced with DSM 7) works as expected. I.e. the package repair will retry the download and if it succeeds the package should be correctly installed.

Since the DSM Developer Guide is not very useful, we need some investigations to make this work.
The only hint in this guide is, that such packages are not able to start.
We have to find out first, whether the repair tries to start the package only, or any of the preinstall and postinstall functions are called again.

[th0ma7]

Agreed but that's outside the scope of this pr. I believe that what was the intent in here is now captured and could be merged.

Additional optimizations you propose should be done in a separate pr.

[mreid-tt]

@th0ma7, is this good to merge now?

[th0ma7]

I believe so. @hgy59 are you fine with comments above and merging?

[th0ma7]

Catching-up... @hgy59 any issues with merging this? (see comments above)

[hgy59]

I would drop the whole pip check/update/downgrade stuff in install_python_virtualenv ()

and add --disable-pip-version-check to pip install command in install_python_wheels ()

[th0ma7]

and add --disable-pip-version-check to pip install command in install_python_wheels ()

forgot, good call, done

[th0ma7]

I would drop the whole pip check/update/downgrade stuff in install_python_virtualenv ()

I've been re-reading the code and indeed it's useless: its been using default python3 from PATH and checking its pip version, thus updating if not matching default python3 package (being called in the first place from PATH).

Now only printing it out.

(untested yet but change is trivial) - feel free to test in the meantime.

[th0ma7]

Tested this morning and working well. It actually expedite the installation a bit.