Merge remote-tracking branch 'origin/develop' into 533-run-ci-tests-o…

…n-fedora

.github/actions/unit-tests/action.yml

@@ -17,16 +17,16 @@ runs:
   steps:
     - name: Install researcher dependencies
       run: | 
+        source ~/.bashrc
+        conda clean -i -y
         ./scripts/configure_conda researcher node
         ./scripts/fedbiomed_run network
       shell: bash -l {0}
 
     - name: Run unit tests 
       run: |
-          PYTHONPATH=${PYTHONPATH:-$PWD} conda run -n fedbiomed-researcher nosetests -w ${{ inputs.test-dir }} \
-          --cover-xml --cover-erase --with-coverage --cover-package=fedbiomed --with-xunit -v --process-restartworker \
-          -e test_round_0
-      shell: bash -l {0}
+          PYTHONPATH=${PYTHONPATH:-$PWD} conda run -n fedbiomed-researcher pytest -v --cov=fedbiomed --cov-report term --cov-report xml:${{ inputs.test-dir }}/coverage.xml ${{ inputs.test-dir }}
+      shell: bash  -l {0}
 
     - name: Get coverage rate
       id: coverage-rate

.gitignore

@@ -177,6 +177,9 @@ Thumbs.db #thumbnail cache on Windows
 node_modules
 .pnp
 .pnp.js
+gui/ui/.yarn
+gui/ui/.pnp.cjs
+gui/ui/.pnp.loader.mjs
 
 # testing
 coverage

CHANGELOG.md

@@ -30,7 +30,7 @@
 
 ## 2023-02-08 version 4.2
 
-- add support for docker-compose v.2 file syntax
+- add support for `docker compose` v.2 file syntax
 - fix model weights computation occurring during aggregation, by sending dataset sample size from node to researcher
 - fix GUI regression failure, after merging MP-SPDZ certificate generation - such issue was freezing some web browsers
 - fix incoherent tag handling: make explicit the way datasets are tagged on nodes

README.md

@@ -22,22 +22,22 @@ According to our coding rules, the develop branch is usable, tests and tutorials
 
 ## Install and run in development environment
 
-Fed-BioMed is developed under Linux Fedora, should be easily ported to other Linux distributions.
+Fed-BioMed is developped under Linux Fedora & Ubuntu, should be easily ported to other Linux distributions.
 It runs also smoothly on macOSX (mostly tested on macOSX 12: Monterey).
 
 This README.md file provide a quick start/installation guide for Linux.
 
 Full installation instruction are also available at: https://fedbiomed.org/latest/tutorials/installation/0-basic-software-installation/
 
-An installation guide is also provided for Windows10, which relies on WSL2: https://fedbiomed.org/latest/user-guide/installation/windows-installation/
+An installation guide is also provided for Windows11, which relies on WSL2: https://fedbiomed.org/latest/user-guide/installation/windows-installation/
 
 
 ### Prerequisites :
 
 To ensure fedbiomed will work fine, you need to install before :
 
 * docker
-* docker-compose
+* docker compose v2 (aka docker compose plugin)
 * conda
 
 ### clone repo

RELEASE.md

@@ -78,7 +78,7 @@ git pull -p
   - or checkout the `release/$RELEASE_TAG` branch and push it again to re-create on the remote
     ```bash
     git checkout release/$RELEASE_TAG
-    git push origin release/$RELEASE_TAG
+    git push
     ```
 - in github create a pull request for `release/$RELEASE_TAG` to `develop`
   * one can auto-assign the PR, and doesn't need a review for this PR
@@ -196,7 +196,7 @@ Release principle: follow the [gitflow](https://www.atlassian.com/git/tutorials/
 
     ```bash
     git checkout hotfix/$HOTFIX_NAME
-    git push origin hotfix/$HOTFIX_NAME
+    git push
     ```
 
 - in github create a pull request for `hotfix/$HOTFIX_NAME` to `develop`

docs/developer/ci.md

@@ -39,7 +39,7 @@ CI pipeline currently contains :
 
 To view CI test output and logs:
 
-- view the pull request in gitlab (select `Pull requests` in top bar, then select your pull request).
+- view the pull request in github (select `Pull requests` in top bar, then select your pull request).
 - click on the `Checks` at the top bar of the pull request and select the `Check` that you want to display.
 - Click on the jobs to see its console output. 
 

docs/developer/definition-of-done.md

@@ -13,7 +13,7 @@ The Definition of Done is a set of items that must be completed and quality meas
 
 ## Review of the code 
 
-The reviewer can question any aspect of the increment in coherence with [Usage and Tools](./usage_and_tools.md#merge-request), exchange with the developer (good practice : leave a gitlab trace of the exchanges), approve it or not.
+The reviewer can question any aspect of the increment in coherence with [Usage and Tools](./usage_and_tools.md#merge-request), exchange with the developer (good practice : leave a github trace of the exchanges), approve it or not.
 
 - Be specific in the pull request about what to review (critical code or properties of the code).
 - Coding style: inspire from PEP-8.

docs/tutorials/installation/0-basic-software-installation.md

@@ -19,23 +19,23 @@ This tutorial gives steps for installing Fed-BioMed components (network, node, r
 
 Fed-BioMed is developed and tested under up to date version of :
 
-* **Linux Fedora**, should also work or be easily ported under most Linux distributions (Ubuntu, etc.)
+* **Linux Ubuntu and Fedora**, should also work or be easily ported under most Linux distributions
 * **MacOS**
 
-Check specific guidelines for installation on [Windows 10](../../user-guide/installation/windows-installation.md).
+Check specific guidelines for installation on [Windows 11](../../user-guide/installation/windows-installation.md).
 
 
 ## Software packages
 
  The following packages are required for Fed-BioMed :
 
  * [`docker`](https://docs.docker.com)
- * [`docker-compose`](https://docs.docker.com/compose)
+ * [`docker compose` v2](https://docs.docker.com/compose): don't confuse it with the obsolete `docker-compose` v1
  * [`conda`](https://conda.io)
  * `git`
 
 
-### Install docker and docker-compose
+### Install `docker` and `docker compose`
 
 #### Linux Fedora
 
@@ -62,20 +62,21 @@ Check with the account used to run Fed-BioMed that docker is up and can be used
 $ docker run hello-world
 ```
 
-Install docker-compose and git :
+Install `docker compose` and `git` :
 ```
-$ sudo dnf install -y docker-compose git
+$ sudo dnf install -y docker-compose-plugin git
 ```
 
 #### MacOS
 
-Install docker and docker-compose choosing one of the available options for example :
+Install `docker` and `docker compose` choosing one of the available options for example :
 
 * official full [Docker Desktop](https://docs.docker.com/desktop/mac/install/) installation process, please check product license
 * your favorite third party package manager for example :
-    * macports provides [docker](https://ports.macports.org/port/docker/) [docker-compose](https://ports.macports.org/port/docker-compose/) and [git](https://ports.macports.org/port/git/) ports
-    * homebrew provides [docker](https://formulae.brew.sh/formula/docker) [docker-compose](https://formulae.brew.sh/formula/docker-compose) and [git](https://formulae.brew.sh/formula/git) formulae
-
+    * macports provides [docker](https://ports.macports.org/port/docker/) and [git](https://ports.macports.org/port/git/) ports
+    * homebrew provides [docker](https://formulae.brew.sh/formula/docker) and [git](https://formulae.brew.sh/formula/git) formulae
+    * don't use the `docker-compose` v1 from macports or homebrew !
+    * for `docker compose` v2, adapt the [manual plugin install procedure](https://docs.docker.com/compose/install/linux/#install-the-plugin-manually) by picking the [proper binary for your hardware](https://github.com/docker/compose/releases)
 
 Check with the account used to run Fed-BioMed docker is up and can be used by the current account without error :
 
@@ -85,7 +86,7 @@ $ docker run hello-world
 
 #### Other
 
-Connect under an account with administrator privileges, install [`docker`](https://docs.docker.com/engine/install), ensure it is started and give docker privilege for the account used for running Fed-BioMed. Also install [`docker-compose`](https://docs.docker.com/compose/install/) and `git`
+Connect under an account with administrator privileges, install [`docker`](https://docs.docker.com/engine/install), ensure it is started and give docker privilege for the account used for running Fed-BioMed. Also install [`docker compose` v2](https://docs.docker.com/compose/install/) and `git`
 
 Check with the account used to run Fed-BioMed docker is up and can be used by the current account without error :
 

docs/tutorials/medical/medical-image-segmentation-unet-library.ipynb

@@ -38,6 +38,7 @@
     "\n",
     "```shell\n",
     "mkdir -p ${FEDBIOMED_DIR}/notebooks/data\n",
+    "pip install tqdm\n",
     "download_and_split_ixi.py -f ${FEDBIOMED_DIR}\n",
     "```\n",
     "\n",

docs/user-guide/deployment/deployment-vpn-node2.md

@@ -34,13 +34,13 @@ For each node, choose a **unique** node tag (eg: *NODE2TAG* in this example) tha
 
     ```bash
     [user@server $] cd ${FEDBIOMED_DIR}/envs/vpn/docker
-    [user@server $] docker-compose exec vpnserver bash -ci 'python ./vpn/bin/configure_peer.py genconf node NODE2TAG'
+    [user@server $] docker compose exec vpnserver bash -ci 'python ./vpn/bin/configure_peer.py genconf node NODE2TAG'
     ```
 
     The configuration file is now available on the server side in path `${FEDBIOMED_DIR}/envs/vpn/docker/vpnserver/run_mounts/config/config_peers/node/NODE2TAG/config.env` or with command :
 
     ```bash
-    [user@server $] docker-compose exec vpnserver cat /config/config_peers/node/NODE2TAG/config.env
+    [user@server $] docker compose exec vpnserver cat /config/config_peers/node/NODE2TAG/config.env
     ```
 
 * copy the configuration file from the server side **to the node side** via a secure channel, to path `/tmp/config2.env` on the node.
@@ -58,13 +58,13 @@ For each node, choose a **unique** node tag (eg: *NODE2TAG* in this example) tha
 * start `node2` container
 
     ```bash
-    [user@node $] docker-compose up -d node2
+    [user@node $] docker compose up -d node2
     ```
 
 * retrieve the `node2`'s publickey
 
     ```bash
-    [user@node $] docker-compose exec node2 wg show wg0 public-key | tr -d '\r' >/tmp/publickey2-nodeside
+    [user@node $] docker compose exec node2 wg show wg0 public-key | tr -d '\r' >/tmp/publickey2-nodeside
     ```
 
 * copy the public key from the node side **to the server side** via a secure channel (see above), to path `/tmp/publickey2-serverside` on the server.
@@ -73,7 +73,7 @@ For each node, choose a **unique** node tag (eg: *NODE2TAG* in this example) tha
 
     ```bash
     [user@server $] cd ${FEDBIOMED_DIR}/envs/vpn/docker
-    [user@server $] docker-compose exec vpnserver bash -ci "python ./vpn/bin/configure_peer.py add node NODE2TAG $(cat /tmp/publickey2-serverside)"
+    [user@server $] docker compose exec vpnserver bash -ci "python ./vpn/bin/configure_peer.py add node NODE2TAG $(cat /tmp/publickey2-serverside)"
     ```
 
 * check containers running on the node side
@@ -101,7 +101,7 @@ For each node, choose a **unique** node tag (eg: *NODE2TAG* in this example) tha
 * do initial node configuration
 
     ```bash
-    [user@node $] docker-compose exec -u $(id -u) node2 bash -ci 'export FORCE_SECURE_AGGREGATION='${FORCE_SECURE_AGGREGATION}'&& export MPSPDZ_IP=$VPN_IP && export MPSPDZ_PORT=14002 && export MQTT_BROKER=10.220.0.2 && export MQTT_BROKER_PORT=1883 && export UPLOADS_URL="http://10.220.0.3:8000/upload/" && export PYTHONPATH=/fedbiomed && export FEDBIOMED_NO_RESET=1 && eval "$(conda shell.bash hook)" && conda activate fedbiomed-node && ENABLE_TRAINING_PLAN_APPROVAL=True ALLOW_DEFAULT_TRAINING_PLANS=True ./scripts/fedbiomed_run node configuration create'
+    [user@node $] docker compose exec -u $(id -u) node2 bash -ci 'export FORCE_SECURE_AGGREGATION='${FORCE_SECURE_AGGREGATION}'&& export MPSPDZ_IP=$VPN_IP && export MPSPDZ_PORT=14002 && export MQTT_BROKER=10.220.0.2 && export MQTT_BROKER_PORT=1883 && export UPLOADS_URL="http://10.220.0.3:8000/upload/" && export PYTHONPATH=/fedbiomed && export FEDBIOMED_NO_RESET=1 && eval "$(conda shell.bash hook)" && conda activate fedbiomed-node && ENABLE_TRAINING_PLAN_APPROVAL=True ALLOW_DEFAULT_TRAINING_PLANS=True ./scripts/fedbiomed_run node configuration create'
     ```
 
 
@@ -110,7 +110,7 @@ Optionally launch the node GUI :
 * start `gui2` container
 
     ```bash
-    [user@node $] docker-compose up -d gui2
+    [user@node $] docker compose up -d gui2
     ```
 
 * check containers running on the node side
@@ -144,7 +144,7 @@ Setup the node by sharing datasets and by launching the Fed-BioMed node:
     * connect to the container
 
         ```bash
-        [user@node $] docker-compose exec -u $(id -u) node2 bash -ci 'export MPSPDZ_IP=$VPN_IP && export MPSPDZ_PORT=14002 && export MQTT_BROKER=10.220.0.2 && export MQTT_BROKER_PORT=1883 && export UPLOADS_URL="http://10.220.0.3:8000/upload/" && export PYTHONPATH=/fedbiomed && export FEDBIOMED_NO_RESET=1 && eval "$(conda shell.bash hook)" && conda activate fedbiomed-node && bash'
+        [user@node $] docker compose exec -u $(id -u) node2 bash -ci 'export MPSPDZ_IP=$VPN_IP && export MPSPDZ_PORT=14002 && export MQTT_BROKER=10.220.0.2 && export MQTT_BROKER_PORT=1883 && export UPLOADS_URL="http://10.220.0.3:8000/upload/" && export PYTHONPATH=/fedbiomed && export FEDBIOMED_NO_RESET=1 && eval "$(conda shell.bash hook)" && conda activate fedbiomed-node && bash'
         ```
 
     * start the Fed-BioMed node, for example in background: