Setup MultiVec2Text in LUMI

1. add the following into .profile and .bashrc files and source them afterwards:

export SBATCH_ACCOUNT=project_xxx
export SALLOC_ACCOUNT=project_xxx

export HF_HOME="/scratch/project_xxx/.cache"
export HF_DATASETS_CACHE="/scratch/project_xxx/.cache/datasets"
export DATASET_CACHE_PATH="/scratch/project_xxx/vec2text_private/.cache"

2. Install the pytorch module

export EBU_USER_PREFIX=/scratch/project_xxx/

module load LUMI partition/container EasyBuild-user
eb PyTorch-2.2.0-rocm-5.6.1-python-3.10-singularity-20240315.eb
module load PyTorch/2.2.0-rocm-5.6.1-python-3.10-singularity-20240315

3. Afterwards, everytime to use the pytorch singularity container:

module load LUMI
module --ignore_cache load PyTorch/2.2.0-rocm-5.6.1-python-3.10-singularity-20240315

interactive mode:

singularity shell $SIF

4. Setup the MultiVec2text

4.1 install requirements: pip install -r requirements.txt

4.2 setup wandb

wandb login

input the wandb token

4.3 setup accelerate

accelerate config

multi-gpu, num_machines=1, no meixed_precision, etc.