Create 'lite' launcher for RP

[mtitov]

Proposed launcher will make a batch submission with RP script as a target application.

• use PMGRLaunchingComponent._prepare_pilot to create job description
• use PilotLauncher to make a submission

(*) PR for #3094 should be in place

[mtitov]

This is how it will look for the user: radical-pilot-launcher rp_app.py -c job_descr.json (where job_descr.json includes project, queue, runtime, etc.)

The same job_descr should be used by RP for the pilot description when it starts within the batch job. Thus, user doesn't need to keep this information in sync between json file and PilotDescription inside rp_app.py, and RP will know which one should be used.

(*) one of the options is to use env variables for the provided config json file, keep them as part of the batch script, and PilotDescription will use them instead of ones that are set (the case when RP determines that it started within the batch job, it can look for a corresponding env variables)

[mtitov]

Launcher to run IMPECCABLE workflow(s) on Frontier

#!/bin/bash

WORK_DIR=$PWD/runinfo
if ! test -d "${WORK_DIR}"; then
  mkdir -p "${WORK_DIR}"
fi

BASE_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]:-$0}"; )" &> /dev/null && pwd 2> /dev/null; )"
CONFIG=$(cat "${BASE_DIR}"/config.json)

ACCOUNT=$(echo "${CONFIG}" | jq -r .run_description.project)
PARTITION=$(echo "${CONFIG}" | jq -r .run_description.queue)
NNODES=$(echo "${CONFIG}" | jq .run_description.nodes)
RUNTIME=$(echo "${CONFIG}" | jq .run_description.walltime)
CORE_SPEC=$(echo "${CONFIG}" | jq .node_description.core_spec)
SMT=$(echo "${CONFIG}" | jq .node_description.smt)
PROJ_HOME_DIR=$(echo "${CONFIG}" | jq -r .env.PROJ_HOME_DIR)

while getopts "m:n:t:" OPTION; do
    case $OPTION in
        m)  MODULE="$OPTARG"  ;;
        n)  NNODES="$OPTARG"  ;;
        t)  RUNTIME="$OPTARG" ;;
        *)  exit 1            ;;
    esac
done

if ! test -z "$MODULE"; then
    OPT_MODULE="-m $MODULE"
fi
OPT_NNODES="-n $NNODES"
OPT_RUNTIME="-t $RUNTIME"


cat > /tmp/$$.slurm <<EOT
#!/bin/sh

#SBATCH --account          $ACCOUNT
#SBATCH --partition        $PARTITION
#SBATCH --nodes            $NNODES
#SBATCH --time             $(echo "$RUNTIME" | awk '{printf "%02d:%02d:00", $1/60, $1%60}')
#SBATCH --job-name         "impeccable.rct"
#SBATCH --chdir            $WORK_DIR
#SBATCH --output           "slurm.out"
#SBATCH --error            "slurm.err"
#SBATCH --core-spec        $CORE_SPEC
#SBATCH --threads-per-core $SMT
#SBATCH --export           NONE

export RADICAL_PILOT_BASE=$WORK_DIR

source "$PROJ_HOME_DIR/miniconda/bin/activate" rct
python \
 $BASE_DIR/impeccable.py $OPT_MODULE $OPT_NNODES $OPT_RUNTIME \
 > $WORK_DIR/run_report.txt 2>&1 </dev/null

EOT

echo "Submitting IMPECCABLE workflow (sbatch script: /tmp/$$.slurm)"
sbatch /tmp/$$.slurm