diff --git a/bc_vt_deeplabcut/CHANGELOG.md b/bc_vt_deeplabcut/CHANGELOG.md new file mode 100644 index 0000000..6c2bac4 --- /dev/null +++ b/bc_vt_deeplabcut/CHANGELOG.md @@ -0,0 +1,10 @@ +# Changelog +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) +and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html). + +## 1.0.0 - 2022-02-06 +### Added +- Initial release! + diff --git a/bc_vt_deeplabcut/DevGuide.md b/bc_vt_deeplabcut/DevGuide.md new file mode 100644 index 0000000..b2e94bc --- /dev/null +++ b/bc_vt_deeplabcut/DevGuide.md @@ -0,0 +1,45 @@ + +# Batch Connect - DeepLabCut App + +An example Batch Connect app that launches a DeepLabCut application within a +batch job. + +## Prerequisites + +This Batch Connect app requires the following software be installed on the +**compute nodes** that the batch job is intended to run on (**NOT** the +OnDemand node): + +- [OpenSSL](https://www.openssl.org/) 1.0.1+ (used to hash the Jupyter Notebook + server password) +- [Lmod](https://www.tacc.utexas.edu/research-development/tacc-projects/lmod) + 6.0.1+ or any other `module purge` and `module load ` based CLI + used to load appropriate environments within the batch job before launching + the Jupyter Notebook server. +- [Singularity](https://sylabs.io/guides/3.0/user-guide/installation.html) Singularity is a container platform. It allows you to create and run containers that package up pieces of software in a way that is portable and reproducible. ... Your container is a single file, and you don't have to worry about how to install all the software you need on each different operating system and system. + +### Development Installation + +```bash +cd /home//ondemand/dev +git clone git@github.com:AdvancedResearchComputing/OnDemandApps.git +``` + +### System Installation +```bash +cd /var/www/ood/apps/sys +git clone git@github.com:AdvancedResearchComputing/OnDemandApps.git +``` +> Make sure to give executable permissions for *image file* + + +- Make necessary description changes to **[manifest.yml](./manifest.yml)** +## Contributing + +1. Fork it +2. Create your feature branch (`git checkout -b my-new-feature`) +3. Commit your changes (`git commit -am 'Add some feature'`) +4. Push to the branch (`git push origin my-new-feature`) +5. Create a new Pull Request + +--- \ No newline at end of file diff --git a/bc_vt_deeplabcut/LICENSE.txt b/bc_vt_deeplabcut/LICENSE.txt new file mode 100644 index 0000000..3494185 --- /dev/null +++ b/bc_vt_deeplabcut/LICENSE.txt @@ -0,0 +1,24 @@ +Copyright (c) 2020 Virginia Tech + +Copyright (c) 2016-2018 Ohio Supercomputer Center + +MIT License + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice shall be +included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND +NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE +LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/bc_vt_deeplabcut/README.md b/bc_vt_deeplabcut/README.md new file mode 100644 index 0000000..ab72fba --- /dev/null +++ b/bc_vt_deeplabcut/README.md @@ -0,0 +1,63 @@ +[![GitHub License](https://img.shields.io/badge/license-MIT-green.svg)](https://opensource.org/licenses/MIT) + +# Batch Connect - VT Deeplabcut + +A Batch Connect app designed for [Advanced Research Computing]'s VT OnDemand that launches +within a batch job. For other apps and assistance with Open OnDemand, see [Open OnDemand]. + +## Prerequisites + +This Batch Connect app requires the following software be installed on the +**compute nodes** that the batch job is intended to run on (**NOT** the +OnDemand node): + + +- [COMSOL Multiphysics] 5.1+ +- [Xfce Desktop] 4+ + +For VNC server support: + +- [TurboVNC] 2.1+ +- [websockify] 0.8.0+ + +For hardware rendering support (coming soon ...): + +- [X server] +- [VirtualGL] 2.3+ + +**Optional** software: + +- [Lmod] 6.0.1+ or any other `module purge` and `module load ` based + CLI used to load appropriate environments within the batch job + +[Open OnDemand] http://openondemand.org/ +[Advanced Research Computing] https://arc.vt.edu/ +[DeepLabCut]: https://github.com/DeepLabCut/DeepLabCut +[Xfce Desktop]: https://xfce.org/ +[TurboVNC]: http://www.turbovnc.org/ +[websockify]: https://github.com/novnc/websockify +[X server]: https://www.x.org/ +[VirtualGL]: http://www.virtualgl.org/ +[Lmod]: https://www.tacc.utexas.edu/research-development/tacc-projects/lmod + +## Install + +Use git to clone this app: + +```sh +git clone +cd +``` + +You will not need to do anything beyond this as all necessary assets are +installed. You will also not need to restart this app as it isn't a Passenger +app. + +Again, you do not need to restart the app as it isn't a Passenger app. + +## Contributing, see https://osc.github.io/ood-documentation/master/install-ihpc-apps.html for inspiration. +1. Fork it +2. Create your feature branch (`git checkout -b my-new-feature`) +3. Commit your changes (`git commit -am 'Add some feature'`) +4. Push to the branch (`git push origin my-new-feature`) +5. Create a new Pull Request diff --git a/bc_vt_deeplabcut/form.js b/bc_vt_deeplabcut/form.js new file mode 100644 index 0000000..1ccf300 --- /dev/null +++ b/bc_vt_deeplabcut/form.js @@ -0,0 +1,61 @@ +'use strict' + +/** + * Fix num cores, allowing blanks to remain + */ +function fix_num_cores() { + let node_type_input = $('#batch_connect_session_context_node_type'); + let node_type = node_type_input.val(); + let num_cores_input = $('#num_cores'); + + if(num_cores_input.val() === '') { + return; + } + + if(node_type === 'hugemem') { + set_ppn_owens_hugemem(num_cores_input); + } else { + set_ppn_owens_regular(num_cores_input); + } +} + +/** + * Sets the PPN limits available for Owens hugemem nodes. + * + * hugemem reservations are always assigned the full node + * + * @param {element} num_cores_input The input for num_cores + */ +function set_ppn_owens_hugemem(num_cores_input) { + const NUM_CORES = 48; + num_cores_input.attr('max', NUM_CORES); + num_cores_input.attr('min', NUM_CORES); + num_cores_input.val(NUM_CORES); +} + +/** + * Sets the PPN limits available for non hugemem Owens nodes. + * + * @param {element} num_cores_input The input for num_cores + */ +function set_ppn_owens_regular(num_cores_input) { + const NUM_CORES = 28; + num_cores_input.attr('max', NUM_CORES); + num_cores_input.attr('min', 0); + num_cores_input.val(Math.min(NUM_CORES, num_cores_input.val())); +} + + +/** + * Change the maximum number of cores selected. + */ +function set_node_type_change_handler() { + let node_type_input = $('#batch_connect_session_context_node_type'); + node_type_input.change(node_type_input, fix_num_cores); +} + +$(document).ready(function() { + // Set the max value to be what was set in the last session + fix_num_cores(); + set_node_type_change_handler(); +}); \ No newline at end of file diff --git a/bc_vt_deeplabcut/form.yml.erb b/bc_vt_deeplabcut/form.yml.erb new file mode 100644 index 0000000..8340b90 --- /dev/null +++ b/bc_vt_deeplabcut/form.yml.erb @@ -0,0 +1,86 @@ +# Batch Connect app configuration file +# +--- +cluster: + - 'tinkercliffs' + - 'infer' + +# Define attribute values that aren't meant to be modified by the user within +# the Dashboard form +attributes: + +# specify widgets for user modifiable job settings + bc_queue: + help: | + - To request a GPU enabled queue, preface it with v100_. Example: v100_normal_q, t4_normal_q + + bc_num_hours: + label: Number of hours (min-1, max-48) + required: true + value: 1 + min: 1 + max: 72 + step: 1 + + bc_num_slots: 1 +# label: Number of nodes (currently confined to 1) +# required: true +# value: 1 +# min: 1 +# max: 1 +# step: 1 + + bc_account: + widget: text_field + label: Account + value: "" + help: | + - The allocation you would like to use for SLURM. + + num_cores: + widget: number_field + label: Number of cores per node (min-1, max-128) + value: 1 + min: 1 + max: 128 + step: 1 + + gpu_per_node: + widget: number_field + label: Number of GPUs per node (min-0, max-2) + value: 0 + min: 0 + max: 2 + step: 1 + help: | + - If requesting GPU nodes, you must enter a GPU-enabled Partition above or the job will fail. + + Container: deeplabcut-24Sep2021-gui.sif +# widget: select +# label: "use container" +# help: "This defines the remote DeepLabCut version to run." +# options: +# <%- Dir.glob('/projects/arcsingularity/*deeplabcut*sif').each do |f| -%> +# <%- sif = f.sub('/projects/arcsingularity/','') -%> +# - [ "Container: <%= sif %>", "<%= sif %>" ] +# <%- end -%> + + bc_vnc_resolution: + required: true + +# All of the attributes that make up the Dashboard form (in respective order), +# and made available to the submit configuration file and the template ERB +# files +# +form: + - cluster + - Container + - bc_account + - reservation + - bc_queue + - bc_num_hours + - bc_num_slots + - num_cores + - gpu_per_node + - bc_vnc_resolution + - bc_email_on_started diff --git a/bc_vt_deeplabcut/icon.png b/bc_vt_deeplabcut/icon.png new file mode 100644 index 0000000..d08173a Binary files /dev/null and b/bc_vt_deeplabcut/icon.png differ diff --git a/bc_vt_deeplabcut/manifest.yml b/bc_vt_deeplabcut/manifest.yml new file mode 100644 index 0000000..06ac331 --- /dev/null +++ b/bc_vt_deeplabcut/manifest.yml @@ -0,0 +1,14 @@ +--- +name: DeepLabCut +category: Interactive Apps +role: batch_connect +description: | + This app will launch a [DeepLabCut] in a container on one node. Currently works on [TinkerCliffs] and [Infer]. For issues or requests, please submit a github issue for the [desktop OOD app] or the [desktop container].

**NOTE** this app does not work without installing DeepLabCut in your profile first, so the first time you use this app it will take ~5min to start!! + + [TinkerCliffs]: https://www.arc.vt.edu/TinkerCliffs + [Infer]: https://www.arc.vt.edu/infer + [DragonsTooth]: https://www.arc.vt.edu/dragonstooth + [Cascades]: https://www.arc.vt.edu/cascades + [Huckleberry]: https://www.arc.vt.edu/huckleberry + [desktop OOD app]:https://github.com/AdvancedResearchComputing/ARC-ondemand-apps/tree/master/bc_vt_desktop + [DeepLabCut]: https://github.com/DeepLabCut/DeepLabCut diff --git a/bc_vt_deeplabcut/submit.yml.erb b/bc_vt_deeplabcut/submit.yml.erb new file mode 100644 index 0000000..00826f8 --- /dev/null +++ b/bc_vt_deeplabcut/submit.yml.erb @@ -0,0 +1,26 @@ +# Job submission configuration file +# +--- + +# +# Configure the content of the job script for the batch job here +# @see http://www.rubydoc.info/gems/ood_core/OodCore/BatchConnect/Template +# +batch_connect: + template: "basic" + set_host: "host=$(hostname -s)" + conn_params: + - ondemand_host +# +# Configure the job script submission parameters for the batch job here +# @see http://www.rubydoc.info/gems/ood_core/OodCore/Job/Script +# +script: + copy_environment: true + native: + - "--nodes=<%= bc_num_slots.blank? ? "1" : bc_num_slots.to_i %>" + - "--cpus-per-task=<%= num_cores.blank? ? "1" : num_cores %>" + - "--time=<%= bc_num_hours + ":0:0" %>" + - "--reservation=<%= reservation %>" + - "--account=<%= bc_account %>" + - "--gres=gpu:<%= gpu_per_node.blank? ? "0" : gpu_per_node.to_i %>" diff --git a/bc_vt_deeplabcut/template/after.sh.erb b/bc_vt_deeplabcut/template/after.sh.erb new file mode 100644 index 0000000..c719112 --- /dev/null +++ b/bc_vt_deeplabcut/template/after.sh.erb @@ -0,0 +1,13 @@ +# Wait for the X server to start + +while [ ! -f ./connection.rc ]; do sleep 1; done +source connection.rc + +echo "Waiting for VNC Server to open port ${host}:${port}..." +if wait_until_port_used "${host}:${port}" 2400; then + echo "Discovered VNC Server listening on port ${port}!" +else + echo "Timed out waiting for VNC Server to open port ${port}!" + clean_up 1 +fi +sleep 2 diff --git a/bc_vt_deeplabcut/template/before.sh.erb b/bc_vt_deeplabcut/template/before.sh.erb new file mode 100644 index 0000000..8464a3d --- /dev/null +++ b/bc_vt_deeplabcut/template/before.sh.erb @@ -0,0 +1,21 @@ +echo starting begin.sh +if [ -z ${HOME+x} ]; +then + export HOME=$(echo ~) + source /etc/profile + source /etc/bashrc + source $HOME/.bashrc +fi + +# Export the module function if it exists +[[ $(type -t module) == "function" ]] && export -f module +hostname +pwd + +module reset +module load containers/singularity + +singularity run --cleanenv --nv \ + --bind `pwd`:/cwd,$TMPDIR:/TMPDIR,/projects,/work/$USER,/cm \ + /projects/arcsingularity/ubuntu-xfce-v0.2.0.sif -d "Desktop-$(hostname)" & + diff --git a/bc_vt_deeplabcut/template/script.sh.erb b/bc_vt_deeplabcut/template/script.sh.erb new file mode 100644 index 0000000..54d2434 --- /dev/null +++ b/bc_vt_deeplabcut/template/script.sh.erb @@ -0,0 +1,30 @@ +#!/bin/bash -l + +while [ ! -f ./connection.rc ]; do sleep 1; done + +cat connection.rc +source connection.rc + +env > env.txt + +pwd +#unset XDG_RUNTIME_DIR + +module list + +echo running on `hostname` +echo starting singularity +echo running on ${host}:${port} + +module load cuDNN/8.0.4.30-CUDA-11.1.1 +## fix up singularity +unset SINGULARITY_TMPDIR +## this seems to be needed on a100s, not infer +export SINGULARITYENV_LD_LIBRARY_PATH=$LD_LIBRARY_PATH + +singularity run --nv --env DISPLAY=$DISPLAY --writable-tmpfs --bind=/apps,/cm,$TMPDIR:$TMPDIR,/projects:/projects,/work,`pwd`:/data \ + /projects/arcsingularity/<%= context.Container %> + +sleep 6000 +wait $! + diff --git a/bc_vt_deeplabcut/view.html.erb b/bc_vt_deeplabcut/view.html.erb new file mode 100644 index 0000000..214f774 --- /dev/null +++ b/bc_vt_deeplabcut/view.html.erb @@ -0,0 +1,9 @@ +

+ + + + + +